CIS 700: Integrated Intelligence for Robotics

Fall 2016, University of Pennsylvania

Instructor: Eric Eaton, Ph.D.

Schedule
Course Description
Prerequisites
Course Format
Syllabus
Instructor
Policies

Schedule

The following schedule lists the tentative dates, presenters, and readings for each topic in this course. Topics and readings are subject to change. We may end up shifting topics slightly earlier or later in the schedule, depending on unanticipated events. This is especially the case for topics colored in blue, which have very tentative dates at this point and are more likely to shift than topics coming up soon.

Wk	Date	Topic	Reading	Comments
1
1	W 8/31	Introduction to Course & Project (Prof. Eaton) Introduction to ROS (Guest speaker: Chris Clingerman)
2	M 9/5	Labor Day (no class)
2	W 9/7	Discussion: Project Planning	E. Eaton, et al. Design of a low-cost platform for autonomous mobile service robots. In IJCAI-16 Workshop on Autonomous Mobile Service Robots. RoboCup@Home Website RoboCup@Home 2016 Rulebook (skim only)
3	M 9/12	Architectures for Integrating Perception, Learning and Control: case studies of the Stanford STAIR & CMU CoBots (Yeung)	J. Biswas and M. Veloso. Localization and Navigation of the CoBots Over Long-term Deployments. International Journal of Robotics Research 32(14):1679-1694, 2013. M. Quigley, et al. STAIR: Hardware and Software Architecture. AAAI 2007 Robotics Workshop. (skim only)	Other resources: The Stanford STAIR Project website: http://stair.stanford.edu/ The CMU CoBots Project website: http://www.cs.cmu.edu/~coral/projects/cobot/ (Be sure to watch the videos!)
3	W 9/14	Discussion: Project Planning
4	M 9/19	Architectures for Integrating Perception, Learning and Control: the DARPA Robotic Challenge (Gilbert)	Johnson, et al. Team IHMC's Lessons Learned from the DARPA Robotics Challenge Trials. Journal of Field Robotics 32(2): 192-208, 2015. Stentz, et al. CHIMP, the CMU Highly Intelligent Mobile Platform. Journal of Field Robotics 32(2): 209-228, 2015. (read the article, but skim the hardware details) Murphy. Meta-analysis of Autonomy at the DARPA Robotics Challenge Trials. Journal of Field Robotics 32(2): 189-191, 2015. (skim)
4	W 9/21	Discussion		Getting Started Due
5	M 9/26	Architectures for Integrating Perception, Learning and Control: the DARPA Grand & Urban Challenges	Urmson, et al. Autonomous driving in urban environments: Boss and the urban challenge. Journal of Field Robotics 25(8): 425-466, 2008. Montemerlo, et al. Junior: The Stanford entry in the Urban Challenge. Journal of Field Robotics 25(9): 569-597, 2008. (Skim, but focus on similarities/differences with Urmson et al.)
5	W 9/28	Architectures for Integrating Perception, Learning, and Control: Layered Learning (Baucom)	Leottau, Ruiz-del-Solar, MacAlpine, and Stone. A Study of Layered Learning Strategies Applied to Individual Behaviors in Robot Soccer. RoboCup-2015: Robot Soccer World Cup XIX, Lecture Notes in Artificial Intelligence, Springer Verlag, Berlin, 2016.
6	M 10/3	Computer Vision for Object Recognition & Scene Understanding (Prof. Eaton)	Torralba et al. Using the Forest to See the Trees: Exploiting Context for Visual Object Detection and Localization. CACM 2009. [related code]	Focus Group proposals due Other resources: Lazebnik et al. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. CVPR 2006. [15 scenes dataset] [libpmk] [Matlab]
6	W 10/5	Deep Learning for Object Recognition & Scene Understanding (Rostami)	Razavian, Azizpour, Sullivan, Carlsson. CNN Features off-the-shelf: an Astounding Baseline for Recognition. arXiv:1403.6382, 2015 Farabet, Couprie, Najman, LeCun. Learning Hierarchical Features for Scene Labeling. PAMI 2013.	Other resources: Talk by Yann LeCun [video] [slides] (optional, but really good if you'd like an overview of convolutional nets) Slides by Jitendra Malik (skim)
7	M 10/10	Deep Reinforcement Learning (Thompson)	Levine, Finn, Darrell, Abbeel. End-to-End Training of Deep Visuomotor Policies. ICRA 2015. [project website & videos] Mnih et al. Human-level control through deep reinforcement learning. Nature 518: 529-533, 2015.	Project Team proposals due
7	W 10/12	Discussion on Project Designs
8	M 10/17	Inverse Reinforcement Learning (Brooks & Mendez)	Abbeel & Ng. Apprenticeship Learning via Inverse Reinforcement Learning. ICML 2004. Coates, Abbeel, & Ng. Learning for Control from Multiple Demonstrations. ICML 2008. [slides]	Other resources: Stanford Helicopter project website
8	W 10/19	Multi-task and Transfer Learning (Rostami & Willett)	Sutton et al. Horde: A Scalable Real-time Architecture for Learning Knowledge from Unsupervised Sensorimotor Interaction. AAMAS 2011. White, Modayil, & Sutton. Scaling Lifelong Off-Policy Learning. EWRL 2012.	Project Design Due
9	M 10/24	Lifelong Machine Learning (Prof. Eaton)	P. Ruvolo & E. Eaton. ELLA: An Efficient Lifelong Learning Algorithm. ICML 2013. (Read the paper, but skim the theory sections; ignore the proofs) H. Bou Ammar, et al. Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning. IJCAI 2015.	Other resources: H. Bou Ammar, et al. Online Multi-Task Learning for Policy Gradient Methods. ICML 2014.
9	W 10/26	RoboBrain Project & Cloud Robotics (Wright)	Saxena, et al. RoboBrain: Large-Scale Knowledge Engine for Robots. arXiv:1412.0691	Other resources: 12-minute talk on RoboBrain by Ashutosh Saxena Longer Talk on RoboBrain by Ashutosh Saxena RoboEarth project website
10	M 10/31	Intermediate Task Demonstration 1		Intermediate Documentation Due
10	W 11/2	Novel Object Grasping & Manipulation (Willett)	Saxena, Driemeyer, & Ng. Robotic Grasping of Novel Objects using Vision. International Journal of Robotics Research (IJRR) 27(2): 157-173, 2008. [project website]	Other resources: Lenz, Lee, & Saxena. Deep Learning for Detecting Robotic Grasps. International Journal of Robotics Research (IJRR), 2014. [project website]
11	M 11/7	Grasping in Cluttered Environments (Baucom & Yeung)	Wong, Kaelbling, Lozano-Pérez. Data association for semantic world modeling from partial views. International Journal of Robotics Research 34, 2015.
11	W 11/9	Planning in Dynamic Environments (Gilbert & Thompson)	Sven Koenig & Maxim Likhachev. D* Lite. AAAI 2002. Sandip Aine and Maxim Likhachev. Anytime Truncated D*: Anytime Replanning with Truncation. Proceedings of the International Symposium on Combinatorial Search (SoCS) 2013.	Other resources: Maxim Likhachev, Geoff Gordon and Sebastian Thrun. ARA: Anytime A with Provable Bounds on Sub-Optimality. NIPS 2004. (background that could help you understand the Aine & Likhachev paper)
12	M 11/14	Intermediate Task Demonstration 2 HTN Planning in Continuous Deployment (Prof. Eaton)	Handout - Reading Packet on Planning and HTNs	Intermediate Documentation Due
12	W 11/16	Anytime Path Planning/Re-planning (Kent)	Mike Phillips, Benjamin Cohen, Sachin Chitta and Maxim Likhachev. E-Graphs: Bootstrapping Planning with Experience Graphs. RSS 2012. Mike Phillips, Andrew Dornbush, Sachin Chitta and Maxim Likhachev. Anytime Incremental Planning with E-Graphs. ICRA 2013.
13	M 11/21	Discussion on Projects
13	W 11/23	No Class (Friday class schedule)
14	M 11/28	HRI: Learning from Demonstration (Mendez)	Suay, Toris & Chernova. A Practical Comparison of Three Robot Learning from Demonstration Algorithms. International Journal of Social Robotics 4(4): 319-330, 2012. Akgun, Cakmak, Jiang, & Thomaz. Trajectories and Keyframes for Kinesthetic Teaching: A Human-Robot Interaction Perspective. HRI 2012.	Other resources: Akgun, Cakmak, Jiang, & Thomaz. Keyframe-based Learning from Demonstration. International Journal of Social Robotics 4(4): 343-355, 2012.
14	W 11/30	Intermediate Task Demonstration 3		Intermediate Documentation Due Optional Resources: Collaborative Problem Solving Toris, Kent, & Chernova. Unsupervised learning of multi-hypothesized pick-and-place task templates via crowdsourcing. ICRA 2015. Yanco et al. Analysis of Human-robot Interaction at the DARPA Robotics Challenge Trials. Journal of Field Robotics 32(3): 420-444, 2015. (Skim only)
15	M 12/5	HRI: National Language Interfaces (Brooks)	Tellex et al. Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation. AAAI 2011. Deits, Tellex, et al. Clarifying Commands with Information-Theoretic Human-Robot Dialog. Journal of Human-Robot Interaction 1(1): 78-95, 2012. (skim)	Videos: Robotic forklift following commands Direction understanding on a robotic helicopter Direction understanding for the PR2
15	W 12/7	Social, Economic & Privacy Considerations of Personal Robots (Kent & Wright)	Sharkey. The Ethical Frontiers of Robotics. Science 322(5909):1800-1801, 2008. Lichocki, Billard, & Kahn. The Ethical Landscape of Robotics. IEEE Robotics & Automation Magazine 18(1):39-50, 2011.	Other resources: Prescott et al. Robot Companions: Ethical, Legal, and Social Issues. Tech Report, 2012.
16	M 12/12	In-class working day to coordinate final projects		Optional resources: Multi-Robot Coordination & RoboCup Soccer Yan, Jouandeau & Ali Cherif. A Survey and Analysis of Multi-Robot Coordination. International Journal of Advanced Robotic Systems 10(399), 2013. Multi-Robot Coordination Slides
	Wed 12/21 @2pm	Final Project Showcase		All Final Project Submissions Due by 12/22

Course Description

In order for robots to operate alongside humans in complex, unstructured, uncertain environments, they require substantial intelligence. However, the field of artificial intelligence (AI) has fragmented into various subfields, each studying different aspects of intelligence in relative isolation. The problem of designing intelligent robotic systems that persist in everyday environments provides an opportunity to reintegrate these different aspects of AI into a complete intelligent system.

In this project-based seminar course, students will study and develop an intelligent personal robot assistant, integrating perception, manipulation, learning, planning, and interaction. The resulting versatile robot will be capable of learning and performing a variety of tasks in real-world environments and collaborating effectively with humans. In addition, students will study a variety of advanced AI topics, including high-level perception and reasoning, scalable knowledge representation, lifelong/multi-task learning, integration of perception and control, learning from demonstration, and human-robot interaction.

Prerequisites

At least TWO (2) of the following courses:

CIS 519 or 520 - (Intro. to) Machine Learning
CIS 521 - Fundamentals of AI
CIS 580 or 581 - Machine Perception or Computer Vision
ESE 650 - Learning in Robotics
MEAM 520 - Introduction to Robotics
MEAM 620 - Robotics

Course Format

This course will include two major components:

A seminar-style discussion of various topics in integrated intelligence

We will study a variety of advanced AI techniques for perception, manipulation, reasoning and learning, and methods for integrating these techniques together into a complete intelligent system.
Students will be expected to come to each class prepared to discuss the required reading, having prepared reading summaries.
Students will take turns leading the discussion. Discussion leaders will also be responsible for preparing and delivering a brief presentation on the daily topic.

A semester-long project developing an integrated intelligent personal assistant robot

Students will work in interdisciplinary teams (balancing backgrounds in AI, control, mechatronics, etc.) to design and build intelligent personal assistant robots that are capable of performing a variety of tasks in real-world environments and interacting with humans.
These robots will be expected to operate in the uncontrolled environments of the hallways, elevators, and various rooms of the engineering buildings at Penn.
There will be various milestones, including an initial proposal, design, status reports/demos, and a final report with demonstration.
In addition to developing the intelligent robot, this project will give students experience with developing research proposals, crafting development plans, conducting literature reviews, conducting evaluations, and presenting research results.
Further details on the project will be discussed during the first class.

Optional Textbook

Although there are plenty of online resources on ROS, I would highly recommend that you pick up the following textbook:

Programming Robots with ROS: A Practical Introduction to the Robot Operating System (1st Edition)
by Morgan Quigley, Brian Gerkey, William D. Smart. O'Reilly.

Syllabus

We will study the following topics:

The Robot Operating System (ROS)
Case studies in integrated AI: Stanford STAIR, DARPA urban & robotics challenges, RoboCup, Robo Brain Project, UPenn MAGIC, CMU CoBots
Scalable learning: Online learning, transfer and multi-task learning
Integrating perception, learning & control: architectures, deep learning for object recognition/scene understanding, and deep reinforcement learning
Novel object recognition & manipulation: novel object grasping and manipulation, grasping in cluttered environments
Localization & hierarchical planning: localization and HTN planning in continuous deployment, anytime path planning/replanning, planning in dynamic environments
Human-Robot Interaction: collaborative problem solving, learning from demonstration
Evaluation methodologies and experiment design
Social, economic, safety, & privacy considerations/ethics of personal robots
Project formulation, motivation, design, and presentation

These topics and due dates are all subject to change. Readings for each of these topics will include a variety of journal articles, conference papers, and technical reports.

Key Due Dates

Wed 9/21 - Project Proposals / Getting Started
Wed 10/12 - Project Design
Mon 10/31 - Intermediate Milestones and Documentation
Mon 11/21 - Intermediate Milestones and Documentation
TBA - Final Project Showcase & Report Due

In addition to the major project milestone dates:

Reading summaries on the current day's topic will be due via email before the start of class

Instructor

Photo of Eric Eaton

INSTRUCTOR

Eric Eaton, Ph.D.

E-mail: -- Make certain you put "[CIS 700]" at the start of the subject line to ensure a quicker response.

Office Hours: Mondays/Wednesdays 10:30-11:30am

Office: Levine 264

Course Policies

Communication

Attendance and active participation are expected in every class. Participation includes asking questions, contributing answers, proposing ideas, and providing constructive comments.

As you will discover, I am a proponent of two-way communication and I welcome feedback during the semester about the course. I am available to answer student questions, listen to concerns, and talk about any course-related topic (or otherwise!). Come to office hours! This helps me get to know you. You are welcome to stop by and chat. There are many more exciting topics to talk about that we won't have time to cover in-class.

Whenever you e-mail me, be sure to use a meaningful subject line and include the phrase "[CIS 700]" at the beginning of the subject line. Your e-mail will catch my attention and I will respond quicker if you do this. I make an effort to respond to e-mails within 24 hours on weekdays and 48 hours on weekends. However, unless it is a private matter, you should be posting your questions/issues to Piazza.

Although computer science and robotics work can be intense and solitary, please stay in touch with me and the other students in the course, particularly if you feel stuck on a topic or project and can't figure out how to proceed. Often a quick e-mail, face-to-face conference, or Piazza post can reveal solutions to problems and generate renewed creative and scholarly energy. It is essential that you begin assignments and projects early, since we will be covering a variety of challenging topics in this course.

We will be using Piazza as the course message board. We also make course-wide announcements through Piazza, be sure to sign up for it. You are responsible for the content of all announcements on Piazza.

All questions about course material, assignments, and the project should be posted to Piazza, rather than e-mailed to the instructor. Other students invariably have the same questions as you do, and posting them to Piazza encourages discussion.
A major part of this course is collaboration both within and among teams. Piazza can be a great way to problem-solve as a group.
Post questions publicly as much as possible (you can set your posts to be anonymous to your classmates).
Answer your classmates' posts. Answering questions on Piazza is a great way to learn, and it counts for participation credit.
Don't forget to search the message board before posting! Your question may have been already asked and answered.

Grading

Your grade will be based upon your paper summaries and reading journal, topic presentations, seminar participation, and the semester project. All assignments must be submitted according to the assignment submission instructions.

At the end of the semester, final grades will be calculated as a weighted average of all grades according to the following weights:

Paper Summaries:	15%
Topic Presentations:	20%
Seminar and Course Participation:	10%
Project - Proposal / Getting Started Task:	5%
Project - Design:	7%
Project - Intermediate Milestone Performance / Documentation:	8%
Project - Final Report / Showcase / Documentation / Final Task Performance:	35%
Total:	100%

All grades are given individually, including for the team components (e.g., the final project report). This means that all members of the team may not receive identical grades for the same component; individual adjustments may be made based on the reported contribution of each team member.

Incomplete grades will be given only for verifiable medical illness or other such dire circumstances.

All graded work will receive a percentage grade between 0% and 100%. Here is how the percentage grades will map to final letter grades; percentages are not rounded:

Percentage	Letter grade	Percentage	Letter grade
97% <=	A+ (4.0)	77% <=	C+ (2.3)
93% <=	A (4.0)	73% <=	C (2.0)
90% <=	A- (3.7)	70% <=	C- (1.7)
87% <=	B+ (3.3)	67% <=	D+ (1.3)
83% <=	B (3.0)	60% <=	D (1.0)
80% <=	B- (2.7)	< 60%	F (0.0)

The instructor reserves the right to adjust the percentage ranges for each letter grade upward in your favor.

Academic Integrity

All work in this course is subject to the University's Academic Integrity policy. Violations of the academic integrity policy or the course collaboration policy will incur consequences according to university regulations. Penalties for academic dishonesty may lower the final grade in the course. If one student shares materials inappropriately with another student, both the donor and the recipient of the code are in violation of the academic integrity policy and will be referred to the Office of Student Conduct.

If required by any assignment, you must list all people you worked with or consulted, and all resources you consulted during the completion of the assignment.

Submission and Late Policy

All work must be turned in either in hard-copy or electronic submission, depending on the instructions given in the assignment. E-mailed submissions will not be accepted, unless specified in the assignment instructions. Extensions will be given only in the case of verifiable medical excuses or other such dire circumstances, if requested in advance.

Late submissions will not be accepted in this course. All work must be turned in on-time.

Collaboration Policy

I want to encourage you to discuss the material and work together to understand it. Here are my thoughts on collaborating with other students, faculty, etc.:

The readings and seminar topics are GROUP WORK. Please discuss the readings and associated topics with each other. Work together to understand the material.
The reading summaries are INDIVIDUAL WORK. You are not permitted to work together on preparing (i.e., writing up) the summaries (although you may discuss the readings to understand them) -- these must be your own individual work.

The Getting Started Task is ISOLATED TEAM WORK. Teams are NOT permitted to share code for the Getting Started Task, but may ask high-level questions of other teams.
Although each team will be judged on its own performance, you are encouraged to collaborate across teams on the project. Share the difficulties you're having with each other, and work together to solve problems. However, you must credit the help from others!
You are also permitted to collaborate with people outside of this course on the project. Just remember that the project is ultimately your responsibility, and you will be graded based on YOUR effort, not the effort of anyone external to your team.
You are permitted to use and consult code and other materials found on the internet for any aspect of this course, but you must provide proper attribution.
Any written aspect of the project is ISOLATED TEAM WORK -- the written documents must be completed solely by members of your team.

Summary table:

Individual Work	Individual or Partnered Work	Isolated Team Work (only members of your project team)	Open Collaboration (but remember, you will be graded on YOUR effort)
reading summaries	topic presentation(s)	"getting started" task all written project documents	understanding the readings and topics completing the project / tasks (you must document all assistance)

If you have any questions as to what types of collaborations are allowed and which are dishonest, please ask me before you make a mistake.

Electronic Devices

I have no problem with you using computers or tablets to take notes or consult reference materials during class. Tempting though it may be, please do not check e-mail or visit websites that are not relevant to the course during class. It is a distraction, both for you and (more importantly) for your fellow classmates. Please silence your phones and computers when you enter class.