CIS 419/519 Introduction to Machine Learning

Fall 2015, University of Pennsylvania

Instructor:  Eric Eaton, Ph.D.

Syllabus and Schedule

Contact

INSTRUCTOR

Eric Eaton, Ph.D.

Teaching Assistant

Aayushi Dwivedi (Master's student, CIS)

Teaching Assistant

Libin Jia (Master's student, CIS)

Teaching Assistant

Aditya Mandal (Master's student, Embedded systems)

Teaching Assistant

Maria Peifer (2nd year PhD student, ESE)

Teaching Assistant

Mohammad Rostami (3rd year PhD student, ESE)

Teaching Assistant

Ishan Srivastava (Master's student, CIS)

Course Information

Course Description

Machine learning has been essential to the success of many recent technologies, including autonomous vehicles, search engines, genomics, automated medical diagnosis, image recognition, and social network analysis, among many others. This course will introduce the fundamental concepts and algorithms that enable computers to learn from experience, with an emphasis on their practical application to real problems.  This course will introduce supervised learning (decision trees, logistic regression, support vector machines, Bayesian methods, neural networks and deep learning), unsupervised learning (clustering, dimensionality reduction), and reinforcement
learning.  Additionally, the course will discuss evaluation methodology and recent applications of machine learning, including large scale learning for big data and network analysis.

Prerequisites: CIS121

Course Website: http://www.seas.upenn.edu/~cis519/

Time:  Monday/Wednesday, noon to 1:30 pm
Location:  Skirkanich Hall Auditorium (Berger Auditorium)

Comparison to CIS 520 (Machine Learning)

Due to overwhelming demand, Penn is offering two different machine learning courses this semester:  CIS 419/519 (Introduction to Machine Learning) and CIS 520 (Machine Learning).  This section briefly describes the differences between these courses.

CIS 419/519 Introduction to Machine Learning (this course!) is an introductory-level course in machine learning (ML) with an emphasis on applying ML techniques. The course is cross-listed between undergraduate (419) and graduate (519) versions; the graduate course 519 has somewhat different requirements as described below.  CIS 419/519 is intended for students who are interested in the practical application of existing machine learning methods to real problems, rather than in the statistical foundations and theory of ML covered in CIS 520.  Just because it is listed as "introductory" does not necessarily mean that it is "easier".

CIS 520 Machine Learning is a more mathematically rigorous course in statistical machine learning that provides the background necessary to design and use new ML algorithms.  Consequently, CIS 520 requires students to have basic knowledge of linear algebra (matrices, eigenvectors, etc.) It uses Matlab and is said to require a lot of work, but prepares students to conduct ML research.
 
CIS 519 is NOT a prerequisite for CIS 520.  However, it makes little sense to take CIS 519 after having already taken CIS 520.  You certainly may take CIS 419/519 first and then later take CIS 520.

Essentially, you should take CIS 419/519 if:

• You're more interested in the practical application of machine learning to real problems, or
  
• You don't feel that you have the mathematical background to proceed directly into CIS 520.

And, you should take CIS 520 if you're confident in your mathematical background and:

• You're interested in pursuing research in machine learning that requires the development of new ML methods, or
  
• You're more interested in the statistical foundations and theory of machine learning methods.

Additional Requirements for CIS 519

Students registered for the graduate version of this course (CIS 519) will be required to complete additional work throughout the semester.  This work will include additional components to the homework, additional requirements on the course project, and (possibly) different or additional questions on the exams.

Since the two versions have different requirements, you cannot complete the course as CIS 419 and later petition to have it changed to CIS 519 for graduate credit; if you're considering changing this course to CIS 519 for graduate credit, you should register for the graduate version now.

Text & Software

Textbooks

	CIS 419/519 Course Reading Packet This is a collection of readings that will be used throughout the course. There is NOT a single reading packet you need to obtain -- readings will be distributed incrementally throughout the semester, either in hard-copy or posted online throughout the course.
	Learning From Data by Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.T. Lin. AML Book
	Machine Learning: The Art and Science of Algorithms That Make Sense of Data by Peter Flach Cambridge University Press An electronic copy is available online here via the Penn library.

• Reinforcement Learning: An Introduction by Sutton and Barto, MIT Press, 1998.  (Full text available online; on reserve in Penn library)
• Machine Learning by Tom Mitchell, McGraw Hill, 1997.  (On reserve in Penn library)
  
• A Course in Machine Learning by Hal Daumé III.
• Machine Learning Lecture Notes by Andrew Ng.

For a more advanced treatment of machine learning topics, I would recommend one of the following books:

• Pattern Recognition and Machine Learning by Bishop, Springer, 2006.  (On reserve in Penn library)
• Machine Learning: A Probabilistic Perspective by Kevin P. Murphy, MIT Press, 2012.  (On reserve in Penn library)
• The Elements of Statistical Learning, 2nd edition by Hastie, Tibshirani and Friedman, Springer-Verlag, 2008.  (Available online)
• Bayesian Reasoning and Machine Learning by David Barber, Cambridge University Press, 2012. (Available online)
• Convex Optimization by Stephen Boyd and Lieven Vandenberghe, Cambridge University Press, 2004. (Available online)
• Information Theory, Inference, and Learning Algorithms by David Mackay, Cambridge University Press, 2003. (Available online)
• Deep Learning by Yoshua Bengio, Ian Goodfellow, and Aaron Courville.  (Available online)

Software

• python -- we'll be using python throughout the course to implement various ML algorithms and run experiments
• Google Developer Python Tutorial (highly recommended as a way to master python in just a few hours!) 
• NumPy Tutorial (also highly recommended!)
  
• Python tutorial (work at least through section 5; skip sections 2, 3.1.3)
• Python quick reference
• scikit-learn
  
• Weka -- be sure to use the "Stable Book 3rd Edition" version.  Weka is built using java, so you can download it into your home directory and run it directly there.
  

Course Policies

Communication

Attendance and active participation are expected in every class. Participation includes asking questions, contributing answers, proposing ideas, and providing constructive comments.

As you will discover, I am a proponent of two-way communication and I welcome feedback during the semester about the course. I am available to answer student questions, listen to concerns, and talk about any course-related topic (or otherwise!). Come to office hours! This helps me get to know you. You are welcome to stop by and chat. There are many more exciting topics to talk about that we won't have time to cover in-class.

Whenever you e-mail me, be sure to use a meaningful subject line and include the phrase "[CIS 419]" or "[CIS 519] at the beginning of the subject line. Your e-mail will catch my attention and I will respond quicker if you do this. I make an effort to respond to e-mails within 24 hours on weekdays and 48 hours on weekends.  However, unless it is a private matter, you should be posting your questions/issues to Piazza.

Although computer science work can be intense and solitary, please stay in touch with me and the TAs, particularly if you feel stuck on a topic or project and can't figure out how to proceed. Often a quick e-mail, face-to-face conference, or Piazza post can reveal solutions to problems and generate renewed creative and scholarly energy. It is essential that you begin assignments and projects early, since we will be covering a variety of challenging topics in this course.

• All questions about course material and assignments should be posted to Piazza, rather than e-mailed to the instructor or the TAs.  Other students invariably have the same questions as you do, and posting them to Piazza encourages discussion.
• Post questions publicly as much as possible (you can set your posts to be anonymous to your classmates).
• Answer your classmates' posts.  Answering questions on Piazza is a great way to learn, and it counts for participation credit. 
• Don't forget to search the message board before posting!  Your question may have been already asked and answered.
  

Grading

Your grade will be based upon five homework assignments, two exams, and a course project.  Assignments must be submitted according to the assignment submission instructions.

At the end of the semester, final grades will be calculated as a weighted average of all grades according to the following weights:

The project grade will be broken down further in the Project Description.

Incomplete grades will be given only for verifiable medical illness or other such dire circumstances.

All graded work will receive a percentage grade between 0% and 100%.  Here is how the percentage grades will map to final letter grades; percentages are not rounded:

Academic Integrity

Submission and Late Policy

Exams

There will be two exams in this course.  The exams will be closed-book and closed-notes.  They will cover material from lectures, homeworks, and assigned readings (including topics not discussed in class).  So, keep up with those readings!

Collaboration Policy

I want to encourage you to discuss the material and work together to understand it. Here are my thoughts on collaborating with other students:

• The readings and lecture topics are group work. Please discuss the readings and associated topics with each other. Work together to understand the material. I highly recommend forming a reading group to discuss the material -- we will explore many challenging ideas and it helps to have multiple people working together to understand them.

• It is fine to discuss the topics covered in the homeworks, to discuss approaches to problems, and to sketch out general solutions. However, you MUST write up the homework answers, solutions, and programs individually. You are not permitted to share specific solutions, mathematical results, program code, knowledge representations, experimental results, etc. If you made any notes or worked out something on a white board with another person while you were discussing the homework, you shouldn't use those notes while writing up your answer, however tempted you may be to do so.

• You are fully permitted to (and should!) discuss projects with members of your team. I also encourage you to work outside of your team to understand the other topics in the course.
  

• Under ABSOLUTELY NO circumstances should you share computer code with another student outside of your project team.  Similarly, you are not permitted to use or consult code found on the internet for any of your assignments.

• Exams and papers, of course, must be your own individual work.

If you have any questions as to what types of collaborations are allowed and which are dishonest, please ask me before you make a mistake.

Electronic Devices

I have no problem with you using computers or tablets to take notes or consult reference materials during class.  Tempting though it may be, please do not check e-mail or visit websites that are not relevant to the course during class.  It is a distraction, both for you and (more importantly) for your fellow classmates.  Please silence your phones and computers when you enter class.

Reference Links