CS 181

Intelligent Machines: Perception, Learning and Uncertainty

Harvard University

Prof. David C. Parkes

Spring 2010

Teaching Staff

David C. Parkes
Office: Maxwell Dworkin 229
Email: parkes -at- eecs.harvard.edu
Office hours: Wed 2:30-3:30
TF: John Lai
Email: jklai -at- post.harvard.edu
Office Hours: TBD
Head TF: Victor Shnayder
Email: shnayder -at- gmail.com
Office Hours: TBD

Send email to the teaching staff at cs181 -at- fas.harvard.edu

Extension School Class

Course web page: CSCI E-181
Extension School TF: Swapna Reddy
Email: sreddy -at- seas.harvard.edu
Office Hours: TBD

Course Description

This course provides a broad and rigorous introduction to machine learning, probabilistic reasoning and decision making in uncertain environments.

The course should be of interest to undergraduate students in computer science, applied mathematics, sciences and engineering, and lower-level graduate students, looking to gain an introduction to the tools of machine learning and probabilistic reasoning with applications to data-intensive problems in the applied sciences, natural sciences and social sciences.

For students with interests in the fundamentals of artificial intelligence, this course will address three central, related questions in the design and engineering of intelligent systems. How can a system process its perceptual inputs in order to obtain a reasonable picture of the world? How can we build programs that learn from experience? And how can we design systems to deal with the inherent uncertainty in the real world?

Our approach to these questions will be both theoretical and practical. We will develop a mathematical underpinning for the methods of machine learning and probabilistic reasoning. We will look at a variety of successful algorithms and applications. We will also discuss the motivations behind the algorithms, and the properties that determine whether or not they will work well for a particular task.


There will be a recommended weekly section led by the teaching staff. Sections will be interactive and contain discussion of questions and examples related to the course. They will include discussion of homework assignments, and worked out examples of questions that might be good preparation for the midterms.

Textbooks and References

There are no required texts for this class. Extensive lecture notes will be handed out. But there are several useful books, and they are strongly recommended for extra reading:
If you only own one book on artificial intelligence, this is the one you should have. It is extensive, thorough, and full of interesting and useful insights. If you buy this book, make sure to get the third edition (which is just released).
A general introduction to the topic of machine learning.
An excellent and affordable book on machine learning. Goes into greater technical detail than we will go in this course.
The classic reference on Bayesian networks.
An easy-to-read book by pioneers in reinforcement learning.


Requirements and Grading

There are five homework assignments and will make up 50% of the final grade (the breakdown of grade is 10%, 10%, 13%, 10% and 7% for assignments 1-5 respectively). There are two midterms, each counting for 15% of the final grade. The second midterm will occur on the last day of class, and will focus on the material in the second half of the course. There is also a final project. This is a group exercise involving the development of an intelligent agent that will compete in competition against other agents in identifying tasty from poisonous plants and moving around a virtual world! The design, analysis, evaluation and, to some extent, performance of your agent will count for 20% of the final grade.


Assignments will be done in groups of 2 or 3 people. The course staff can help you to find a group. The goal of the assignments in the course is twofold: to help you master the technical material, and to show you how the techniques we are learning can be used to build powerful and cool applications. There will be five homework assignments, each of which will usually be due two weeks after being handed out. Each assignment will generally be centered around a particular methodology and task and involve programming. You will need to answer some theoretical problems, write a program to solve the task, and evaluate your program through experiments to compare the performance of different algorithms and methods.

Late Policy

All homework should be submitted electronically, by the beginning of class on the due date. Submissions received after that time will be marked late. Late assignments may be submitted by the beginning of class one week after the assignment's due date. Submissions received after this time will not be considered. Each student will have six free late days in the semester. After the late days are used up, assignments will be penalized by 10% per day. An extra late day will be accumulated each day at 1:00 PM.

Final Project

In place of a final exam, the final project for this class will take the form of a challenge problem. This problem will require many of the techniques discussed in class, as well as additional creativity. The project will be done in teams of 2 or 3. You'll be designing and analyzing a foraging agent in a virtual world! The project will be due in the middle of reading period and there will be a tournament between agents submitted by different teams at the end of reading week.

Course Plan

  • Introduction
    Mon Jan 25: Introduction
  • Learning (Supervised and Unsupervised)
    Wed Jan 27: Decision Trees and Classification
    Mon Feb 1: Overfitting, Description Length and Cross-validation
    Wed Feb 3: Neural Networks I- Perceptrons
    Mon Feb 8: Neural Networks II- Multi-Layer
    Wed Feb 10: Neural Networks III- Model selection and applications
    Mon Feb 15: No class President's Day
    Wed Feb 17: Non-parametric (Nearest-neighbor, kernel methods)
    Mon Feb 22: Support Vector Machines
    Wed Feb 24: Clustering Algorithms (Hierarchical and k-means)
    Mon Mar 1: Computational Learning Theory
    Wed Mar 3: Ensemble Learning and Boosting
    Mon Mar 8: Guest lecture
  • First Midterm
    Wed Mar 10: First midterm
  • Spring break
    Mon Mar 15: No class
    Wed Mar 18: No class
  • Learning with Probabilistic Models
    Mon Mar 22: Maxmium Likelihood, Bayesian and MAP Estimation
    Wed Mar 24: Naive Bayes and Bayesian Networks
    Mon Mar 29: Bayesian Networks- Modeling
    Mon Apr 5: Bayesian Networks- Inference
    Wed Apr 7: Approximate inference and EM algorithm
    Mon Apr 12: Hidden Markov Models
    Wed Apr 14: Decision Making I (Markov Decision Processes)
    Mon Apr 19: Decision Making II (Partially-observable MDPs)
    Wed Apr 21: Learning to Act (Reinforcement Learning)
  • Closing
    Mon Apr 26: Wrap-up (Current Directions)
  • Second Midterm
    Wed Apr 28: Second midterm
  • Final Project
    Tue May 4: Turn project in
    Thu May 6: Tournament