Prof. David C. Parkes
MW 1:00 - 2:30, Maxwell Dworkin G125
- David C. Parkes
- Office: Maxwell Dworkin 229
- Email: parkes -at- eecs.harvard.edu
- Office hours: Wed 2:30-3:30
- TF: John Lai
- Email: jklai -at- post.harvard.edu
- Office Hours: TBD
- Head TF: Victor Shnayder
- Email: shnayder -at- gmail.com
- Office Hours: TBD
Send email to the teaching staff at
cs181 -at- fas.harvard.edu
Extension School Class
- Course web page: CSCI E-181
- Extension School TF: Swapna Reddy
- Email: sreddy -at- seas.harvard.edu
- Office Hours: TBD
This course provides a broad and rigorous introduction to machine learning,
probabilistic reasoning and decision making in uncertain environments.
The course should be of interest to undergraduate
students in computer science, applied mathematics, sciences
and engineering, and lower-level graduate students, looking to gain an introduction
to the tools of machine learning and probabilistic reasoning with
applications to data-intensive problems in the applied sciences,
natural sciences and social sciences.
For students with interests in the fundamentals of artificial
intelligence, this course will address three central, related
questions in the design and engineering of intelligent systems. How
can a system process its perceptual inputs in order to obtain a
reasonable picture of the world? How can we build programs that learn
from experience? And how can we design systems to deal with the
inherent uncertainty in the real world?
Our approach to these questions will be both theoretical and
practical. We will develop a mathematical
underpinning for the methods of machine learning and probabilistic
reasoning. We will look at a variety of successful algorithms and applications.
We will also discuss the motivations behind the
algorithms, and the properties that determine
whether or not they will work well for a
There will be a recommended weekly section led by the teaching
staff. Sections will be interactive and contain discussion of
questions and examples related to the course. They will include
discussion of homework assignments, and worked out examples of
questions that might be good preparation for the midterms.
Textbooks and References
There are no required texts for this class. Extensive lecture notes
will be handed out.
But there are several useful books, and they are strongly recommended for extra
- Artificial Intelligence: A Modern Approach ,
Stuart J. Russell and Peter Norvig, Prentice Hall, third Edition, 2009.
- If you only own one book on artificial intelligence, this is the one
you should have. It is extensive, thorough, and
full of interesting and useful insights. If you
buy this book, make sure to get the third
edition (which is just released).
- Machine Learning, Tom Mitchell, McGraw-Hill, 1997.
- A general introduction to the topic of machine learning.
- Pattern Recognition and Machine Learning, Christopher M. Bishop,
Springer, 2006. [Amazon] [web]
- An excellent and affordable book on machine learning. Goes into greater technical
detail than we will go in this course.
- Probabilistic Reasoning in Intelligent Systems, Judea
Pearl, Morgan Kaufmann, 1988. [Amazon]
- The classic reference on Bayesian networks.
- Reinforcement Learning: An Introduction, R. S. Sutton and
A. G. Barto, MIT Press, 1998. [Amazon] [web] [read]
- An easy-to-read book by pioneers in reinforcement learning.
- Students should be comfortable with writing non-trivial programs (e.g. CS 51). In particular, we will provide support code in C, C++ and Python
and students should know
one of these languages. However, you are free to write your code in
a language of your choice; you will just need to interface to the
code that we provide.
- Students should have a background in
basic probability theory (e.g. STAT 110). If you haven't had a
formal probability course, but are
very comfortable with the basics of
probability, you will probably be
Requirements and Grading
There are five homework assignments and will make up 50%
the final grade (the breakdown of grade is 10%, 10%, 13%, 10% and 7% for
assignments 1-5 respectively).
There are two midterms, each counting for 15% of the final grade.
The second midterm will occur on the last day of class, and will focus
on the material in the second half of the course.
There is also a final project. This is a group exercise involving the
development of an intelligent agent that will compete in competition
against other agents in identifying tasty from poisonous plants and
moving around a virtual world! The design, analysis, evaluation and,
to some extent, performance of your agent will count for 20% of the final grade.
Assignments will be done in groups of 2 or 3 people. The course
staff can help you to find a group. The goal of the assignments
in the course is twofold: to help you master the technical material,
and to show you how the techniques we are learning can be used to
build powerful and cool applications. There will be five homework
assignments, each of which will usually be due two weeks after being
handed out. Each assignment will generally be centered around a
particular methodology and task and involve programming. You will
need to answer some theoretical problems, write a program to solve the
task, and evaluate your program through experiments to compare the
performance of different algorithms and methods.
All homework should be submitted electronically, by the beginning of
class on the due date. Submissions received after that time will be
marked late. Late assignments may be submitted by the beginning of
class one week after the assignment's due date. Submissions received
after this time will not be considered.
Each student will have six free late days in the semester. After the
late days are used up, assignments will be penalized by 10% per day.
An extra late day will be accumulated each day at 1:00 PM.
In place of a final exam, the final project for this class will take
the form of a challenge problem. This problem will require many of
the techniques discussed in class, as well as additional creativity.
The project will be done in teams of 2 or 3. You'll be designing and
analyzing a foraging agent in a virtual world! The project will be
due in the middle of reading period and there will be a
tournament between agents submitted by different teams at the end of reading week.
Mon Jan 25: Introduction
- Learning (Supervised and Unsupervised)
Wed Jan 27: Decision Trees and Classification
Mon Feb 1: Overfitting, Description Length and Cross-validation
Wed Feb 3: Neural Networks I- Perceptrons
Mon Feb 8: Neural Networks II- Multi-Layer
Wed Feb 10: Neural Networks III- Model selection and applications
Mon Feb 15: No class President's Day
Wed Feb 17: Non-parametric (Nearest-neighbor, kernel methods)
Mon Feb 22: Support Vector Machines
Wed Feb 24: Clustering Algorithms (Hierarchical and k-means)
Mon Mar 1: Computational Learning Theory
Wed Mar 3: Ensemble Learning and Boosting
Mon Mar 8: Guest lecture
- First Midterm
Wed Mar 10: First midterm
- Spring break
Mon Mar 15: No class
Wed Mar 18: No class
- Learning with Probabilistic Models
Mon Mar 22: Maxmium Likelihood, Bayesian and MAP Estimation
Wed Mar 24: Naive Bayes and Bayesian Networks
Mon Mar 29: Bayesian Networks- Modeling
Mon Apr 5: Bayesian Networks- Inference
Wed Apr 7: Approximate inference and EM algorithm
Mon Apr 12: Hidden Markov Models
Wed Apr 14: Decision Making I (Markov Decision Processes)
Mon Apr 19: Decision Making II (Partially-observable MDPs)
Wed Apr 21: Learning to Act (Reinforcement Learning)
Mon Apr 26: Wrap-up (Current Directions)
- Second Midterm
Wed Apr 28: Second midterm
- Final Project
Tue May 4: Turn project in
Thu May 6: Tournament