Instructor

If you are unable to make the instructor's office hours, please send email to make an appointment.

Course web page: http://www.eecs.harvard.edu/~avi/CS282

Upcoming readings:
Nov 22: NO CLASS (Thanksgiving)
Nov 27:*Halpern, An Analysis of First-Order Logics of Probability (sections 1-4 only).
Nov 27:*Koller & Pfeffer, Probabilistic Frame-Based Systems
Nov 29:*Koller & Pfeffer, SPOOK: A System for Probabilistic Object-Oriented Knowledge Representation
Nov 29:Friedman, Getoor, Koller & Pfeffer, Learning Probabilistic Relational Models
Dec 4: Pearl, Causation, Action and Counterfactuals.
Dec 4:Heckerman, A Tutorial on Learning with Bayesian Networks (sections 15-16).
Dec 6: Shachter 1, Evaluating Influence Diagrams.
Dec 6: Matheson, Using Influence Diagrams to Value Information and Control.
Dec 11: Paek & Horvitz, Conversation as Action Under Uncertainty.
Dec 11: *Pasula, Russell, Osland & Ritov, Tracking Many Objects With Many Sensors.

Presentations:
Vince Conitzer's presentation on context-specific independence.
 

Course Description

• Reasoning Under Uncertainty
• Bayesian networks and Markov networks
• Bayesian network inference
• Approximate inference: Markov Chain Monte Carlo and variational methods
• Further structure in Bayesian networks
• Hybrid continuous/discrete networks
• Learning Bayesian networks
• Hidden Markov models and Dynamic Bayesian networks
• Integrating logic and probability
• Pearl's theory of causation
• Decision-Theoretic Planning
• Influence diagrams
• Value of information and control
• Markov decision problems
• Partially observable Markov decision problems




---

Requirements

• Reading the materials and participating in class discussion (50%). Students will submit a one paragraph summary of the readings for each class, and a question for discussion, due before class.
• Course project (30%). The project may be applied, involving the application of probabilistic reasoning to an interesting domain with uncertainty, or a research project investigating a small, self-contained problem in reasoning under uncertainty. Due date: January 12th.
• Presentation of one paper in class (10%). Papers particularly suited for student presentation are marked by an asterisk in the reading list.
• Review of a relevant paper from a recent conference (10%). Due date: January 12th.




---

Syllabus (The dates below are obsolete - we're about a week behind)

Date	Topic	Readings
Sep 18	Introduction
Sep 20	Markov networks and Bayesian networks	Pearl 1
Sep 25	Markov networks and Bayesian networks (continued)	Pearl 1
Sep 27	Qualitative Probabilistic Networks, intro to BN inference	Druzdel-Henrion, Dechter
Oct 2	BN inference: cutset conditioning, junction tree algorithm	Peot-Shachter, Huang-Darwiche
Oct 4	BN inference: differential approach, causal independence	Darwiche, Heckerman-Breese
Oct 9	COLUMBUS DAY	NO CLASS
Oct 11	Context-specific independence	BFGK, Poole 1
Oct 16	Object-oriented Bayesian networks	Pfeffer, chapter 4
Oct 18	Approximate inference: sampling methods	Mackay
Oct 23	Approximate inference: variational methods	JGJS
Oct 25	Approximate inference: other approaches	Poole 2, MWJ, RKD
Oct 30	Hybrid Bayesian networks	Lauritzen
Nov 1	Bayesian network learning	Heckerman (sections 1-10)
Nov 6	Bayesian network learning (continued)	Heckerman (11-14), Friedman
Nov 8	Hidden Markov models, dynamic BNs	Rabiner-Juang, Kjaerulff
Nov 13	Approximate inference in temporal models	Doucet, Boyen-Koller
Nov 15	Logic and probability	Nilsson, Halpern
Nov 20	Relational probability models	Pfeffer, chapters 5 & 6
Nov 22	Causation	Pearl 2, Heckerman (15-16)
Nov 27	Influence diagrams	Shachter 1, JJD, Matheson
Nov 29	Influence diagrams (continued), Markov Decision Problems	Shachter 2, BDH (sections 1-3)
Dec 4	Markov Decision Problems (continued)	BDH (sections 4-6)
Dec 6	Partially observable MDPs	KLC
Dec 11	Reinforcement learning	KLM, Kearns-Singh
Dec 13	Project Presentations
Dec 18	Project Presentations

---

Readings

Pearl 1, Probabilistic Reasoning in Intelligent Systems.
*Druzdzel & Henrion, "Belief Propagation in Qualitative Probabilistic Networks"
Dechter, Bucket Elimination: A Unifying Framework for Probabilistic Inference.
*Peot & Shachter, "Fusion and Propagation with Multiple Observations in Belief Networks"
Huang & Darwiche, Inference in Belief Networks: A Procedural Guide.
*Darwiche, A Differential Approach to Inference in Belief Networks.
*Heckerman & Breese, Causal Independence for Probability Assessment and Inference Using Bayesian Networks.
*Boutilier, Friedman, Goldszmidt & Koller, Context-Specific Independence in Bayesian Networks.
*Poole 1, Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference.
Koller & Pfeffer, Object-oriented Bayesian networks.
Mackay, Introduction to Monte Carlo Methods
Jordan, Ghahramani, Jaakkola & Saul, An Introduction to Variational Methods for Graphical Models.
*Poole 2, Probabilistic conflicts in a search algorithm for estimating posterior probabilities in Bayesian networks.
*Murphy, Weiss & Jordan, Loopy belief propagation for approximate inference: an empirical study.
*Rish, Kask & Dechter, Empirical Evaluation of Approximation Algorithms for Probabilistic Decoding.
*Lauritzen, Propagation of Probabilities, Means and Variances in Mixed Graphical Association Models.
Heckerman, A Tutorial on Learning with Bayesian Networks.
*Friedman, Learning Belief Networks in the Presence of Missing Values and Hidden Variables.
*Rabiner & Juang, An Introduction to Hidden Markov Models.
*Kjaerulff, dHugin: A Computational System for Dynamic Time-Sliced Bayesian Networks.
*Doucet, On Sequential Simulation-Based Methods for Bayesian Filtering.
*Boyen & Koller, Tractable Inference for Complex Stochastic Processes.
*Nilsson, Probabilistic Logic and Probabilistic Logic Revisited
Nov 27:*Halpern, An Analysis of First-Order Logics of Probability.
Pearl 2, Causation, Action and Counterfactuals.
*Shachter 1, Evaluating Influence Diagrams.
*Jensen, Jensen & Dittmer, From Influence Diagrams to Junction Trees.
*Matheson, Using Influence Diagrams to Value Information and Control.
*Shachter 2, Bayes-Ball: The Rational Pastime.
Boutilier, Dean & Hanks, Decision-Theoretic Planning: Structural Assumptions and Computational Leverage.
Kaelbling, Littman & Cassandra, Planning and Acting in Partially Observable Stochastic Domains.
Kaelbling, Littman & Moore, Reinforcement Learning: A Survey.
*Kearns & Singh, Near-Optimal Reinforcement Learning in Polynomial Time.