CS 286r Syllabus, Spring 2006.

MW 1-2.30pm, Maxwell Dworkin 319

This schedule is subject to change. Many other papers are related to this topic. See here for additional reading.

Lec. No. Date Key Topic Readings Lecture notes Assignments
1 Wed 2/1 Lecture: Introduction Background reading:
On Learnable Mechanism Design, by D.C.Parkes. In Collectives and the Design of Complex Systems, Kagan Tumer and David Wolpert (eds.) , pages 107-131, Springer-Verlag, 2004.
On the Agenda(s) of Research on Multi-Agent Learning. by Y.Shoham, R.Powers and T.Grenager. In Proc. 2004 AAAI Fall Symposium on Artificial Multi-Agent Learning
Rationality and Bounded Rationality, by R.J.Aumann. In Games and Economic Behavior, 21, 2-14 (1997)
2 Mon 2/6 Lecture: Game Theory I (Normal Form Games) Avail. from the TFs Reading: M.Osborne and A.Rubinstein, A course in Game Theory, MIT Press 1994, pp.11-65 PDF PDF Homework 1 out (due 2/13)
3 Wed 2/8 Lecture: Game Theory II (Extensive form games, Repeated games.) Avail from the TFs Reading: M.Osborne and A.Rubinstein, A course in Game Theory, MIT Press 1994, pp.89-115, 140-146, 160-161, 199-204, 212-218. PDF
4 Mon 2/13 Lecture: Game Theory III: Folk theorems; Games of Incomplete information; Bayesian consistency. Reading: Avail. from the TFs D.Fudenberg and J.Tirole, Game Theory, MIT Press, 1991, pp.145-160, 203-206
Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.209-223
Gibbons, Game Theory for Applied Economists, PUP, 1992.pp.173-183, 224-235
Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.321-350, 364-365
Continuation of lecture notes from 2/8. Also: PDF
Hand-written notes: PDF
5 Wed 2/15 Lecture: Mechanism Design (Revelation Principle, VCG, truthful mechanisms) Reading: Mechanism Design, Parkes, Chapter 2 in PhD dissertation, Iterative Combinatorial Auctions: Achieving Economic and Computational Efficiency, Department of Computer and Information Science, University of Pennsylvania, May 2001. PDF Homework 1 due.
PDF Homework 2 out (due 2/27)
Mon 2/21 Holiday.
6 Wed 2/22 Lecture: Nash implementation A Crash Course in Implementation Theory, M.O.Jackson in Social Choice and Welfare, Vol. 18, No. 4, 2001, pp 655-708. Not sections 2.9, 4 or 5 PDF
Hand-written notes: PDF
7 Mon 2/27 Lecture: Sequential decision theory (MDPs, Reinforcement learning.) Reinforcement Learning: A Survey L.P.Kaelbling, M. L. Littman and A. W. Moore in Journal of Artificial Intelligence Research 4 (1996) 237-285 For class: Sections 1--5 (not 4.1), 6-6.1.1, 6.3 PDF
RL ExamplesPDF
Homework 2 due. PDF Homework 3 out (due 3/6)
8 Wed 3/1 Discussion: GT Learning Survey paper (not for discussion): Adaptive Heuristics, H.Peyton Young to appear in The New Palgrave, 2nd Edition
For discussion:
Learning Mixed Equilibria , D.Fudenberg and D.Kreps in Games and Economic Behavior 5, 320-367 (1993) pp.320-333, 338-341;
On the non-convergence of Fictitious Play in Coordination Games, D.Foster and H.P.Young, Games and Economic Behavior 25(1) 79-96. 1998 pp.79-83, 94-95;
The evolution of conventions, H.P.Young, Econometrica 61(1):57-84, 1993 pp.62-63
Julius Degesys
9 Mon 3/6 Discussion: AI Learning (Intro paper. Not for discussion.) Markov games as a framework for multi-agent reinforcement learning., M.L.Littman, In Proc. 11th Int. Conf. on Machine Learning, 157--163 1994.
For discussion:
Nash Q-Learning for General-Sum Stochastic Games by Junling Hu and Michael P. Wellman, in Journal of Machine Learning Research 4 (2003)
Ilan Lobel PDF
10 Wed 3/8 Discussion: GT Learning Calibrated learning and Correlated equilibrium by D.Foster and R.Vohra, in Games and Economic Behavior 21, 40-55 1997
Regret in the On-Line Decision Problem, by D.Foster and R.Vohra, in Games and Economic Behavior 29, 7--35 (1999). PAGES 7--21 ONLY
James Burns PDF
11 Mon 3/13 Discussion: AI Learning Correlated Q-learning. K.Hall and A.Greenwald, in Proc. 20th Int. Conf. on Machine Learning. 242-249, 2003
Cyclic Equilibria in Markov Games, M.Zinkevich, A.Greenwald and M.L.Littman, in Proc NIPS 2005. NOT APPENDIX.
Haoqi Zhang PDF
12 Wed 3/15 Discussion: GT Learning Rational learning leads to Nash equilibrium, by E.Kalai and E.Lehrer, Econometrica 61, 1019-1045 1993. PAGES 1019--1035 ONLY
Beliefs in Repeated Games, by John Nachbar, Econometrica. 73(2) 459-480. 2005. PAGES 459--474 ONLY
Ivo Parashkavov, Florin Constantin PDF
13 Mon 3/20 Discussion: AI Learning Efficient Learning Equilibrium, R.Brafman and M.Tennenholtz. in Artif. Intell. 159(1-2): 27-47 (2004) Neal Gupta PDF
14 Wed 3/22 Discussion: Nash memory A Novel Method for Strategy Acquisition in N-Player Games, S. Phelps, M. Marcinkewitz, S. Parsons and P. McBurney. To appear in Proc. AAMAS'06.
A Game-Theoretic Memory Mechanism for Coevolution Sevan C. Ficici and Jordan B. Pollack in Proc. GECCO 2003.
Evan Sprecher, Angela Sze PDF PDF
Spring break.
15 Mon 4/3 Discussion: AI Learning On the Agenda(s) of Research on Multi-Agent Learning. by Y.Shoham, R.Powers and T.Grenager. In Proc. 2004 AAAI Fall Symposium on Artificial Multi-Agent Learning
Learning against opponents with bounded memory. by R.Powers and Y.Shoham, In Proc. IJCAI 2005.
Philip Hendrix, Ece Kamar
16 Wed 4/5 Discussion: Applications We will be discussing applications of the techniques we have covered. Please submit two paragraphs on such potential applications Applications Action Items
17 Mon 4/10 Discussion: Hayek machine Evolution of Cooperative Problem solving in an artificial economy, E.B.Baum and I.Durdanovic in Neural Computation 12 (12): 2743-2775 (2000) Quang Duong
18 Wed 4/12 Discussion: Partial Intermediation On partially-controlled multiagent systems, R.Brafman and M.Tennenholtz in Journal of Artificial Intelligence Research, 4:477-- 507, 1996. Katy Milkman
19 Mon 4/17 Discussion: Inverse RL Algorithms for Inverse Reinforcement Learning, A.Y.Ng and S.Russell in Proc. ICML 2000.
Apprenticeship Learning via Inverse Reinforcement Learning, P.Abbeel and A.Y.Ng, in Proc. ICML 2004
Chih-Han Yu, Ian Rose PDF PDF
Project proposals due
20 Wed 4/19 Discussion: Collective Design Overcoming Communication Restrictions in Collectives, by K.Tumer and A.K.Agogino, in Proc. IJCNN 2004. THROUGH SECTION II.B ONLY
Learning Sequences of Actions in Collectives of Autonomous Agents, by K.Tumer, A.K.Agogino and D.H.Wolpert, in Proc. AAMAS'02. PAGE 1, AND THEN 2.3 ONWARDS ONLY.
Reinforcement Learning in Large Multi-Agent Systems, A.Agogino and K.Tumer in Proc. AAMAS'05. WHOLE PAPER!
Sheel Ganatra, Charlie Frogner
21 Mon 4/24 Discussion: Coordinated RL Q-Decomposition for Reinforcement Learning Agents, S.Russell and A.L.Zimdars, in Proc. 20th Int. Conf. on Machine Learning (ICML-2003)
An Overview of MAXQ Hierarchical Reinforcement Learning, by Dietterich, T. G. (2000) In B. Y. Choueiry and T. Walsh (Eds.) Proceedings of the Symposium on Abstraction, Reformulation and Approximation SARA 2000, Lecture Notes in Artificial Intelligence (pp. 26-44), New York: Springer Verlag.
Erik Schultink, Jimmy Sun
22 Wed 4/26 Discussion: Nash implementation in CS On Decentralized Incentive Compatible Mechanisms for Partially Informed Environments, by Ahuva Mu'alem, in Proc. EC-05 Neil Mehta and Ariel Kleiner
23 Mon 5/1 Discussion: Repeated Games in CS Repeated-Game Modeling of Multicast Overlays, by M.Afergan and R.Sami to appear in IEEE INFOCOM 2006. Rohan Murty and Joshua Dezube
24 Wed 5/3 Conclusions
Wed 5/3 Student Presentations, 4-5.30pm Project presentations
Wed 5/17 Final projects due, noon Projects due

final project ideas

main page