>
Lec. No.  Date  Key  Topic  Readings  Lecture notes  Assignments  

1  Wed 2/1  Lecture: Introduction 
Background reading: On Learnable Mechanism Design, by D.C.Parkes. In Collectives and the Design of Complex Systems, Kagan Tumer and David Wolpert (eds.) , pages 107131, SpringerVerlag, 2004. On the Agenda(s) of Research on MultiAgent Learning. by Y.Shoham, R.Powers and T.Grenager. In Proc. 2004 AAAI Fall Symposium on Artificial MultiAgent Learning Rationality and Bounded Rationality, by R.J.Aumann. In Games and Economic Behavior, 21, 214 (1997) 

2  Mon 2/6  Lecture: Game Theory I (Normal Form Games)  Avail. from the TFs Reading: M.Osborne and A.Rubinstein, A course in Game Theory, MIT Press 1994, pp.1165  Homework 1 out (due 2/13)  
3  Wed 2/8  Lecture: Game Theory II (Extensive form games, Repeated games.)  Avail from the TFs Reading: M.Osborne and A.Rubinstein, A course in Game Theory, MIT Press 1994, pp.89115, 140146, 160161, 199204, 212218.  
4  Mon 2/13  Lecture: Game Theory III: Folk theorems; Games of Incomplete information; Bayesian consistency. 
Reading: Avail. from the TFs D.Fudenberg and J.Tirole, Game Theory, MIT Press, 1991, pp.145160,
203206
Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.209223 Gibbons, Game Theory for Applied Economists, PUP, 1992.pp.173183, 224235 Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.321350, 364365 
Continuation of lecture notes from 2/8. Also:
Handwritten notes: 

5  Wed 2/15  Lecture: Mechanism Design (Revelation Principle, VCG, truthful mechanisms)  Reading: Mechanism Design, Parkes, Chapter 2 in PhD dissertation, Iterative Combinatorial Auctions: Achieving Economic and Computational Efficiency, Department of Computer and Information Science, University of Pennsylvania, May 2001.  Homework 1 due. Homework 2 out (due 2/27)  
Mon 2/21  Holiday.  
6  Wed 2/22  Lecture: Nash implementation  A Crash Course in Implementation Theory, M.O.Jackson in Social Choice and Welfare, Vol. 18, No. 4, 2001, pp 655708. Not sections 2.9, 4 or 5 
Handwritten notes: 

7  Mon 2/27  Lecture: Sequential decision theory (MDPs, Reinforcement learning.)  Reinforcement Learning: A Survey L.P.Kaelbling, M. L. Littman and A. W. Moore in Journal of Artificial Intelligence Research 4 (1996) 237285 For class: Sections 15 (not 4.1), 66.1.1, 6.3 
RL Examples 
Homework 2 due. Homework 3 out (due 3/6)  
8  Wed 3/1  Discussion: GT Learning 
Survey paper (not for discussion):
Adaptive Heuristics, H.Peyton Young to appear in The New Palgrave, 2nd Edition
For discussion: Learning Mixed Equilibria , D.Fudenberg and D.Kreps in Games and Economic Behavior 5, 320367 (1993) pp.320333, 338341; On the nonconvergence of Fictitious Play in Coordination Games, D.Foster and H.P.Young, Games and Economic Behavior 25(1) 7996. 1998 pp.7983, 9495; The evolution of conventions, H.P.Young, Econometrica 61(1):5784, 1993 pp.6263 
Julius Degesys  
9  Mon 3/6  Discussion: AI Learning 
(Intro paper. Not for discussion.) Markov games as a framework for multiagent reinforcement
learning., M.L.Littman, In Proc. 11th Int. Conf. on Machine Learning, 157163 1994.
For discussion: Nash QLearning for GeneralSum Stochastic Games by Junling Hu and Michael P. Wellman, in Journal of Machine Learning Research 4 (2003) 
Ilan Lobel


10  Wed 3/8  Discussion: GT Learning 
Calibrated learning and Correlated equilibrium
by D.Foster and R.Vohra, in Games and Economic Behavior 21, 4055 1997
Regret in the OnLine Decision Problem, by D.Foster and R.Vohra, in Games and Economic Behavior 29, 735 (1999). PAGES 721 ONLY 
James Burns  
11  Mon 3/13  Discussion: AI Learning 
Correlated Qlearning. K.Hall and A.Greenwald,
in Proc. 20th Int. Conf. on Machine Learning. 242249, 2003
Cyclic Equilibria in Markov Games, M.Zinkevich, A.Greenwald and M.L.Littman, in Proc NIPS 2005. NOT APPENDIX.  Haoqi Zhang


12  Wed 3/15  Discussion: GT Learning  Rational learning leads to Nash equilibrium,
by E.Kalai and E.Lehrer, Econometrica 61, 10191045 1993. PAGES 10191035 ONLY
Beliefs in Repeated Games, by John Nachbar, Econometrica. 73(2) 459480. 2005. PAGES 459474 ONLY 
Ivo Parashkavov, Florin Constantin  
13  Mon 3/20  Discussion: AI Learning  Efficient Learning Equilibrium, R.Brafman and M.Tennenholtz. in Artif. Intell. 159(12): 2747 (2004)  Neal Gupta  
14  Wed 3/22  Discussion: Nash memory  A Novel Method for Strategy Acquisition in NPlayer Games,
S. Phelps, M. Marcinkewitz, S. Parsons and P. McBurney. To appear in Proc.
AAMAS'06.
A GameTheoretic Memory Mechanism for Coevolution Sevan C. Ficici and Jordan B. Pollack in Proc. GECCO 2003.  Evan Sprecher, Angela Sze  
Spring break.  
15  Mon 4/3  Discussion: AI Learning  On the Agenda(s) of Research on MultiAgent
Learning. by Y.Shoham, R.Powers and T.Grenager.
In Proc. 2004 AAAI Fall Symposium on Artificial MultiAgent Learning
Learning against opponents with bounded memory. by R.Powers and Y.Shoham, In Proc. IJCAI 2005. 
Philip Hendrix, Ece Kamar 

16  Wed 4/5  Discussion: Applications  We will be discussing applications of the techniques we have covered. Please submit two paragraphs on such potential applications  Applications Action Items  
17  Mon 4/10  Discussion: Hayek machine  Evolution of Cooperative Problem solving in an artificial economy, E.B.Baum and I.Durdanovic in Neural Computation 12 (12): 27432775 (2000)  Quang Duong 

18  Wed 4/12  Discussion: Partial Intermediation  On partiallycontrolled multiagent systems, R.Brafman and M.Tennenholtz in Journal of Artificial Intelligence Research, 4:477 507, 1996.  Katy Milkman 

19  Mon 4/17  Discussion: Inverse RL  Algorithms for Inverse Reinforcement
Learning, A.Y.Ng and S.Russell in Proc. ICML 2000.
Apprenticeship Learning via Inverse Reinforcement Learning, P.Abbeel and A.Y.Ng, in Proc. ICML 2004 
ChihHan Yu, Ian Rose 
Project proposals due 

20  Wed 4/19  Discussion: Collective Design  Overcoming Communication
Restrictions in Collectives, by K.Tumer and A.K.Agogino, in Proc. IJCNN 2004. THROUGH SECTION II.B ONLY
Learning Sequences of Actions in Collectives of Autonomous Agents, by K.Tumer, A.K.Agogino and D.H.Wolpert, in Proc. AAMAS'02. PAGE 1, AND THEN 2.3 ONWARDS ONLY. Reinforcement Learning in Large MultiAgent Systems, A.Agogino and K.Tumer in Proc. AAMAS'05. WHOLE PAPER! 
Sheel Ganatra, Charlie Frogner  
21  Mon 4/24  Discussion: Coordinated RL  QDecomposition for Reinforcement Learning
Agents, S.Russell and A.L.Zimdars,
in Proc. 20th Int. Conf. on Machine Learning (ICML2003)
An Overview of MAXQ Hierarchical Reinforcement Learning, by Dietterich, T. G. (2000) In B. Y. Choueiry and T. Walsh (Eds.) Proceedings of the Symposium on Abstraction, Reformulation and Approximation SARA 2000, Lecture Notes in Artificial Intelligence (pp. 2644), New York: Springer Verlag. 
Erik Schultink, Jimmy Sun  
22  Wed 4/26  Discussion: Nash implementation in CS  On Decentralized Incentive Compatible Mechanisms for Partially Informed Environments, by Ahuva Mu'alem, in Proc. EC05  Neil Mehta and Ariel Kleiner 

23  Mon 5/1  Discussion: Repeated Games in CS  RepeatedGame Modeling of Multicast Overlays, by M.Afergan and R.Sami to appear in IEEE INFOCOM 2006.  Rohan Murty and Joshua Dezube  
24  Wed 5/3  Conclusions  
Wed 5/3  Student Presentations, 45.30pm  Project presentations  
Wed 5/17  Final projects due, noon  Projects due 