|Lec. No.||Date||Key||Topic||Readings||Lecture notes||Assignments|
|1||Wed 2/1||Lecture: Introduction||
On Learnable Mechanism Design, by D.C.Parkes. In Collectives and the Design of Complex Systems, Kagan Tumer and David Wolpert (eds.) , pages 107-131, Springer-Verlag, 2004.
On the Agenda(s) of Research on Multi-Agent Learning. by Y.Shoham, R.Powers and T.Grenager. In Proc. 2004 AAAI Fall Symposium on Artificial Multi-Agent Learning
Rationality and Bounded Rationality, by R.J.Aumann. In Games and Economic Behavior, 21, 2-14 (1997)
|2||Mon 2/6||Lecture: Game Theory I (Normal Form Games)||Avail. from the TFs Reading: M.Osborne and A.Rubinstein, A course in Game Theory, MIT Press 1994, pp.11-65||Homework 1 out (due 2/13)|
|3||Wed 2/8||Lecture: Game Theory II (Extensive form games, Repeated games.)||Avail from the TFs Reading: M.Osborne and A.Rubinstein, A course in Game Theory, MIT Press 1994, pp.89-115, 140-146, 160-161, 199-204, 212-218.|
|4||Mon 2/13||Lecture: Game Theory III: Folk theorems; Games of Incomplete information; Bayesian consistency.||
Reading: Avail. from the TFs D.Fudenberg and J.Tirole, Game Theory, MIT Press, 1991, pp.145-160,
Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.209-223
Gibbons, Game Theory for Applied Economists, PUP, 1992.pp.173-183, 224-235
Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.321-350, 364-365
Continuation of lecture notes from 2/8. Also:
|5||Wed 2/15||Lecture: Mechanism Design (Revelation Principle, VCG, truthful mechanisms)||Reading: Mechanism Design, Parkes, Chapter 2 in PhD dissertation, Iterative Combinatorial Auctions: Achieving Economic and Computational Efficiency, Department of Computer and Information Science, University of Pennsylvania, May 2001.||Homework 1 due.|
Homework 2 out (due 2/27)
|6||Wed 2/22||Lecture: Nash implementation||A Crash Course in Implementation Theory, M.O.Jackson in Social Choice and Welfare, Vol. 18, No. 4, 2001, pp 655-708. Not sections 2.9, 4 or 5||
|7||Mon 2/27||Lecture: Sequential decision theory (MDPs, Reinforcement learning.)||Reinforcement Learning: A Survey L.P.Kaelbling, M. L. Littman and A. W. Moore in Journal of Artificial Intelligence Research 4 (1996) 237-285 For class: Sections 1--5 (not 4.1), 6-6.1.1, 6.3||
|Homework 2 due. Homework 3 out (due 3/6)|
|8||Wed 3/1||Discussion: GT Learning||
Survey paper (not for discussion):
Adaptive Heuristics, H.Peyton Young to appear in The New Palgrave, 2nd Edition
Learning Mixed Equilibria , D.Fudenberg and D.Kreps in Games and Economic Behavior 5, 320-367 (1993) pp.320-333, 338-341;
On the non-convergence of Fictitious Play in Coordination Games, D.Foster and H.P.Young, Games and Economic Behavior 25(1) 79-96. 1998 pp.79-83, 94-95;
The evolution of conventions, H.P.Young, Econometrica 61(1):57-84, 1993 pp.62-63
|9||Mon 3/6||Discussion: AI Learning||
(Intro paper. Not for discussion.) Markov games as a framework for multi-agent reinforcement
learning., M.L.Littman, In Proc. 11th Int. Conf. on Machine Learning, 157--163 1994.
Nash Q-Learning for General-Sum Stochastic Games by Junling Hu and Michael P. Wellman, in Journal of Machine Learning Research 4 (2003)
|10||Wed 3/8||Discussion: GT Learning||
Calibrated learning and Correlated equilibrium
by D.Foster and R.Vohra, in Games and Economic Behavior 21, 40-55 1997
Regret in the On-Line Decision Problem, by D.Foster and R.Vohra, in Games and Economic Behavior 29, 7--35 (1999). PAGES 7--21 ONLY
|11||Mon 3/13||Discussion: AI Learning||
Correlated Q-learning. K.Hall and A.Greenwald,
in Proc. 20th Int. Conf. on Machine Learning. 242-249, 2003
Cyclic Equilibria in Markov Games, M.Zinkevich, A.Greenwald and M.L.Littman, in Proc NIPS 2005. NOT APPENDIX.
|12||Wed 3/15||Discussion: GT Learning||Rational learning leads to Nash equilibrium,
by E.Kalai and E.Lehrer, Econometrica 61, 1019-1045 1993. PAGES 1019--1035 ONLY
Beliefs in Repeated Games, by John Nachbar, Econometrica. 73(2) 459-480. 2005. PAGES 459--474 ONLY
|Ivo Parashkavov, Florin Constantin|
|13||Mon 3/20||Discussion: AI Learning||Efficient Learning Equilibrium, R.Brafman and M.Tennenholtz. in Artif. Intell. 159(1-2): 27-47 (2004)||Neal Gupta|
|14||Wed 3/22||Discussion: Nash memory||A Novel Method for Strategy Acquisition in N-Player Games,
S. Phelps, M. Marcinkewitz, S. Parsons and P. McBurney. To appear in Proc.
A Game-Theoretic Memory Mechanism for Coevolution Sevan C. Ficici and Jordan B. Pollack in Proc. GECCO 2003.
|Evan Sprecher, Angela Sze|
|15||Mon 4/3||Discussion: AI Learning|| On the Agenda(s) of Research on Multi-Agent
Learning. by Y.Shoham, R.Powers and T.Grenager.
In Proc. 2004 AAAI Fall Symposium on Artificial Multi-Agent Learning
Learning against opponents with bounded memory. by R.Powers and Y.Shoham, In Proc. IJCAI 2005.
|Philip Hendrix, Ece Kamar
|16||Wed 4/5||Discussion: Applications||We will be discussing applications of the techniques we have covered. Please submit two paragraphs on such potential applications||Applications Action Items|
|17||Mon 4/10||Discussion: Hayek machine||Evolution of Cooperative Problem solving in an artificial economy, E.B.Baum and I.Durdanovic in Neural Computation 12 (12): 2743-2775 (2000)||Quang Duong
|18||Wed 4/12||Discussion: Partial Intermediation||On partially-controlled multiagent systems, R.Brafman and M.Tennenholtz in Journal of Artificial Intelligence Research, 4:477-- 507, 1996.||Katy Milkman
|19||Mon 4/17||Discussion: Inverse RL||Algorithms for Inverse Reinforcement
Learning, A.Y.Ng and S.Russell in Proc. ICML 2000.
Apprenticeship Learning via Inverse Reinforcement Learning, P.Abbeel and A.Y.Ng, in Proc. ICML 2004
|Chih-Han Yu, Ian Rose||
Project proposals due
|20||Wed 4/19||Discussion: Collective Design||Overcoming Communication
Restrictions in Collectives, by K.Tumer and A.K.Agogino, in Proc. IJCNN 2004. THROUGH SECTION II.B ONLY
Learning Sequences of Actions in Collectives of Autonomous Agents, by K.Tumer, A.K.Agogino and D.H.Wolpert, in Proc. AAMAS'02. PAGE 1, AND THEN 2.3 ONWARDS ONLY.
Reinforcement Learning in Large Multi-Agent Systems, A.Agogino and K.Tumer in Proc. AAMAS'05. WHOLE PAPER!
|Sheel Ganatra, Charlie Frogner|
|21||Mon 4/24||Discussion: Coordinated RL||Q-Decomposition for Reinforcement Learning
Agents, S.Russell and A.L.Zimdars,
in Proc. 20th Int. Conf. on Machine Learning (ICML-2003)
An Overview of MAXQ Hierarchical Reinforcement Learning, by Dietterich, T. G. (2000) In B. Y. Choueiry and T. Walsh (Eds.) Proceedings of the Symposium on Abstraction, Reformulation and Approximation SARA 2000, Lecture Notes in Artificial Intelligence (pp. 26-44), New York: Springer Verlag.
|Erik Schultink, Jimmy Sun|
|22||Wed 4/26||Discussion: Nash implementation in CS||On Decentralized Incentive Compatible Mechanisms for Partially Informed Environments, by Ahuva Mu'alem, in Proc. EC-05||Neil Mehta and Ariel Kleiner
|23||Mon 5/1||Discussion: Repeated Games in CS||Repeated-Game Modeling of Multicast Overlays, by M.Afergan and R.Sami to appear in IEEE INFOCOM 2006.||Rohan Murty and Joshua Dezube|
|Wed 5/3||Student Presentations, 4-5.30pm||Project presentations|
|Wed 5/17||Final projects due, noon||Projects due|