| Lec. No. |
Date |
Key |
Topic |
Readings |
Lecture notes |
Assignments |
| 1 |
Wed 2/1 | |
Lecture: Introduction |
Background reading:
On Learnable
Mechanism Design, by D.C.Parkes.
In Collectives and the Design of Complex Systems, Kagan Tumer and David Wolpert (eds.) , pages 107-131, Springer-Verlag, 2004.
On the Agenda(s) of Research on Multi-Agent
Learning. by Y.Shoham, R.Powers and T.Grenager.
In Proc. 2004 AAAI Fall Symposium on Artificial Multi-Agent Learning
Rationality and Bounded Rationality,
by R.J.Aumann. In Games and Economic Behavior, 21, 2-14 (1997)
|
|
|
2 |
Mon 2/6 | |
Lecture: Game Theory I (Normal Form Games) |
Avail. from the TFs
Reading: M.Osborne and A.Rubinstein, A course in Game Theory,
MIT Press 1994, pp.11-65
|
|
Homework 1 out (due 2/13) |
| 3 |
Wed 2/8 | |
Lecture: Game Theory II (Extensive form games, Repeated games.) |
Avail from the TFs Reading: M.Osborne and A.Rubinstein, A course in Game Theory,
MIT Press 1994, pp.89-115, 140-146, 160-161, 199-204, 212-218.
|
|
|
| 4 |
Mon 2/13 | |
Lecture: Game Theory III: Folk theorems; Games of Incomplete information; Bayesian consistency. |
Reading: Avail. from the TFs D.Fudenberg and J.Tirole, Game Theory, MIT Press, 1991, pp.145-160,
203-206
Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.209-223
Gibbons, Game Theory for Applied Economists, PUP, 1992.pp.173-183, 224-235
Fudenberg and Tirole Game Theory, MIT Press, 1991. pp.321-350, 364-365
|
Continuation of lecture notes from 2/8. Also:
Hand-written notes:
|
|
| 5 |
Wed 2/15 | |
Lecture: Mechanism Design (Revelation Principle, VCG, truthful mechanisms) |
Reading:
Mechanism Design, Parkes, Chapter 2
in PhD dissertation, Iterative Combinatorial Auctions: Achieving Economic and
Computational Efficiency,
Department of Computer and Information Science, University of
Pennsylvania, May 2001.
|
|
Homework 1 due.
Homework 2 out (due 2/27)
|
|
Mon 2/21 | |
Holiday.
|
|
|
| 6 |
Wed 2/22 | |
Lecture: Nash implementation |
A Crash Course in Implementation Theory, M.O.Jackson in
Social Choice and Welfare, Vol. 18, No. 4, 2001, pp 655-708. Not sections 2.9, 4 or 5
|
Hand-written notes:
|
|
| 7 |
Mon 2/27 | |
Lecture: Sequential decision theory (MDPs, Reinforcement
learning.) |
Reinforcement Learning: A Survey
L.P.Kaelbling, M. L. Littman and A. W. Moore
in Journal of Artificial Intelligence Research 4 (1996) 237-285
For class: Sections 1--5 (not 4.1), 6-6.1.1, 6.3
|
RL Examples
|
Homework 2 due.
Homework 3 out (due 3/6)
|
| 8 |
Wed 3/1 | |
Discussion: GT Learning |
Survey paper (not for discussion):
Adaptive Heuristics, H.Peyton Young to appear in The New Palgrave, 2nd Edition
For discussion:
Learning Mixed Equilibria ,
D.Fudenberg and D.Kreps in Games and Economic Behavior 5, 320-367 (1993) pp.320-333, 338-341;
On the non-convergence of Fictitious Play in
Coordination Games, D.Foster and H.P.Young,
Games and Economic Behavior
25(1) 79-96. 1998 pp.79-83, 94-95;
The evolution of conventions, H.P.Young, Econometrica 61(1):57-84, 1993 pp.62-63
|
Julius Degesys |
|
| 9 |
Mon 3/6 | |
Discussion: AI Learning |
(Intro paper. Not for discussion.) Markov games as a framework for multi-agent reinforcement
learning., M.L.Littman, In Proc. 11th Int. Conf. on Machine Learning, 157--163 1994.
For discussion:
Nash Q-Learning for General-Sum
Stochastic Games by Junling Hu and Michael P. Wellman, in
Journal of Machine Learning Research 4 (2003)
|
Ilan Lobel
|
|
| 10 |
Wed 3/8 | |
Discussion: GT Learning |
Calibrated learning and Correlated equilibrium
by D.Foster and R.Vohra, in Games and Economic Behavior 21, 40-55 1997
Regret in the On-Line Decision Problem, by D.Foster and R.Vohra, in Games and Economic Behavior 29, 7--35 (1999). PAGES 7--21 ONLY
|
James Burns
|
|
| 11 |
Mon 3/13 | |
Discussion: AI Learning |
Correlated Q-learning. K.Hall and A.Greenwald,
in Proc. 20th Int. Conf. on Machine Learning. 242-249, 2003
Cyclic Equilibria in Markov
Games, M.Zinkevich, A.Greenwald and M.L.Littman, in Proc NIPS 2005.
NOT APPENDIX.
| Haoqi Zhang
|
|
| 12 |
Wed 3/15 |
|
Discussion: GT Learning |
Rational learning leads to Nash equilibrium,
by E.Kalai and E.Lehrer, Econometrica 61, 1019-1045 1993. PAGES 1019--1035 ONLY
Beliefs in Repeated Games, by John Nachbar,
Econometrica. 73(2) 459-480. 2005. PAGES 459--474 ONLY
|
Ivo Parashkavov, Florin Constantin
|
|
|
| 13 |
Mon 3/20 | |
Discussion: AI Learning |
Efficient Learning Equilibrium, R.Brafman and M.Tennenholtz. in Artif. Intell. 159(1-2): 27-47 (2004)
|
Neal Gupta
|
|
-->
| 14 |
Wed 3/22 | |
Discussion: Nash memory |
A Novel Method for Strategy Acquisition in N-Player Games,
S. Phelps, M. Marcinkewitz, S. Parsons and P. McBurney. To appear in Proc.
AAMAS'06.
A Game-Theoretic Memory Mechanism
for Coevolution Sevan C. Ficici and Jordan B. Pollack in Proc. GECCO 2003.
| Evan Sprecher, Angela Sze
|
|
|
| |
Spring break. |
| 15 |
Mon 4/3 | |
Discussion: AI Learning |
On the Agenda(s) of Research on Multi-Agent
Learning. by Y.Shoham, R.Powers and T.Grenager.
In Proc. 2004 AAAI Fall Symposium on Artificial Multi-Agent Learning
Learning against opponents with bounded memory.
by R.Powers and Y.Shoham, In Proc. IJCAI 2005.
|
Philip Hendrix, Ece Kamar
|
|
| 16 |
Wed 4/5 | |
Discussion: Applications |
We will be discussing applications of the techniques we have
covered. Please submit two paragraphs on such potential applications
|
Applications
Action Items
|
|
| 17 |
Mon 4/10 | |
Discussion: Hayek machine |
Evolution of Cooperative Problem solving in an artificial
economy, E.B.Baum and I.Durdanovic in Neural Computation 12 (12): 2743-2775 (2000)
|
Quang Duong
|
|
| 18 |
Wed 4/12 | |
Discussion: Partial Intermediation |
On partially-controlled multiagent systems, R.Brafman
and M.Tennenholtz in Journal of Artificial Intelligence Research, 4:477-- 507, 1996.
|
Katy Milkman
|
|
| 19 |
Mon 4/17 | |
Discussion: Inverse RL |
Algorithms for Inverse Reinforcement
Learning, A.Y.Ng and S.Russell in Proc. ICML 2000.
Apprenticeship Learning via Inverse Reinforcement
Learning, P.Abbeel and A.Y.Ng, in Proc. ICML 2004
|
Chih-Han Yu, Ian Rose
|
Project proposals due |
| 20 |
Wed 4/19 | |
Discussion: Collective Design |
Overcoming Communication
Restrictions in Collectives, by K.Tumer and A.K.Agogino, in Proc. IJCNN 2004. THROUGH SECTION II.B ONLY
Learning Sequences of Actions
in Collectives of Autonomous Agents, by K.Tumer, A.K.Agogino and D.H.Wolpert, in Proc. AAMAS'02. PAGE 1, AND THEN 2.3 ONWARDS ONLY.
Reinforcement Learning in Large Multi-Agent
Systems, A.Agogino and K.Tumer in Proc. AAMAS'05. WHOLE PAPER!
|
Sheel Ganatra, Charlie Frogner |
|
| 21 |
Mon 4/24 | |
Discussion: Coordinated RL |
Q-Decomposition for Reinforcement Learning
Agents, S.Russell and A.L.Zimdars,
in Proc. 20th Int. Conf. on Machine Learning (ICML-2003)
An Overview of MAXQ Hierarchical Reinforcement Learning, by Dietterich, T. G. (2000) In B. Y. Choueiry and T. Walsh (Eds.) Proceedings of the Symposium on Abstraction, Reformulation and Approximation SARA 2000, Lecture Notes in Artificial Intelligence (pp. 26-44), New York: Springer Verlag.
|
Erik Schultink, Jimmy Sun |
|
| 22 |
Wed 4/26 | |
Discussion: Nash implementation in CS |
On Decentralized Incentive Compatible Mechanisms for Partially Informed Environments, by Ahuva Mu'alem, in Proc. EC-05
|
Neil Mehta and Ariel Kleiner
|
|
| 23 |
Mon 5/1 | |
Discussion: Repeated Games in CS |
Repeated-Game Modeling of Multicast Overlays, by M.Afergan and R.Sami
to appear in IEEE INFOCOM 2006.
|
Rohan Murty and Joshua Dezube |
|
| 24 |
Wed 5/3 | |
Conclusions | | |
|
|
Wed 5/3 | |
Student Presentations, 4-5.30pm |
|
|
Project presentations |
|
Wed 5/17 | |
Final projects due, noon |
|
|
Projects due |
