SCHOOL OF ENGINEERING AND APPLIED SCIENCES
HARVARD UNIVERSITY

CS 266: Biologically-inspired Distributed and Multi-agent Systems

Prof. Radhika Nagpal
Fall 2007


Course Description   Assignments and Guidelines   Schedule and Reading List   Paper Reviews   Project Milestones and Suggestions   Fall06   Staff  

Class Final Project

This page lists several final project ideas, as well as guidelines for each of the three assignment milestones. Projects must be done in groups of two students.

You can also come up with your own project. A good project is one that fits in with your interest and research goals as well as this class; feel free to propose projects that cover side-areas you would like to investigate or that complement your current research. Also, your final project does not need to be a "research" project. It can also be to write a review of research in some subfield or to help design a system for teaching decentralized concepts, or even to build hardware to demonstrate any of the concepts in class.

A key part of doing research is being able to formulate "chunks" of work that you can investigate within reasonable amount of time as a step towards a longer goal. This is especially important for those of you who are graduate students. Part of the exercise in this class will be to formulate reasonable size projects for yourself - you propose what you will do, what you will deliver, how you will evaluate it. Many of the suggestions below are fairly open-ended and will require background research and real thought to turn into an achievable semester long project.

Lastly, start thinking about your projects EARLY! Projects must be done in pairs so this is another thing to think about early on. We will post introductions and have a class picnic to facilitate this process.

Project Milestones

Due Dates:

  • 9/27: Class picnic
  • 10/16: Milestone 1 (proposal)
  • 11/20: Milestone 2 (project update)
  • 12/18: Milestone 3 (presentations and brief results)
  • 1/14: Milestone 4 (final paper due)

Class Picnic: Think about projects and partners
The fourth day of class we will have a picnic and for part of the time I will talk about some of the suggested projects and projects from last term. We will also do "introductions", but make sure you have already looked at the students introductions on the discussion page (and said something about yourself too).

Milestone 1 (due 10/16): Project Proposal
Submit a (<) 3-page proposal on your class project. Your proposal should contain (1) A succinct description of the problem/project you are investigating and its significance (2) A description of background/related work (3) A plan of execution along with a list of 2 week milestones (4) A short bibliography (5) One sentence about where you'd eventually like to go with this project. (6) Your and your partner's names.

In setting a plan and milestones, keep a couple of points in mind. First you should feel free to set milestones that come with the caveat "if all goes well upto this point". Second, make sure that you build in evaluation points into your milestones - how will you evaluate success at intermediate steps? Also for joint person projects, give some indication (and thought) as to who will be doing which parts.

Submissions should be in PDF and be emailed to rad@eecs as well as ianrose@eecs. Same is true for the other assignments.

Milestone 2 (due 11/20): Project Update
The goal of the update is to report your progress so far on achieving your milestones and to bring up any issues/problems that you are experiencing. The update will consist of TWO PARTS: (a) an inclass 5 minutes presentation and (b) a 1-2 page document. The document should have the following parts: (1) Problem statement, 1 paragraph (2) progress on milestones that you defined in your proposal (3) any issues or questions you have moving forward. The presentation should also cover the same points, but necessarily in less detail; I suggest no more than 1 slide for problem statement (to refresh peoples' memory on what you are doing) and 2-3 slides on progress and difficulties so far. Remember that you only have 5 minutes so please plan accordingly!

Please upload the document in PDF format to the iSites site before class. Please e-mail the presentation in either PDF or PowerPoint format to ianrose@eecs so that all of them can be put on a single laptop to make transitions easier. I would like all presentations by 8:00am that morning to make my life easier, but we will be lenient if they are later than this. The beginning of class, however, is a firm deadline for both the presentations and the document submissions.

Milestone 3 (due 12/18): Presentations and Brief Results
Your class project should essentially be complete by the presentation date. We will have short presentations by each group on their final project. You will have 8 minutes to present and 2 minutes for questions. This is a strict time limit! Please send your presentation (ppt format, and movies if you have any) to the TF the night before so that we can do all the presentations from a single laptop.

We suggest having a single presenter and at most 15 slides (possible outline: high-level goal and motivation, problem statement, your approach, progress/results).

In addition you should submit a short document (2-5 pages) BEFORE CLASS to present a summary of your results (e.g. simulations, analysis, performance graphs, etc) since the project update. The idea is of the 2-page report is to give me a status on how far you got by Dec 18. The final paper is due in reading period, so that you can do your paper writing after the presentation deadline. However the goal is for the project work to be completed by Dec 18. That is what we will grade you on.

Final paper (due 1/14):
Submit a 10 page paper describing your project. Also separately submit a text email to me with title, authors, abstract (200 words) that we can use for the webpage.

Suggestions for Final Class Project

The projects below are a small set of problems that I find interesting. But they are a biased set and could end up being trivial or impossible! More importantly, they are only a sliver of what you can do. If you choose to do one of the suggested projects, let me know. As you may have seen, some projects from previous years have been published --- however as a caution, that usually takes a non-trivial amount of work beyond this class. Still its a fun possible outcome.

Understanding how to conduct research productively --- how to define the problem, limit the scope, structure the milstones, create an evaluation strategy --- is an important objective of this course. Please read the guidelines carefully and use the milestones to your advantage.

In order to implement and evaluate many of these ideas it is useful to have simulators. We will provide instructions and short tutorials if necessary with two simulators, Swarm and Peersim, which we recommend that you use for your final project.

ROBOT SWARMS

Dynamic Formations for Robot Swarms

In [Cheng et al 2005] it was shown how a robot swarm could assemble into a desired static formation by combining two simple group behaviors: contained-gas diffusion and local coordinate formation. This approach is attractive because the formation automatically self-repairs when robots are adedd or removed, by adjusting inter-robot density. Last year [CK Lin and Ian Rose, cs266 final project] showed how to extend this so that the formation moves towards a goal, like a coordinated flock, and is able to flow through obstacle fields.

PROJECT: Adapt the approach to work for DYNAMIC SHAPES. In biology, fish schools achieve not only static formations but also dynamic formations (for example, see the cool fountain head effect [SBS, chapter 11]). How can one achieve similar group behaviors in a robot swarm? Is there a simple way to specify a "dynamic" shape?

References
Robust and Self-repairing Formation Control for Swarms of Mobile Agents (aka Shapebugs), Jimming Cheng, Winston Cheng, Nagpal, National Conference on Artificial Intelligence (AAAI '05), July 2005.
Coordinated Swarm Movement with Obstacle Avoidance, CK Lin, Ian Rose, CS266 Final project 2006.
Flocks, Herds, and Schools: A Distributed Behavioral Model, C. Reynolds, SIGGRAPH, 1987.
Self-Organizing Biological Systems (course library), Chapter 11, "Fish Schooling".

Swarm Programming Language

Mataric's work [Mataric 1995] on basis behaviors is one of the few attempts at creating a language for programming robot swarms, by defining a set of low-level primitives for group behavior. However there is no systematic way to create programs for given high level goals and the result of composing the primitives is often hard to predict. The problem of "synthesis" remains open and important, as evidenced by the recent MURI grant proposal (see page 19) on swarms. Work in Amorphous Computing [Nagpal et al 2003] suggests that looking for high level programming languages that describe classes of tasks may make the global-to-local problem tractable. Recently Chris Jones and others (Mataric Group, USC) have amade some progress on principled ways to program swarms.

PROJECT: Design a swarm programming language. You can pick a restricted domain, for example exploration and manipulation of unstructured environments (example environmental monitoring, search and rescue). What are the appropriate primitives and what is a good language for describign tasks? How would the high-level description map to low-level robot behaviors? Similar efforts in sensor network languages (e.g. TinyDB) may provide some interesting guidance.

References
Designing and Understanding Adaptive Group Behavior, M. Mataric, Adaptive Behavior 4:1, Dec 1995, 51-80.
Programming Methodology for Biologically-Inspired Self-Assembling Systems, Nagpal et al, AAAI Spring Symp., 2003.
Stupid Robot Tricks: A Behavior-Based Distributed Algorithm Library for Programming Swarms of Robots James McLurkin, Master's Thesis, M.I.T., 2004.
Darpa MURI 2005 call for proposals (see page 19)
TinyDB, Madden et al, TODS 2005.

Exploration and Foraging with Microinsect Swarms

One benefit of small robots is that they can get to places that larger ones can't; they can navigate through small holes (body size) and over (and up!) obstacles many times their size. For example, the cockroach-like microrobot being built in Prof Robert Wood's lab would be able to move through rubble and climb up the side of a couch. A swarm of such microinsects could be programmed to operate in complex and dangerous environments and collectively achieve tasks with reliability even though the individuals are cheap, unreliable and disposable. One example task is exploring a unknown environment and locating "hotspots" -- for example locating a "survivor" in a search and rescue operation, or locating "cracks" in a structure that need to be repaired. The next step would be to recruit more microrobots and make a trail from the nest (base) to the hotspot, perhaps to lead rescuers or repair robots to the hotspot. Once that is taken care of, exploration can resume.

PROJECT: design ant-inspired foraging algorithms for microrobots. Although ant algorithms seem to be a good match [see SI], its is unclear how to implement pheromones [Koenig] and the microrobots pose many restrictions on individual capability. However it might be possible to trade "multiplicity" for intelligence - can one use some fraction of microrobots as stationary pheromones? or can pheromone values be transmitted through gossip. How do we adapt ant behaviors to work with ant-like microrobot?

References
Prof Robert Wood's lab at Harvard. (If you are interested in this project, we'd like to give you more details)
Terrain Covering Ant Robots, S. Koenig, Georgia Tech.
Swarm Intelligence [SI], Chapter 2 (course library).
Autonomous Robots, chapter on multi-robot control, Bekey, MIT Press, 2005. (course library)

BIO-INSPIRED MULTI-AGENT MODELS

The idea of topics in this area is to explore what kinds of principles can be adapted from biology into an artificial context. As a first step, rather than solve the "general" problem all at once its good to pick a example shape or structure and try to solve it first. Then see if it can be generalized to a larger class of structures. (note: this hint goes for most research projects..). Many of the Amorphous Computing examples can be thought of in this light.

Hydra-like Regeneration:

Regeneration in biology is quite spectacular. Many organisms can regenerate lost limbs and eyes and even regenerate bodies through growth; this has been explored in Amorphous Computing langauges such as [Kondacs 03]. But another much less well udnerstood type of regeneration is Morphallaxis is a kind of regeneration where when part of a structure is removed, the remaining structure reorganizes into a smaller version of the same shape. For example, this is how the hydra regenerates [Wolpert 02] -- if you cut off the head, it develops a new head and foot that are proportionally correct for the remaining body. There are some interesting ideas discussed on how one could achieve such regeneration on simple patterns like the "french flag" pattern.

PROJECT: demonstrate an artificial 2D pattern system that can rearrange its pattern when part of the region is removed. For example, can the simple "inverter" pattern in OSL [Nagpal 2002] adapt if half of the field is removed forming some other pattern or shape? Is it possible to articulate a organizing principle for morphallaxis, in a similar way that the rule of normal neighbors explains limb regeneration?

References
Principles of Development, Chapter on Regeneration, Wolpert, 2002. (course library)
Hydra regeneration description.
Programmable Self-Assembly Using Biologically-Inspired Multiagent Control, R. Nagpal, AAMAS 2002.
Biologically-inspired Self-Assembly of 2D Shapes, Using Global-to-local Compilation, A. Kondacs, IJCAI 2003

Global-to-local Programming for Branching Patterns

In our body, branching structures are a fundamental piece of design, from our lungs, to cappilaries to salivary glands. Lindenmayer showed how the development of many branching structures, from plants to capillary networks, can be captured by very simple and succinct grammer rules. These rules describe both how the new branches form as well as how the original structure grows, and how the environment affects the structure. However the rules are a high level description, how cells or agents could implement them is unclear.

PROJECT: Implement a global-to-local compiler that starts with some simple Lindemayer grammar for branching and converts it to agent rules (for axample on a 2D amorphous computer [abelson00] or a growing amorphous computer [kondacs03]). Can one at a high level describe the type of branching desired (short fat buds like the lung, or far apart long branches like the capillaries) and automatically derive the agent program?

References
Lindenmayer systems and branching morphogenesis, webpage by Prusinkiewicz et al.
Algorithmic Beauty of Plants, Prusinkiewicz, Lindenmayer, 1990. (course library)
Amorphous Computing, Abelson et al, CACM 2000.

P2P AND SENSOR NETWORKS

There are many problems in peer-to-peer networks and sensor networks where decentralized solutions are important as well as solutions that can easily adapt to nodes entering and leaving on a regular basis. As a result many bio-inpired algorithms are getting increased attention, both in implementation and analysis.

Collective synchronization on networks

Collective synchronization, inspired by the famous Mirollo and Strogatz model of firefly synchronization, is recently being explored for designing protocols for sensor networks. Not only has it been applied to many applications (e.g. distributed antennas, slot synchronization), but it also has led to novel algorithms such as "DESYNC" [degesys07]. Here are some ideas for more projects along that line of research.

  • Synchronization on mobile networks: How well does the Mirollo and Strogatz (MS) model work if the nodes are mobile, for example a network of mobile robots? Are there better ways to design the neighbor synchronization function so that the behavior and performance is improved?

  • Frequency-adapting synchronization for distributed sensing: Synchronization can be used to control a distributed sensor network that is sampling a large region on a periodic basis. However the rate of sampling (or rather the rate of synchronization pulsing) effects both acuracy and energy in opposing ways. The more frequently nodes pulse and adjust, the less clock skew can cause sample times to diverge, but this uses up more energy. The heart plays an interesting game -- when we sleep the heartbeat slows, but when an alarmign event happens the heartbeat quickly increases. Design a similar system that samples data at a low base rate but then can rapidly increase the rate when interesting events happen.

  • Applications of DESYNC: Desynchronization creates a round-robin pattern and has been applied to scheduling communication. But it can also be applied to other problems: (1) Determining "sleep cycles" and sleep durations for a set of nodes that cover the same area. How would you implement this, given that a node can not listen while it is sleeping. Can this work in a multi-hop setting? (2) Interleaving circuits for round-robin timing. If you are interested in circuit design, can desync be implemented in hardware circuit? If so it could be used for interleaving slow A2D converters to create a fast one, or for interleaving access to a shared bus.

References
Synchronization of pulse-coupled biological oscillators, Mirollo, R.and Strogatz, S., SIAM J. Appl. Math. 50: 1645-1662, 1990;
Decentralized Synchronization Protocols with Nearest Neighbor Communication, Lucarelli, Wang, Sensys 2004.(multi-hop sensor networks)
Firefly-inspired Heartbeat Synchronization in Overlay Networks, Ozalp Babaoglu, Toni Binci, Mark Jelasity, SASO 2007.(p2p networks)
DESYNC: Self-Organizing Desynchronization and TDMA on Wireless Sensor Networks. Degesys et al, IPSN 2007.

Stigmergy-based routing protocols for P2P systems

In the P2P world, publications on structured overlays greatly outnumber those on unstructured overlays, despite the fact that in practice unstructured overlays have been and are quite popular. Research some of the current strategies employed by unstructured overlays and the perceived weaknesses of these systems. What new search & routing algorithms can be developed to addresses these weaknesses? What if the "rules of the game" are changed - e.g. what if you consider all peers to be trustworthy and thus each peer allocates a small amount of disk space to be publicly writable; does this open the door to stigmergy-like protocols? If so, can you relax the assumption of perfect trustworthiness at all?

References:
Making Gnutella-like P2P Systems Scalable, Chawathe et al.
Characterizing Unstructured Overlay Topologies in Modern P2P File-Sharing Systems, Daniel Stutzbach and Reza Rejaie

P2P energy management Decentralized power generation + distributed sensing + adaptive electricity output & usage = A better power grid. At least that is the general (nutshell) idea of the "Smart Grid". Research this idea, including what progress has been made and what problems remain to be solved. Design a decentralized algorithm for this domain. How autonomous can individuals be - i.e. how much do I need to coordinate with the power company? Does this transformation require a top-down approach (e.g. driven by the Federal govt, or at least by large corporate consortiums)? Can decentralized "smart" grid achieve a higher efficiency?

References:
Mazza, Patrick, "The Smart Energy Network: Electricity's Third Great Revolution," c. 2003 http://climatesolutions.org/pubs/pdfs/SmartEnergy.pdf

(Note: This is more of an industry-driven movement than an academia-driven movement so you keep that in mind when searching for papers and references)

Global-to-local Programming for P2P and sensor networks.

Recently there has been some significant work on designing and implementing global "general purpose" languages for sensor networks. These general purpose languages reflect well studied programming models, such as Databases (SQL) and Stream Processing (Proto, Regiment), and are supported by underlying distributed algorithms. One limitation is that they have focussed very strongly on sensing as an application (e.g. gathering information about a region and relaying it to an observer). They have not looked as strongly at "distributed actuation", where the end result of a computation may affect different parts of the network. As an example, consider temperature sensors that also control heaters, or more generally a smart building that can control its environment and direct its inhabitants. How does one adapt these programmign mdoels to include actuation - or does one need completely diffeernt model?

PROJECT: Consider a "smart" building environment. Design algorithms and ideas for how to describe tasks in this environment.

PROJECT2: Many branching structures in our body constantly adapt to the needs in the environment - for example capillary networks are influenced by the need and level of oxygenation and this drives the growth and death of branches. Can one design a global programming languages that describes such "environmentally-adaptive" shapes, rather than static structures?

References
Infrastructure for Engineered Emergence on Sensor/Actuator Networks (proto), Beal, Bachrach, IEEE Intelligent Systems, (Vol. 21, No. 2) 2006.
TinyDB, Madden et al, TODS 2005.

MATHEMATICAL ANALYSIS OF SWARM ALGORITHMS

Recently, several groups have made progress is showing how canonical swarm behaviors, like firefly synchronization and bird flocking, can be thought of as concensus problems in networked multi-agent systems with "nearest-neighbor" type interactions. This analysis uses techniques from graph theory, markov chains, control theory, etc and appears to be quite generally applicable.

PROJECT: Analyze collective transport using this methodology. Collective transport, where ants collectively move large and irregular objects, relies on very simple agent/ant rules and yet appears to be very robust. Can collective transport be thought of and analyzed as a distributed concensus problem?

PROJECT2: Design and analyze a "ant" algorithm that would lead to collective bridge building (as we saw in the video where ants connect to build a bridge to the honey). Is it mathematically the same as collective transport?

References
Swarm Intelligence [SI], Chapter on Collective Transport (course library).
Consensus and Cooperation in Networked Multi-Agent Systems, R. Olfati-Saber, J. A. Fax, and R. M. Murray., Proceedings of the IEEE, vol. 95, no. 1, pp. 215-233, Jan. 2007.
Flocking for Multi-Agent Dynamic Systems: Algorithms and Theory. R. Olfati-Saber, IEEE Trans. on Automatic Control, vol. 51(3), pp. 401-420, Mar. 2006.

EVOLVING MULTI-AGENT RULES

Evolutionary methods, such as genetic algorithms and genetic programming, provide ways to do general optimization. But they also provide ways to search the landscape of solution space for a given problem. In the latter case, such methods can reveal unexpected solutions and new insights into a proble. However, such methods have been used mostly for single agent systems, evolving multi-agent rules is less common.

PROJECT: Use genetic programming to evolve agent rules in a well-defined space for which we have current hand-designed solutions.

  • For example, take the collective construction by robot swarms [Werfel 06] and assume that an unspecified number of robots will build a fixed size cube. Can one evolve rules such that these robots avoid gaps while building? If the blocks are inert (cannot store information), what is the best that robots can do?
  • Consider firefly (pulse-coupled) synchronization -- given the same agent models/capabilities, what type of algorithm does evolutionary methods produce?

References
Fundamentals of Natural Computing, Chapter 3, de Castro, CRC 2007 (will be handed out in class or scanned in soon).
What's AI done for me lately? Genetic programming's human-competitive results, Koza et al, IEEE Intelligent Systems, May 2003.
Evolving cellular automata to perform computations: A review of recent work Mitchell, Crutchfield, and Das.
Synchronization of pulse-coupled biological oscillators, Mirollo, R.and Strogatz, S., SIAM J. Appl. Math. 50: 1645-1662, 1990;
Distributed Construction by Mobile Robots with Enhanced Building Blocks, Werfel, Bar-Yam, Rus, Nagpal, ICRA 2006.

HARDWARE PROJECTS

Hardware projects are also allowed, for example (a) implementing any of the bio-inspired approaches on a real network (e.g. MoteLab or PlanetLab) or real robot (e.g. LegoNXT or iCreate) (b) building actual hardware, for example building a low-cost robot for a robot swarm.

Note that to accomplish a hardware project in such a short amount of time, you must already have significant background experience -- e.g. designing circuit boards, or mechanical parts, or even lego building -- based on what you propose to do. We can provide support but not actual hands-on background/lab training.

PROJECTS involving robots:

  • Design a low-cost robot for a robot swarm, that can achieve the propeties of the proprietary iRobot swarm [McLurkin05]. There are several low-cost candidate platforms being explored, for example SUMO, and SMURV, however the inter robot communication and coordination is missing. This project assumes you are comfortable building controllers for sensors and robots.
  • Implement (a) RFID reader/writer or (b) IR ring communication for the Lego NXT robot. The first allows one to implement pheromone based stigmergy using lego robots. There is some precidence here, requires experience in EE circuit board design and PIC programming. The latter allows one to implement formation control, and will require significantly more EE experience. The advantage of the lego robot is that its form is highly flexible -- we could use this with legged robots or wheeled ones with equal ease.

References
Stupid Robot Tricks: A Behavior-Based Distributed Algorithm Library for Programming Swarms of Robots James McLurkin, Master's Thesis, M.I.T., 2004. (description of the iRobot Swarm)
SUMO robot and SMURV robots, based on the ICreate platform (rhoomba)
The Lego NXT mindstorms website