CS 222 -- Algorithms at the End of the Wire

Handouts and Class Materials

Here are some old notes:
Notes 1 (Markov Chains)
Notes 2 (Entropy)
Notes 3 (Arithmetic coding)
Notes 4 (LZ schemes + Arithmetic coding)
Notes 5 (Burrows-Wheeler)
Notes 6 (SEQUITUR)

Announcements

[4/30] In class we had a sign-up sheet for projects. If you didn't sign up for a time, please see me and do so.
[4/30] Officially, you owe me a project update on 4/30. If you forgot about that, get one to me as soon as possible -- a printout left under my door is fine.
[4/7] Assignment 3 is now up. Please check it out.
[3/9] Assignment 2 is now up. Please check it out. OOPS! It's not due March 26, which is during break! It's due April 2 the week you get back from break!!
[3/1] Project description is now available. Please read it!
[2/13] Assignment 1 is now up. Please check it out.
[1/29] Due to the seminar nature of the course, we will not be having regular sections.
[1/29] In past years, this class was also given via the extension school, and lectures were taped. Lectures are NOT being taped this year, so attendance is obviously more important.

Basic Class Information

Assignments

Assignment 1
Assignment 2 (Not due 3/26, due 4/2.)
Assignment 3 (Not due 3/26, due 4/2.)

Notes

How to read a research paper notes.

Readings

All reading dates are tentative and will be confirmed in class. We may go faster, we may go slower. You should consider this a minimum of what you should be reading for this class. Feel free to explore on the Web or otherwise (additional suggested readings will be listed as well). We are just touching the surface of these topics; there's much more out there.

Unit 1: Search Engines and the Importance of Links

1/29: Background: A Brief History of Generative Models for Power Law and Lognormal Distributions. by Mitzenmacher.
- No summary necessary. Please read this in your spare time; it provides background for a lot of what we'll cover.
- Related paper: Power Laws, Pareto Distributions, and Zipf's Law. by Mark Newman.
1/29: Background: Review Markov chains.
- No summary necessary.
- Start with wikipedia. The first external link is to a useful book chapter on Markov chains.
2/3: Authoritative Sources in a Hyperlinked Environment. by Jon Kleinberg.
- Summary question: How could you possibly improve on Kleinberg's algorithm?
2/3: The PageRank Citation Algorithm. by Brin, Page, Motwani, Winograd.
- Summary questions: How does PageRank differ from Kleinberg's algorithm? How is it the same? What more can you do to improve PageRank?
2/5: In your copious spare time, check out Yahoo Research's list(s) of key scientific challenges .
2/5: Improved Algorithms for Topic Distillation in a Hyperlinked Environment. by Bharat and Henzinger.
- Summary questions: What techniques are suggested for improving on Kleinberg's algorithm? Do they appear worthwhile given the costs?
2/5 Analysis of a Very Large Altavista Query Log. Henzinger, Marais, Moricz, and Silverstein.
- Summary questions: What sort of data do they try to mine from the query log? Which seems the most useful? Can you think of anything they should be looking for but did not?
2/10 Rank Aggregation Methods for the Web. by Dwork, Kumar, Naor, Sivakumar. An alternative version .
- Summary questions: Define Spearman and Kendall distance. Why and how are Markov chains useful for combining rankings?
2/10 PicASHOW: Pictorial Authority Search by Hyperlinks on the Web. by Lempel and Soffer.
- Summary questions: How does PicASHOW relate to Kleinberg's algorithm and PageRank? What are the differences?
2/12 The Eigentrust Algorithm for Reputation Management in P2P Networks by Kamvar, Schlosser, and Garcia-Molina.
2/12 Trust-Based Recommendation Systems by Andersen et al.
- Summary questions: How are the ideas of eigenvalues/PageRank/Markov chains used in the trust setting?
2/17 The Link Prediction Probelm for Social Networks by Liben-Nowell and Kleinberg. No summary needed.
2/17 The Polymath Project on Gowers's blog. Please look over and we will discuss. *** I expect you to look at more than that 1 blog post -- go forward from there! Get to a page or two with actual math arguments! Or go to Terrence Tao's blog and look at an "offshoot" argument. (Terrence Tao's blog is interesting in its own right as well.) Poke around a bit!!! ***
Additional useful papers:
- Graph Structure in the Web by Broder et al.
- The Link Database: Fast Access to Graphs of the Web by Randall et al.
- The publications page of Luis von Ahn . In class we brought up his work on Designing Games with a Purpose and Re-Captcha.
- Assignment 1 is out. See above.

Unit 2: Compression and Basic Information Theory

2/17 + 2/19 We will be covering the basics of compression and information theory, including Huffman coding, arithmetic coding, LZ-style coding, etc. Some good online introductions to the material include:
Introduction to Data Compression, notes by Guy Blelloch.
Information Theory, Inference, and Learning Algorithms, specifically part 1 (Data Compression) of Mackay's Book. (Though the whole book is good.)
2/24 Class cancelled.
2/26 On the Implementation of Minimum-Redundancy Prefix Codes by Moffat and Turpin.
- Summary questions: What is the "standard" way of thinking about implementing Huffman coding, and why is it not the "right" way in practice?
2/26 A Block-Sorting Lossless Data Compression Aglorithm. by Burrows and Wheeler.
- Summary questions: Although the Burrows-Wheeler algorithms is somewhat confusing, the underlying principles behind it are quite sound. Describe these principles.
- A perhaps easier-to-read description can be found at Data Compression with the Burrows-Wheeler Transform. by Mark Nelson.
3/3 Identifying Hierarchical Structure in Sequences: A Linear-Time Algorithm by Nevill-Manning and Witten.
- Summary questions: Explain the basic concept behind SEQUITUR. Why is it good both for compression and determining structure?
3/3 The entropy of English using PPM-based models by Teahan and Cleary.
- Summary questions: What is meant by the entropy of English? How can it be bounded?
[3/1] Project description is now available. Please read it!
3/5 : This will be a catch up day. No summaries/reading, just the background stuff.
3/10 Paper on JPEG.
- Summary questions: See below.
3/10 Wikipedia's take on JPEG ; Wikipedia's take on JPEG 2000; Wikipedia's take on MPEG, especially MPEG-2 (but also 1 and 4).
- Summary questions: Explain the DCT. How does JPEG 2000 differ from the original JPEG. Explain the similarities and differences between MPEG and JPEG. Discuss the reasons for different types of frames.
3/12 On Compressing Social Networks. by various.
- Summary questions: How can we improve this paper?
3/12 Permuting Web Graphs. by Bodli/Martini/Vigna.
- Summary questions: Explain the use of Gray codes in this setting. Explain the utilities of intrinsic and extrinsic information on compression in this setting.
3/12 Whole Program Paths by Jim Larus.
- No summary questions.
Additional useful papers:
- Towards Compressing Web Graphs by Adler and Mitzenmacher.
- The Webgraph Framework 1: Compression Techniques by Boldi and Vigna.
- Algorithms to Accelerate Multiple Regular Expressions Matching for Deep Packet Inspection by Kumar, Dharmapurikar, Yu, Crowley, Turner.
- On the Hardness of Finding Multiple Preset Dictionaries by Mitzenmacher
- Check out the WebGraph home page.

Unit 3: Data Streams and Data Summaries

3/17 Data Streams: Algorithms and Applications. by Muthukrishnan.
- Summary questions: see paper below.
3/17 Computing on Data Streams by Henzinger, Raghavan, and Rajagopalan.
- How is computing on a data stream different from a "standard" input-output algorithm? What are the goals, how do you measure resource usage, what sorts of problems are considered? Where has computing on data streams had successes? What lower bound techniques are there showing that computing on data streams can be hard?
3/19 Network Applications of Bloom Filters: A Survey. by Broder and Mitzenmacher.
- Summary questions: see paper below.
3/19 The Bloomier Filter by Chazelle, Kilian, Rubinfeld, and Tal.
- How does a "Bloomier" filter go beyond a Bloom filter? Can you think of any applications for a Bloomier filter? What properties do these data structures have that make them useful in a variety of settings? What are their flaws?
3/31 Min-wise independent permtuations by Broder, Charikar, Frieze, and Mitzenmacher. Note: just read the first few sections if you want, it's hard going.
- Explain the notion of a sketch. How does min-wise independence compare with other "versions" of independence? Explain variations of min-wise independence.
3/31 Ferret: A toolkit for content-based similarity search of feature-rich data by Lv, Josephson, Wang, Charikar, and Li.
- Does it make sense to have a toolkit for this sort of application? Or do you suspect similarity search is highly content dependent? Explain the difference between filtering and ranking and how they are accomplished in the Ferret toolkit.
4/2 New directions in traffic measurement and accounting: focusing on the elephants, ignoring the mice by Estan and Varghese.
- See below.
4/2 The count-min sketch and its applications by Comorde and Muthukrishnan.
- How are the structures in these papers similar to Bloom filters and how are they different? What are the similarities between this paper and the mice/elephants paper? What are the differences? How much do they matter?
4/7 MapReduce: Simplified Data Processing on Large Clusters
- See below.
4/7 On the Complexity of Processing Massive, Unordered, Distributed Data
- Compare and contrast the theory and practice of the MapReduce paradigm. What kind of tasks might MapReduce not be good for? Are there any eventual scaling problems for this paradigm? Suppose you had access to a large-scale MapReduce system -- what would you most want to use it for?
Additional useful papers:
- The extended version of Muthu's survey by Muthu.
- Compressed Bloom Filters by Mitzenmacher.
- Beyond Bloom Filters: From Approximate Membership Checks to Approximate State Machines, by Bonomi, Mitzenmacher, Panigrahy, Singh, and Varghese.
- Survey of hash-type algorithms for networks stuff.
- Why Simple Hash Functions Work: Using the Entropy in a Data Stream. Mitzenmacher and Vadhan.
- A bibliographic list of streaming papers.
- For fun, read
- Dewitt and Stonebreaker's take on MapReduce.

Unit 4: Coding, and Network Coding

4/9 Coding/Shannon's theorems
Recommended: Information Theory, Inference, and Learning Algorithms, specifically part 2, chaps 8-10 of Mackay's Book. (Though the whole book is good.) Some coding notes.
4/9 Reed-Solomon codes. The class will go over the notes/Shannon's theorem.
4/14 Digital Fountains: A Survey and Look Forward by Mitzenmacher.
- Explain, in your own words, what is a Digital Fountain. How close can we get to a Digital Fountain in practice?
4/14 Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads by Byers, Luby, and Mitzenmacher.
- Do codes offer significant advantages in the setting of downloading in parallel? In what settings? In what settings might they offer few advantages? Where else can you imagine codes offering similar advantages.
4/16 No class.
4/21 Network Coding, an Instant Primer , by Fragouli, Le Boudec,and Widmer.
- What is network coding, and how does it differ from the types of encoding (digital fountain codes, Reed-Solomon, etc.) that we have seen previously? What potential advantages does network coding offer? What are the corresponding costs?
4/21 Network Coding for Large Scale Content Distribution , by Gkantsidis and Rodriguez.
- Is network coding being used for content distribution today? If not, why not, do you think?
4/23 XORs in The air: Practical Wireless Network Coding , by Katti, Rahul, Hu, Katabi, Medard,and Crowcroft.
4/23 Network Coding Meets TCP , by Sundararajan et al.
- Argue based on the papers for today whether network coding will become standard in networking or will be relegated to specialized networks and theoretical work.
4/28 Part 1: Please go to the WWW 2009 page , select a paper, turn in a summary for it, and be prepared to quickly (2 minutes) summarize the paper for the class. (This will happen Tuesday or Thursday depending on time.)
Part 2: Please read this submission . Assume it is for a major networking conference. You've been asked to give a review. Please write a short review (2-3 paragraphs) and be prepared to discuss. You should judge the paper on its innovation, whether it is likely to be useful now or in the immediate future, whether the analysis (theoretical and/or experimental) is correct and well thought out, etc.
4/30 Wrapup
Additional useful papers:
- A link to the informed content delivery paper.
- A link to the Bullet content delivery paper.
- Bram Cohen's initial take on Avalance. And heck, here's his take on on erasure codes generally.
- Look for How Practical is Network Coding, available online.
- For more on practice, I'd recommend checking the publications of Dina Katabi.