CS 222 -- Algorithms at the End of the Wire

Handouts and Class Materials

Announcements

[1/19] I expect sometime next week to be able to send reports to all of you about your reports. Or feel free to come by and talk about them if you wish.
[1/4] Here is the class schedule for talks. Keep in mind you are expected to attend your ENTIRE session. Yes, I know it's not divided equally; the lucky people get to hear five other reports. If you feel left out, please drop by the other session:
Session 1 : 10:15-12:45 MDG125

10:15-10:35 Greg Price and Steve Melendez
10:40-11:00 Joe Barillari and Guille Canas
11:05-11:25 Brad Diephuis and Tom Carriero
11:30-11:50 Enrique Henestroza and Peter Zhivkov
11:55-12:15 Zhenming Liu and Chia-Yung Su
12:20-12:40 Eric Govind and Xavier Larmurier and Sam Fraiberger

Session 2 : 2:55-4:30 MD221

2:55-3:15 Mike Lyons and Steve Chang
3:20-3:40 Hillary Mutisya and Florin Ciocan
3:45-4:05 Ryan Wisnesky and Peter Webb
4:10-4:30 Andrew McCollum
[1/3] Adam will be available tomorrow (Thursday) for assistance on PS 4. Please e-mail him to schedule an appointment.
[12/14] Adam made up some nice notes on channel coding. It will help with the homework.
[12/12] Adam tells me we have MDG125 for the morning project session, and MD221 for the afternoon project session.
[12/12] You should slide your final project under my door by Sunday midnight or so January 14. Sadly, I need grades in by the 17th. Ostensibly, I should read your project before assigning a grade.
[11/24] You won't have summary questions for the Tuesday after Thanksgiving, but they will start up again after that, so look for the questions.
[11/16] Next week Adam will be available for appointments on MONDAY, instead of the standard time, because of the holiday.
[11/9] David MacKay of Cambridge, England will be giving a colloquium talk very related to the theme of the class this Monday, 11/13, at 4:00 pm in MD125. I encourage you all to attend. The abstract follows:
Keyboards are inefficient for two reasons: they do not exploit the predictability of normal language; and they waste the fine analogue capabilities of the user's muscles. I describe Dasher, a communication system designed using information theory. A person's gestures are a source of bits, and the sentences they wish to communicate are the sink. We aim to maximize the number of bits per second conveyed from user into text.
Users can achieve single-finger writing speeds of 35 words per minute and hands-free writing speeds of 25 words per minute.
Dasher is free software, and it works in over one hundred languages.
http://www.inference.phy.cam.ac.uk/mackay/
[11/9] Assignment 3 and a formal description of the project requirements can be found below.
[10/30] For the Burrows-Wheeler problem on the homework assignment, show the Burrows-Wheeler transform and the move-to-front encoding steps. You need not show entropy (Huffman/arithmetic) coding. For the MTF part, you may assume initially the initial configuration is all the letters in alphabetical order.
[10/19] Assignment 2 is out below.
[10/17] Note the schedule change below; some additional papers on compression are included.
[10/16] In problem 5, the statement is incorrect, in that the Hub score should be proportional to the outlinks and the Authority score to the inlinks (they are reversed in the problem statement.)
[10/3] Advanced Fibonacci Review from Adam, if needed.
[10/2] Fibonacci Review from CS124, if needed.
[10/2] Note that there is a typo in the ranking paper regarding the definition of the Kendall-tau distance. On the bottom of page 615, in the defintion of K(sigma,tau), remove the i < j part.
[10/2] You will likely find the first assignment significantly easier if you have access to some sort of mathematical software, such as Matlab, Mathematica, or Maple. (Adam suggests Matlab, but that is just a recommendation.) For this assignment, the main advantage of using such a package is the ability to compute eigenvalues and eigenvectors with built-in commands.
These software packages are available for download from the FAS Computer Services home page (assuming you already have an FAS account):
http://www.fas.harvard.edu/computing/download
Please be aware that these software packages have varying licensing procedures. For example, Mathematica requires a (free) registration with Wolfram Research, whereas Matlab will only run if you are connected to the FAS network. (You can use Matlab from off-campus if you first connect to the FAS network using the FAS VPN client, also available for download at the above link; however, you may have to uninstall any other VPN clients on your system in order to install the FAS client.)
Please let Adam know if you have difficulty getting adequate mathematical software up and running. However, you should direct technical support questions you should contact the FASCS help desk; see
http://www.fas.harvard.edu/computing/kb/kb0672.html
for more information.
[10/2] Due to the seminar style of this course, we will not be holding regular sections.
Adam will be available for office hours every Tuesday from 5:00-7:00pm in MD323. If you plan to attend office hours, you should email him by 4pm to let him know that you are coming and approximately when you are coming, and to give him an idea of what you'd like to talk about. Otherwise, he will likely not be there. If you do not have after-hours access to Maxwell-Dworkin, be sure to let him know so he can let you in.
Of course, Adam always welcomes questions and comments by email, and will do his best to respond promptly.
[9/18] Try this link for the course video.

Basic Class Information

Assignments

Assignment 1. Due October 17.
Assignment 2. Due November 2.
Project Description. Various, first due November 16.
Assignment 3. Due November 30.
Assignment 4. Due January 9.

Notes

How to read a research paper notes.

Readings

Readings available on citeseer are cached in various formats; check out the upper right hand corner. All reading dates are tentative and will be confirmed in class. We may go faster, we may go slower. You should consider this a minimum of what you should be reading for this class. Feel free to explore on the Web or otherwise. We are just touching the surface of these topics; there's much more out there.

Unit 1: Search Engines and the Importance of Links

9/19: Background: A Brief History of Generative Models for Power Law and Lognormal Distributions. by Mitzenmacher.
- No summary necessary.
9/21: Review Markov chains.
- No summary necessary.
9/21: Authoritative Sources in a Hyperlinked Environment. by Jon Kleinberg.
- Summary question: How could you possibly improve on Kleinberg's algorithm?
9/26: Improved Algorithms for Topic Distillation in a Hyperlinked Environment. by Bharat and Henzinger.
- Summary questions: What techniques are suggested for improving on Kleinberg's algorithm? Do they appear worthwhile given the costs?
9/26: The PageRank Citation Algorithm. by Brin, Page, Motwani, Winograd.
- Summary questions: How does PageRank differ from Kleinberg's algorithm? How is it the same? What more can you do to improve PageRank?
9/28 Analysis of a Very Large Altavista Query Log. Henzinger, Marais, Moricz, and Silverstein.
- Summary questions: What sort of data do they try to mine from the query log? Which seems the most useful? Can you think of anything they should be looking for but did not?
9/28 Rank Aggregation Methods for the Web. by Dwork, Kumar, Naor, Sivakumar. An alternative version .
- Summary questions: Define Spearman and Kendall distance. Why and how are Markov chains useful for combining rankings?
10/3 PicASHOW: Pictorial Authority Search by Hyperlinks on the Web. by Lempel and Soffer.
- Summary questions: How does PicASHOW relate to Kleinberg's algorithm and PageRank? What are the differences?
10/3 The Eigentrust Algorithm for Reputation Management in P2P Networks by Kamvar, Schlosser, and Garcia-Molina.
- Summary questions: How are the ideas of eigenvalues/PageRank used in this setting?
10/3 The Link Prediction Probelm for Social Networks by Liben-Nowell and Kleinberg.

Unit 2: Compression and Basic Information Theory

10/5 + 10/10. We will be covering the basics of compression and information theory, including Huffman coding, arithmetic coding, LZ-style coding, etc. Some good online introductions to the material include:
Introduction to Data Compression, notes by Guy Blelloch.
Information Theory, Inference, and Learning Algorithms, specifically part 1 (Data Compression) of Mackay's Book. (Though the whole book is good.)
10/10 On the Implementation of Minimum-Redundancy Prefix Codes by Moffat and Turpin.
- Summary questions: None. Just recommended reading, at your leisure.
10/12 A Block-Sorting Lossless Data Compression Aglorithm. by Burrows and Wheeler.
- Summary questions: See below.
10/12 Data Compression with the Burrows-Wheeler Transform. by Mark Nelson.
- Summary questions: Although the Burrows-Wheeler algorithms is somewhat confusing, the underlying principles behind it are quite sound. Describe these principles.
10/17 Identifying Hierarchical Structure in Sequences: A Linear-Time Algorithm by Nevill-Manning and Witten.
- Summary questions: Explain the basic concept behind SEQUITUR. Why is it good both for compression and determining structure?
10/17 The entropy of English using PPM-based models by Teahan and Cleary.
- Summary questions: What is meant by the entropy of English? How can it be bounded?
10/19 Paper on JPEG.
- Summary questions: See below.
10/19 Wikipedia's take on JPEG ; Wikipedia's take on JPEG 2000; Wikipedia's take on MPEG, especially MPEG-2 (but also 1 and 4).
- Summary questions: Explain the DCT. How does JPEG 2000 differ from the original JPEG. Explain the similarities and differences between MPEG and JPEG. Discuss the reasons for different types of frames.
10/24 Towards Compressing Web Graphs by Adler and Mitzenmacher.
- Summary questions: Why would you want to compress web graphs? What techniques are suggested for compressing web graphs?
10/24 The Webgraph Framework 1: Compression Techniques by Boldi and Vigna.
- Summary questions: What features of Web data can you take advantage of to compress links?
10/24 Algorithms to Accelerate Multiple Regular Expressions Matching for Deep Packet Inspection by Kumar, Dharmapurikar, Yu, Crowley, Turner. Just read through Section 4 if you want.
- Summary questions: What are the similarities to the other works?

Unit 3: Data Streams and Data Summaries

10/26 Data Streams: Algorithms and Applications. by Muthukrishnan.
- Summary questions: see paper below.
10/26 Computing on Data Streams by Henzinger, Raghavan, and Rajagopalan.
- How is computing on a data stream different from a "standard" input-output algorithm? What are the goals, how do you measure resource usage, what sorts of problems are considered? Where has computing on data streams had successes? What lower bound techniques are there showing that computing on data streams can be hard?
10/31 Network Applications of Bloom Filters: A Survey. by Broder and Mitzenmacher.
- No summary necessary for this paper.
10/31 Compressed Bloom Filters. by Michael Mitzenmacher.
- No summary necessary for this paper.
11/2 New directions in traffic measurement and accounting: focusing on the elephants, ignoring the mice by Estan and Varghese.
- See below.
11/2 The count-min sketch and its applications by Comorde and Muthukrishnan.
- How are the structures in these papers similar to Bloom filters and how are they different? What are the similarities between this paper and the mice/elephants paper? What are the differences? How much do they matter?
11/7 Min-wise independent permtuations by Broder, Charikar, Frieze, and Mitzenmacher. Note: just read the first two sections if you want, it's hard going.
- Explain the notion of a sketch. How does min-wise independence compare with other "versions" of independence? Explain variations of min-wise independence.
11/7 Ferret: A toolkit for content-based similarity search of feature-rich data by Lv, Josephson, Wang, Charikar, and Li.
- Does it make sense to have a toolkit for this sort of application? Or do you suspect similarity search is highly content dependent? Explain the difference between filtering and ranking and how they are accomplished in the Ferret toolkit.
11/9 The Bloomier Filter by Chazelle, Kilian, Rubinfeld, and Tal.
- Summary questions forthcoming.
11/9 Beyond Bloom Filters: From Approximate Membership Checks to Approximate State Machines, by Bonomi, Mitzenmacher, Panigrahy, Singh, and Varghese.
- Summary questions forthcoming.

Unit 4: Coding, and Network Coding

11/14 Currently 1/2 Reserved for Project Discussion
11/14 Coding/Shannon's theorems
Recommended: Information Theory, Inference, and Learning Algorithms, specifically part 2, chaps 8-10 of Mackay's Book. (Though the whole book is good.) Some coding notes.
11/16 Reed-Solomon codes. The class will go over the notes/Shannon's theorem.
11/21 Reserved: How not to give a talk talk
11/28 Digital Fountains: A Survey and Look Forward by Mitzenmacher.
11/28 The Capacity of Low-Density Parity Check Codes under Message-Passing Decoding by Richardson and Urbanke.
11/30 Informed Content Delivery Across Adaptive Overlay Networks by Byers, Considine, Mitzenmacher, and Rost.
- What is reconciliation, and why is there the need for it in overlay networks if we are using digital fountain codes? What are the basic advantages of codes in an overlay context?
11/30 Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads by Byers, Luby, and Mitzenmacher.
- Do codes offer significant advantages in the setting of downloading in parallel? In what settings? In what settings might they offer few advantages?
12/5 Network Information Flow R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung.
12/5 Network Coding, an Instant Primer , by Fragouli, Le Boudec,and Widmer.
- What is network coding, and how does it differ from the types of encoding (digital fountain codes, Reed-Solomon, etc.) that we have seen previously? What potential advantages does network coding offer? What are the corresponding costs?
12/7 Network Coding for Large Scale Content Distribution , by Gkantsidis and Rodriguez.
12/7 XORs in The air: Practical Wireless Network Coding , by Katti, Rahul, Hu, Katabi, Medard,and Crowcroft.
- Argue based on the papers for today whether network coding will become standard in networking or will be relegated to specialized networks and theoretical work.
12/12 List Decoding: Algorithms and Applications, by Sudan.
12/14 Adam made up some nice notes on channel coding. It will help with the homework.