CS 222 -- Algorithms at the End of the Wire

Handouts and Class Materials

Announcements

[9/22] There's an "assignment" up on compression. It is not actually due, but is meant to give you problems to try/practice to make sure you understand the material. Some of it is on papers we'll get to later so you might not be able to do all the problems immediately. I think we'll plan one or more sections with the TA Daniel for people who want to work through the problems together.
[9/11] On Canvas there's a video release form you need to sign -- it is necessary for us to record the course. It's in the form of a quiz. Please check on Canvas (a link will be in your Canvas announcements).
[9/11] Just a reminder that schedule may change on readings, etc. as we go.
[9/11] I'm putting up preliminary project directions, as people have asked about them. For the record, it's still very early to talk about projects -- we haven't even started information theory, compression, etc. !! -- but some people want to think ahead, and it's good to have expectations up.
[9/3] Discussions for next week are up.
[8/19] I'm told that there's help for (extension students, but maybe harvard students can use it too)
[8/19] Equipment Requirements: I've been told to put on the syllabus that ipads with stylus or related devices are required for homeworks, office hours, class discussions etc. So I am doing so! This will allow you to borrow equipment from Harvard -- the contact number for this is (617) 495-7777. Of course if you don't manage to borrow one we will find ways to deal but it will certainly be a lot better if you do -- it's a lot easier to work on problems with peers (or with me) with a device.
[8/15] I'll have a Piazza page up soon. This will be for class questions; we'll probably use a separate tool for class discussion issues. The Piazza link is piazza.com/harvard/fall2020/cs222 . I can enter you if you can't sign up.
[8/15] We will plan to use Gradescope+Canvas to turn in assignments. Please make sure you are signed up to Gradescope+Canvas. (This probably won't be set up for a week or two.)

Basic Class Information

Assignments

Project description is now available. Please read it!
Compression problems. To do on your own...

Readings

All reading dates are tentative and will be confirmed in class. We may go faster, we may go slower. We may have to move things for other reasons. You should consider the assigned papers a minimum of what you should be reading for this class. Feel free to explore on the Web or otherwise (and additional suggested readings for each topic will be listed as well). We are just touching the surface of these topics; there's much more out there.

Unit 0: Fun Stuff to Start us off

Please read all these, preferably before class begins, to see if you're interested.

Class 0: One your own, read A Note on Double Pooling Tests.
Class 0 How to read a research paper notes.
Class 0: On your own background, make sure you know Markov chains.
Start with wikipedia. The fourth external link is to a useful book chapter on Markov chains.

Unit 1: Data Sketches

Class 1 [9/3+9/8]: Network Applications of Bloom Filters: A Survey. by Broder and Mitzenmacher.
Class 2 [9/8]: New directions in traffic measurement and accounting: focusing on the elephants, ignoring the mice by Estan and Varghese.
Class 2 [9/8]: The count-min sketch and its applications by Cormode and Muthukrishnan.
- Questions for Discussion Groups: Compare contrast the mice/elephants paper and the count-min sketch paper? How do they describe and define the underlying problem(s) they are considering? How do they formalize their solution(s)? How do they compare?

Class 3 [9/10]: MapReduce: Simplified Data Processing on Large Clusters by Dean and Ghemawat.
Class 3 [9/10]: A Model of Computation for MapReduce by Karloff, Suri and Vassilvitskii.
- Questions for Discussion Groups: Compare and contrast the theory and practice of the MapReduce paradigm. What kind of tasks might MapReduce not be good for? Are there any eventual scaling problems for this paradigm? Suppose you had access to a large-scale MapReduce system -- what would you most want to use it for?

Class 4 [9/15]: Constant Time Updates in Hierarchical Heavy Hitters by Ran Ben Basat, Gil Einziger, Roy Friedman, Marcelo Caggiani Luizelli, Erez Waisbard.
Class 4 [9/15]: Faster and More Accurate Measurement through Additive-Error Counters by Ran Ben Basat, Gil Einziger, Michael Mitzenmacher, Shay Vargaftik.
- Questions for Discussion Groups: The sketching framework offers lots of opportunities, or "tricks", for putting together pieces in various ways to get better results. Describe the base operations that get put together in these papers. Do you find these combinations compelling? Can you think of other areas where putting together a combination of ideas in the right way might be useful?

Class 5 [9/17]: Learning-Based Frequency Estimation Algorithms Chen-Yu Hsu, Piotr Indyk, Dina Katabi and Ali Vakilian.
Class 5 [9/17]: A Model for Learned Bloom Filters, and Optimizing by Sandwiching.
- Questions for Discussion Groups: The idea of combining learning with traditional algorithms is relatively new. Can you describe how these two papers discuss this paradigm? How are they trying to bring together machine learning with more traditional algorithmic analysis?

Additional useful papers and other stuff for Unit 1:

And here are some old notes:
Notes 1 (Markov Chains)
Streaming, Sketching and Sufficient Statistics I , [*** first 40 minutes, through Count-Min sketch] by Graham Cormode.
Algorithms with Predictions , a survey on the new area of using machine learning algorithms and traditional algorithms combined.
Graham Cormode's page -- lots of papers on streaming..
My past student Justin Thaler's page -- lots of papers on streaming..
Andrew McGregor's page -- lots of papers on streaming..
Dewitt and Stonebreaker's take on MapReduce.
Partitioned Learned Bloom Filters.

Unit 2: Compression and Basic Information Theory

[9/22] and ongoing: We will be covering the basics of compression and information theory, including Huffman coding, arithmetic coding, LZ-style coding, etc. Some good online introductions to the material include:

Information Theory, Inference, and Learning Algorithms, specifically part 1 (Data Compression) of Mackay's Book. (Though the whole book is good.)
Introduction to Data Compression, notes by Guy Blelloch.
Class 6 [9/22] For today the plan is (likely to be) that you review the last class's lecture on compression topics, and we discuss/do problems/etc. in class.

Class 7 [9/24] On Compressing Social Networks, by F. Chierichetti, R. Kumar, S. Lattanzi, M. Mitzenmacher, A. Panconesi, and P. Raghavan.
Class 7 [9/24]: Permuting Web and Social Graphs, by P. Boldi, M. Santini, and S. Vigna.
- Questions for 9/24: How are compressing social networks and the web alike, and different? What role does having an underlying model play in determining how to compress these types of structures? What properties of the model(s) appear important?

Class 8 [9/29] An old but useful paper on JPEG.
Class 8 [9/29] Wikipedia's take on JPEG ; Wikipedia's take on JPEG 2000; Wikipedia's take on MPEG, especially MPEG-2 (but also 1 and 4).
- Class 8 [9/29] Questions for 9/29: Explain the DCT. How does JPEG 2000 differ from the original JPEG. Explain the similarities and differences between MPEG and JPEG. Discuss the reasons for different types of frames.

Class 9 [10/1] A Block-Sorting Lossless Data Compression Aglorithm. by Burrows and Wheeler.
A perhaps easier-to-read description can be found at Data Compression with the Burrows-Wheeler Transform. by Mark Nelson.
Class 9 [10/1] Identifying Hierarchical Structure in Sequences: A Linear-Time Algorithm by Nevill-Manning and Witten.
- Questions for 10/1 These papers both follow a theme of transforming the data before actually compressing it. Discuss the different types of transformations used; what properties do they have, and how are they important for both compression and efficiency of compression?

[Class 10] 10/6: Dasher : a Data Entry Interface Using Continuous Gestures and Language Models by Ward, Blackwell, and MacKay.
[Class 10] 10/6: Please see the talk on Dasher here .
- Please also read How to Give a Bad Talk. and How to Give a Good Talk.
- Questions for 10/6: Explain the connection between Dasher and compression, particularly arithmetic coding.

Unit 3: Link Information

[Class 11] 10/8: Authoritative Sources in a Hyperlinked Environment. by Jon Kleinberg.
[Class 11] 10/8: The PageRank Citation Algorithm. by Brin, Page, Motwani, Winograd.
- Questions for 10/8: How does PageRank differ from Kleinberg's algorithm? How is it the same? Can you think of ways to improve Kleinberg's algorithm, or PageRank?

[Class 12] 10/13: We will plan to have guest speaker/interviewee Sergei Vassilvitskii. While the reading below is not required, I would ask you read at least one of the papers below; it is work Sergei has done you might enjoy reading before his visit.
[Class 12] 10/13: k-means++: The Advantages of Careful Seeding by David Arthur and Sergei Vassilvitskii.
[Class 12] 10/13: Competitive Caching with Machine Learned Advice

[Class 13] 10/15: Improved Algorithms for Topic Distillation in a Hyperlinked Environment. by Bharat and Henzinger.
[Class 13] 10/15: Analysis of a Very Large Altavista Query Log. Henzinger, Marais, Moricz, and Silverstein.
- Questions for 10/15: What techniques are suggested for improving on Kleinberg's algorithm? Do they appear worthwhile given the costs?
- Questions for 10/15: What sort of data do they try to mine from the query log? Which seems the most useful? Can you think of anything they should be looking for but did not?

[Class 14] 10/20: Rank Aggregation Methods for the Web. by Dwork, Kumar, Naor, Sivakumar. An alternative version .
[Class 14] 10/20: The Link Prediction Probelm for Social Networks by Liben-Nowell and Kleinberg.
- Questions for 10/20: Define Spearman and Kendall distance. Why and how are Markov chains useful for combining rankings?
- Questions for 10/20: What, from the paper, are the best methods for link prediction? Can you think of additional methods they might not have tried? How might you improve on their methodology?

[Class 15] 10/22 How Not to Review a Paper. by Cormode. This is just for general background, you will not need to write up anything for discussion, but look over before today's class.
[Class 15] 10/22 Thoughts on Reviewing. by Allman. This is just for general background, you will not need to write up anything for discussion, but look over before today's class.
[Class 15] 10/22 Scheduling with Prediction and the Price of Misprediction.
[Class 15] 10/22 What's the Difference? Efficient Set Reconciliation without Prior Context
- Questions for 10/22: Why is scheduling a good example problem for prediction? Are there other natural variations of scheduling you can think of where prediction might be helpful?
- Questions for 10/22: Explain the set reconciliation problem -- can you think of additional possible applications?

[Class 16] 10/27: We will plan to have guest speaker/interviewee Andrei Broder. While the reading below is not required, I would ask you read at least one of the papers below; it is work Andrei has done you might enjoy reading before his visit.
[Class 16] 10/27: Graph structure in the Web by Andrei Broder and many others.
[Class 16] 10/27: On the resemblance and containment of documents by Andrei Broder.
[Class 16] 10/27: Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol by Fan, Cao, Almeida, and Broder.

[Class 17] 10/29: Coding/Shannon's theorems
Recommended background on your own: Information Theory, Inference, and Learning Algorithms, specifically part 2, chaps 8-10 of Mackay's Book. (Though the whole book is good.) NO SUMMARY!
[Class 17] 10/29: Reed-Solomon codes. The class will go over the notes/Reed-Solomon codes.
[Class 17] 10/29: Some coding notes. The class will go over the notes/Shannon's theorem.

[Class 18] 11/03: No class for election day; please work on reviews for mock PC program.

[Class 19] 11/05: Tentatively: last guest speaker.

[Class 20] 11/10: Fountain codes. by MacKay.
[Class 20] 11/10: Digital Fountains: A survey and look forward. by Mitzenmacher.
[Class 20] 11/10 XORs in The air: Practical Wireless Network Coding , by Katti, Rahul, Hu, Katabi, Medard,and Crowcroft.
[Class 20] 11/10: NOTE: not required, just background reading if you like! Network Coding, an Instant Primer , by Fragouli, Le Boudec,and Widmer.
- Questions for 11/10: Explain, in your own words, what is a Digital Fountain. How close can we get to a Digital Fountain in practice?
Questions for 11/10: What is the difference between network coding and digital fountains? Where might each be helpful?

[Class 21] 11/12: Embedded ethics lecture. Please read the below before lecture and answer the discussion question.
[Class 21] 11/12: Quantifying fairness in queueing systems.
- Questions for 11/12: The authors of the paper discuss two scheduling policies: First-In-First-Out and Shortest-Job-First. Which do you think is more fair, and why?

[Class 22] 11/17: A Brief History of Generative Models for Power Law and Lognormal Distributions. by Mitzenmacher.
[Class 22] 11/17: Power-Law Distributions in Empirical Data. by Clauset, Shalizi, and Newman.
[Class 22] 11/17: NOT REQUIRED, ADDITIONAL READING IF INTERESTED. Scale-free networks are rare by Broido and Clauset.
[Class 22] 11/17: NOT REQUIRED, ADDITIONAL READING IF INTERESTED. Editorial: The Future of Power Law Research. by Mitzenmacher.
- Questions for 11/7: Explain some of the controversy behind power law network/scale-free networks in research work. Where can you go wrong when you say, "Here is a power law" in a paper? How instead can you go right? Why is finding a power law interesting (or not).