Harvard CS 242
Mondays and Wednesdays from 4:30-5:45 pm, Maxwell Dworkin G125
Instructor: H.T. Kung
The continued success of artificial intelligence will be tied to the development of AI-specific at-scale compute infrastructure and related methods. This active research area will be key to future developments in the field.
In CS 242, students will learn both the principles and state-of-the-art research in this rapidly advancing field. In order to become effective practitioners and researchers in this field, students must obtain a full-stack view of AI, understanding AI hardware, model-level optimizations, distributed training and deployment, security, and flexible architectures for future models. By studying these topics together, students will acquire an integrated understanding of these topics.
CS 242 will teach the latest developments in the field. AI hardware topics include TPUs and systolic arrays, industry AI chip architectures, memory-on-logic chip architectures, 3D-IC packing for CNNs, in-memory computing, and multi-chip-modules-based architectures. Model-level optimization topics include general matrix to matrix multiplication optimizations, separable kernels, pruning, quantization, term revealing, and NLP accelerators for state-of-the-art language models including transformers, RNNs, and LSTMS. Distributed training and deployment topics include distributed stochastic gradient descent, federated learning, and 5G mobile edge deployments. Security topics include secure distributed learning over the blockchain and secure AI execution methods. Flexible architecture topics include FPGAs and general purpose instructions.
Students will gain hands-on experience through programming assignments, presentations, reading quizzes, midterm, and a final project. Students will have latitude in choosing a final project they are passionate about. They will formulate their projects early in the course, so there will be sufficient time for discussion and iterations with the teaching staff, as well as for system design and implementation.
Recommended Prep: (1) programming experience (Python, MatLab or C/C++ should be fine); (2) basic knowledge in systems and machine organization; (3) familiarity in data structures and algorithms; and (4) maturity in mathematics (e.g., undergraduate linear algebra and statistics).