https://hanlab.mit.edu/courses/2023-fall-65940
Efficient AI Computing,
Transforming the Future.
TinyML and Efficient Deep Learning Computing
This course focuses on efficient machine learning and systems. This is a crucial area as deep neural networks demand extraordinary levels of computation, hindering its deployment on everyday devices and burdening the cloud infrastructure. This course introduces efficient AI computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, data/model parallelism, gradient compression, and on-device fine-tuning. It also introduces application-specific acceleration techniques for large language models and diffusion models. Students will get hands-on experience implementing model compression techniques and deploying large language models (Llama2-7B) on a laptop.
- Live Streaming:https://live.efficientml.ai/
- Time:
Tuesday/Thursday 3:35-5:00pm Eastern Time
- Location:36-156
- Office Hour:
Thursday 5:00-6:00 pm Eastern Time, 38-344 Meeting Room
- Discussion:Piazza
- Homework Submission:Canvas
- Contact:
Instructor
Announcements
- 2023-12-15
Final project: reports, slides and demo videos
- 2023-12-14
Final report and course evaluation due
- 2023-11-09
Mid-term survey: https://forms.gle/xMgCohDLX73cd4af9
- 2023-10-31
Lab 5 is out.
- 2023-10-19
Schedule
Date
Lecture
Logistics
Sep 7
Introduction
Chapter I: Efficient Inference
Sep 14
Pruning and Sparsity (Part I)
Sep 26
Quantization (Part II)
Sep 28
Neural Architecture Search (Part I)
Lab 1 due (extended to Sep 30 at 11:59 p.m)
Oct 3
Neural Architecture Search (Part II)
Oct 10
Student Holiday — No Class
Oct 17
TinyEngine and Parallel Processing
Chapter II: Domain-Specific Optimization
Oct 24
Transformer and LLM (Part II)
Nov 2
Diffusion Model
Chapter III: Efficient Training
Nov 7
Distributed Training (Part I)
Nov 16
Efficient Fine-tuning and Prompt Engineering
Nov 23
Thanksgiving — No Class
Chapter IV: Advanced Topics
Nov 28
Quantum Machine Learning
Nov 30
Noise Robust Quantum ML
Dec 5
Final Project Presentation
Dec 7
Final Project Presentation
Dec 12
Final Project Presentation + Course Summary
Dec 14: Project report and course evaluation due
No comments:
Post a Comment