Week 11: Optimization & Quantization
Making models run fast.
Overview
Quantization reduces numbers from FP32 (32-bit) to INT8 (8-bit), reducing memory by 4x and speeding up math.
Theory & Concepts
Dynamic vs Static Quantization.
Concept Check
Make sure you grasp the core equations before moving to code.
Interactive Lab: Quantization Effect
Toggle quantization to see how color depth (memory) decreases, while the image usually remains recognizable.
💡 Experiment: Try different values to see how the system reacts in real-time.
Code Implementation
Core Logic
Below is the critical implementation from project.py. Analyze how the theory transforms into PyTorch code.
Python
quantized_model = torch.quantization.quantize_dynamic(model, {nn.Linear}, dtype=torch.qint8)
Weekly Project
Benchmark
Quantize BERT and time it.
Step-by-Step Implementation
- Measure Inference Time (FP32).
- Quantize.
- Measure again.
🚀 Challenge: Can you optimize this further? Check the
exercises.py file for bonus tasks.