Week 11: Optimization & Quantization

Making models run fast.

Overview

Quantization reduces numbers from FP32 (32-bit) to INT8 (8-bit), reducing memory by 4x and speeding up math.

Theory & Concepts

Dynamic vs Static Quantization.

Concept Check

Make sure you grasp the core equations before moving to code.

Interactive Lab: Quantization Effect

Toggle quantization to see how color depth (memory) decreases, while the image usually remains recognizable.

💡 Experiment: Try different values to see how the system reacts in real-time.

Code Implementation

Core Logic

Below is the critical implementation from project.py. Analyze how the theory transforms into PyTorch code.

Python
quantized_model = torch.quantization.quantize_dynamic(model, {nn.Linear}, dtype=torch.qint8)

Weekly Project

Benchmark

Quantize BERT and time it.

Step-by-Step Implementation

  • Measure Inference Time (FP32).
  • Quantize.
  • Measure again.
🚀 Challenge: Can you optimize this further? Check the exercises.py file for bonus tasks.