Week 8 Capstone: Noisy Router
Merging Text and Image generation into one system.
Overview
A multi-modal system. If you ask for a picture, it uses Diffusion. If you ask a question, it uses an LLM. The Router makes the decision.
Theory & Concepts
This integrates Week 4 (Router) with Week 7 (LDM).
Concept Check
Make sure you grasp the core equations before moving to code.
Interactive Lab: Multi-Modal Switch
Toggle inputs to see the active path.
LLM Branch
Diffusion Branch
💡 Experiment: Try different values to see how the system reacts in real-time.
Code Implementation
Core Logic
Below is the critical implementation from project.py. Analyze how the theory transforms into PyTorch code.
Python
if 'draw' in prompt:
return self.diffusion.generate(prompt)
else:
return self.llm.generate(prompt)
Weekly Project
Multi-Modal Agent
Combine your modules.
Step-by-Step Implementation
- Import Week 4 Router.
- Import Week 7 LDM.
- Create
NoisyRouterclass.
🚀 Challenge: Can you optimize this further? Check the
exercises.py file for bonus tasks.