Week 8 Capstone: Noisy Router

Merging Text and Image generation into one system.

Overview

A multi-modal system. If you ask for a picture, it uses Diffusion. If you ask a question, it uses an LLM. The Router makes the decision.

Theory & Concepts

This integrates Week 4 (Router) with Week 7 (LDM).

Concept Check

Make sure you grasp the core equations before moving to code.

Interactive Lab: Multi-Modal Switch

Toggle inputs to see the active path.

LLM Branch
Diffusion Branch
💡 Experiment: Try different values to see how the system reacts in real-time.

Code Implementation

Core Logic

Below is the critical implementation from project.py. Analyze how the theory transforms into PyTorch code.

Python
if 'draw' in prompt:
    return self.diffusion.generate(prompt)
else:
    return self.llm.generate(prompt)

Weekly Project

Multi-Modal Agent

Combine your modules.

Step-by-Step Implementation

  • Import Week 4 Router.
  • Import Week 7 LDM.
  • Create NoisyRouter class.
🚀 Challenge: Can you optimize this further? Check the exercises.py file for bonus tasks.