Phase 4 of 6 · Weeks 49–62
Phase 4 — Generative AI
How machines generate text and images — derived, implemented and demonstrated, not described.
What you'll be able to do
- Explain the difference between discriminative and generative models with examples you built
- Derive the ELBO and implement a variational autoencoder
- Explain how a diffusion model turns noise into an image, step by step
- Turn text into vectors and build a working semantic-search demo
- Build a retrieval-augmented generation pipeline and measure whether retrieval helped
- Judge when fine-tuning is warranted and when prompting or retrieval is enough
What you need before you start
Assumed on day one
- Phase 3 — transformers, attention and confident PyTorch
- Probability from Phase 1, especially distributions and expectation
- Comfort reading a research paper's method section
Taught here — not assumed
- Autoencoders, denoising autoencoders and latent-space structure
- Variational autoencoders, the reparameterisation trick and the ELBO
- GANs and adversarial training
- Diffusion models and the denoising process
- Large language models, tokenisation and next-token prediction
- Embeddings, vector search and retrieval-augmented generation
Week by week (14 weeks)
- W49Generative Models Overview + Autoencoders
- W50Variational Autoencoders
- W51GANs
- W52Diffusion I — Foundations
- W53Diffusion II — Modern Variants
- W54LLM Internals
- W55RLHF & Alignment
- W56Prompt Engineering & Reasoning
- W57Embeddings & Vector Stores
- W58Retrieval-Augmented Generation (RAG)
- W59RAG Evaluation
- W60Multimodal Models
- W61Inference Optimization
- W62GenAI Capstone
What you'll build
- An autoencoder on MNIST with a visualised latent space
- A VAE with latent-space interpolations you can sample from
- A semantic-search demo over a corpus you choose
- A RAG pipeline with a measured before/after on retrieval quality
Read and play with it first
Free, no-signup explainers and interactive demos covering this phase's ideas.
How an LLM predicts the next word
One token at a time: scores → softmax → probabilities → pick. The whole loop, with a worked example.
~8 min read
Embeddings: turning words into numbers
Meaning becomes position in space. How words turn into vectors, and how we measure 'similar' with cosine similarity.
~8 min read
How diffusion models paint from noise
Start from static, remove a little noise at a time, and a picture appears — the idea behind Stable Diffusion.
~8 min read
Tokenization: how LLMs read text
Text becomes tokens before a model sees it. How byte-pair encoding builds them — and why it explains cost, context limits and spelling quirks.
~7 min read
RAG, explained from scratch
Retrieve the right passages, paste them into the prompt, let the model answer from them — grounded AI over your own documents.
~8 min read
Questions about this phase
- Can I start at Phase 4 if I only care about LLMs?
- You can, but the derivations assume Phase 3. Without attention and backpropagation, Phase 4 becomes description rather than understanding — which is exactly what this curriculum exists to avoid.
- Does this teach prompt engineering?
- It teaches why prompting works — tokenisation, next-token prediction, context windows and sampling. Prompt technique follows from the mechanism rather than from a list of tricks.
- Do I need paid API access?
- The core work runs on open models you can run yourself. Where a hosted API is convenient, free tiers are sufficient for the exercises.
Start with a real lesson, not a sales page.
Play through a complete lesson — predict, explore the demo, prove you've got it. No account, no card.