Phase 2 of 6 · Weeks 21–32
Phase 2 — Classical Machine Learning
Every classical algorithm implemented by hand before you are allowed to import it — because sklearn is easy and understanding is not.
What you'll be able to do
- Train a model by implementing gradient descent and the normal equation, then compare them
- Build logistic regression from scratch and evaluate it on an imbalanced dataset
- Explain the bias-variance tradeoff using two examples from your own experiments
- Detect data leakage and set up honest train/validation/test splits
- Choose between precision, recall, F1 and AUC — and justify the choice for a given problem
- Cluster unlabelled data with k-means and reduce its dimensionality with PCA
What you need before you start
Assumed on day one
- Phase 1, or equivalent: derivatives, vectors and matrices, probability
- NumPy fluency — array indexing, broadcasting, vectorised operations
- Comfort reading a loss function and knowing what it is minimising
Taught here — not assumed
- Empirical risk minimisation, hypothesis spaces and generalisation
- Regularisation — L1, L2 and Elastic Net — and the geometry of why it works
- Cross-validation, calibration, learning curves and hyperparameter search
- Decision trees, random forests and gradient boosting
- Support vector machines and the kernel trick
Week by week (12 weeks)
- W21ML Foundations
- W22Linear Regression
- W23Logistic Regression & Classification
- W24Evaluation & Model Selection
- W25Decision Trees
- W26Ensembles I — Bagging & Random Forests
- W27Ensembles II — Boosting
- W28SVMs & Kernels
- W29Clustering
- W30Dimensionality Reduction
- W31Naive Bayes, k-NN, Anomaly Detection
- W32Feature Engineering + Phase 2 Capstone
What you'll build
- Linear regression from scratch in NumPy, benchmarked against sklearn on a housing dataset
- Logistic regression from scratch with a full precision/recall/ROC evaluation
- A written explanation of bias-variance in your own words, with two concrete examples
- An end-to-end model-selection study with cross-validation and leakage checks
Read and play with it first
Free, no-signup explainers and interactive demos covering this phase's ideas.
PCA: finding the directions that matter
Find the direction the data is most spread along, keep it, drop the rest — dimensionality reduction, from scratch.
~8 min read
K-means clustering, from scratch
Two steps — assign, then update — repeated until messy guesses snap onto the real groups. Learning without labels.
~7 min read
Decision trees & random forests
How a computer learns which yes/no questions to ask (Gini impurity), and why a forest of trees beats any single one.
~9 min read
Linear vs logistic regression
One predicts a number, the other a yes/no — and a single S-shaped function (the sigmoid) is all that stands between them.
~8 min read
SVMs and the kernel trick
Draw the boundary with the widest margin — then use the kernel trick to separate data that no straight line could.
~8 min read
Questions about this phase
- Why implement algorithms from scratch when sklearn exists?
- Because a one-line `fit()` call teaches you nothing about why a model fails. Building it once means you can debug it forever. You use sklearn immediately afterwards — as a benchmark against your own implementation.
- Is classical ML still worth learning in the age of LLMs?
- Yes. Evaluation discipline, regularisation, the bias-variance tradeoff and leakage detection carry directly into deep learning and generative AI. Most production failures are classical-ML failures.
- What maths does Phase 2 assume?
- Derivatives, matrix multiplication, and basic probability — all covered in Phase 1. If you are unsure, the readiness check tells you which specific foundations to review.
Start with a real lesson, not a sales page.
Play through a complete lesson — predict, explore the demo, prove you've got it. No account, no card.