Linear vs logistic regression

These are the two "hello world" models of machine learning, and they're often confused because of the shared word regression. The difference is simple and worth nailing: linear regression predicts a number; logistic regression predicts a probability (a yes/no). One function turns the first into the second. Let's build both.

Linear regression: fit a line to predict a number

Given data (say, hours studied → exam score), linear regression fits the straight line that best predicts the output:

$\hat{y} = w x + b$

$w$ is the slope, $b$ the intercept — the model tunes them. "Best" means the line that makes the squared error between predictions and real points as small as possible:

$\text{Loss} = \frac{1}{n}\sum_{i=1}^{n}\left(\hat{y}_i - y_i\right)^2$

Drag the line until that error bottoms out (or let gradient descent do it) and you've trained it. The output can be any number — 42, −3, 1000.

The problem with using a line for yes/no

Now suppose the answer isn't a number but a category: is this email spam (1) or not (0)? A straight line is a bad fit — it happily predicts 1.7 or −0.3, which aren't valid probabilities. We need the output squeezed into the range 0 to 1. Enter the sigmoid.

Logistic regression: squash the line through a sigmoid

Logistic regression takes the same linear formula $z = wx + b$ and passes it through the sigmoid function, which smoothly maps any number into $(0, 1)$:

$\sigma(z) = \frac{1}{1 + e^{-z}}, \qquad p = \sigma(wx + b)$

Now the output $p$ is a probability. Decide "spam" if $p > 0.5$, else "not spam."

Worked example: the sigmoid in action

σ(0)  = 1 / (1 + e^0)   = 1 / (1 + 1)      = 0.5    (right on the fence)
σ(2)  = 1 / (1 + e^−2)  = 1 / (1 + 0.135)  ≈ 0.88   (probably yes)
σ(−2) = 1 / (1 + e^2)   = 1 / (1 + 7.389)  ≈ 0.12   (probably no)

Big positive input → near 1; big negative → near 0; zero → exactly 0.5. The sigmoid is the S-shaped "confidence dial" that turns a raw score into a probability.

Training logistic regression doesn't use squared error — it uses cross-entropy loss (also called log loss), which strongly punishes confident-but-wrong predictions:

$\text{Loss} = -\big[\,y\log p + (1-y)\log(1-p)\,\big]$

If the true label $y = 1$ and the model says $p = 0.9$, the loss is tiny; if it confidently says $p = 0.01$, the loss is huge. (This cross-entropy is exactly the "negative log-likelihood" that also shows up when LLMs predict the next token.)

The one-line summary

	Linear regression	Logistic regression
Predicts	a number	a probability (0–1) → yes/no
Formula	$\hat{y} = wx + b$	$p = \sigma(wx + b)$
Loss	squared error	cross-entropy
Use it for	prices, scores, demand	spam, disease, click/no-click

Despite the shared name, logistic regression is a classifier — the sigmoid is the only thing standing between "predict a number" and "predict a category."

Why this matters

Nearly every richer model is these ideas scaled up: a neural network is stacks of linear steps with squashing functions (activations) in between, and its final classification layer is essentially logistic regression with a fancier front end. Master these two and the rest is variations on a theme.

Now try both. Below, drag the line to fit scattered points (linear), then shape the S-curve and watch it turn scores into probabilities (logistic).

---

Reading isn't building. Try a free lesson and fit both models yourself.