What a single neuron computes

"Neural network" sounds like brain magic. It isn't. A network is thousands of copies of one dead-simple unit — the neuron — wired together. If you understand this one unit, you understand the atom that everything (image recognition, language models, all of it) is built from. It takes about five minutes.

A neuron does three things

Give a neuron some input numbers. It:

1. multiplies each input by a weight (how much that input matters),
2. adds them all up, plus a bias (a baseline nudge),
3. passes the result through an activation function (a "switch" that decides how strongly to fire).

As one line:

$\text{output} = \varphi\!\left(w_1 x_1 + w_2 x_2 + \cdots + b\right)$

Here $\varphi$ (the Greek letter "phi") stands for the activation function.

The weights and bias are the numbers the network learns. The activation is a fixed rule. A common one is ReLU — "keep positives, zero out negatives":

$\mathrm{ReLU}(z) = \max(0,\; z)$

So ReLU(2.3) = 2.3, but ReLU(−1) = 0. It's the simplest way to give a neuron an on/off character.

Worked example

A neuron with inputs x = [1, 2], weights w = [0.5, −1], and bias b = 0.5.

weighted sum = (0.5 × 1) + (−1 × 2) + 0.5
             = 0.5 − 2 + 0.5
             = −1

output = ReLU(−1) = 0

This neuron stays silent (0) for this input — the negative evidence outweighed the positive.

Now feed it x = [3, 1]:

weighted sum = (0.5 × 3) + (−1 × 1) + 0.5
             = 1.5 − 1 + 0.5
             = 1.0

output = ReLU(1.0) = 1.0

Same neuron, different input → now it fires (1.0). By tuning the weights and bias, the network teaches each neuron what pattern of inputs should make it react. One neuron might learn "fire when there's a horizontal edge here"; another, "fire when this word is a verb."

Why the activation matters

Without the activation, a neuron would just be $w_1 x_1 + w_2 x_2 + b$ — a straight line. Stack a million straight lines and you still only get a straight line; the network couldn't learn curves or anything interesting. The activation adds a bend. That tiny bit of non-straightness, repeated across many neurons and layers, is what lets networks learn wildly complex patterns — faces, grammar, chess. (Besides ReLU, you'll meet sigmoid, which smoothly squashes any number into a 0–1 range — handy when you want a probability.)

From one neuron to a brain

Put several neurons side by side and they form a layer — each looks for a different pattern in the same input. Feed one layer's outputs into the next, stack a few layers, and you have a neural network. Training (via gradient descent + backpropagation) adjusts every weight and bias until the whole stack produces useful answers.

That's the honest picture: no magic, just multiply-add-switch, repeated a few million times.

Now play with one. Drag the inputs, weights, and bias below and watch the weighted sum and output change in real time. Try to make the neuron switch from silent to firing.

---

Understanding sticks when you build it. Try a free lesson and wire up a neuron yourself.