Convolution: how CNNs see images

How does a computer tell a cat from a dog? It doesn't look at the whole picture at once. It slides a tiny window across the image, hunting for small patterns — edges, corners, textures — and builds up from there. That sliding-and-summing operation is called convolution, and it's the engine inside every convolutional neural network (CNN). Let's compute one by hand.

An image is a grid of numbers

A greyscale image is just a grid of brightness values (0 = black, high = bright). A kernel (or filter) is a much smaller grid — often 3×3 — of numbers. Convolution slides the kernel over every position of the image; at each spot it multiplies overlapping cells and adds them into a single number. Those numbers form a new grid called a feature map that lights up wherever the kernel's pattern appears.

Formally, the output at position $(i,j)$ is:

$(I * K)(i,j) = \sum_{m}\sum_{n} I(i+m,\; j+n)\,K(m,n)$

That double-sum looks fancy but it's exactly "line up the kernel, multiply matching cells, add them up." Let's do it.

Worked example: detecting a vertical edge

Here's a patch of an image — bright on the left, dark on the right (a vertical edge):

image patch          kernel (vertical-edge detector)
10  10   0            1   0  -1
10  10   0            1   0  -1
10  10   0            1   0  -1

Multiply matching cells and add them all up:

row 1:  10×1 + 10×0 + 0×(−1) = 10
row 2:  10×1 + 10×0 + 0×(−1) = 10
row 3:  10×1 + 10×0 + 0×(−1) = 10
total = 30

A big output (30) means: "yes — there's a strong left-to-dark edge here." Now try the kernel on a flat patch (no edge — all the same brightness):

image patch          kernel
10  10  10            1   0  -1
10  10  10            1   0  -1
10  10  10            1   0  -1

each row: 10×1 + 10×0 + 10×(−1) = 0
total = 0

Output 0 — "no vertical edge here." Same kernel, slid everywhere, produces a feature map that's bright exactly along vertical edges. Swap in a different kernel and you detect horizontal edges, or blur, or sharpen. A CNN learns these kernels automatically rather than being handed them.

The three ideas that make it powerful

• Parameter sharing. The same small kernel (just 9 numbers here) is used at every position. So a CNN needs far fewer numbers to learn than a network that connected every pixel to every neuron — and it finds a pattern no matter where in the image it appears.
• Stride & padding. Stride is how far the kernel jumps each step (stride 2 → smaller output). Padding adds a border of zeros so the kernel can reach the edges. These control the output size.
• Stacking builds abstraction. Layer 1's kernels find edges. Layer 2 combines edges into corners and curves. Layer 3 combines those into eyes, wheels, letters. Deep CNNs go from pixels → edges → parts → objects, one convolution at a time.

Why this matters

Convolution is why CNNs dominate images (and a lot of audio and video): it's efficient, it's translation-invariant (a cat in the corner is still a cat), and it mirrors how patterns actually build up from local details. The same "slide a small filter and sum" idea shows up far beyond vision.

Now slide one yourself. In the demo below, move the kernel across the image grid and watch each output cell fill in — and see the feature map light up where the pattern lives.

---

The multiply-and-sum here is just the dot product in a grid. Want to build a CNN? Try a free lesson.