Attention Is All You Need, explained from scratch

In 2017, eight researchers at Google published a paper with a cheeky title: "Attention Is All You Need." It introduced a design called the Transformer. Almost everything you've heard of in AI since — ChatGPT, Claude, Gemini, image generators, code assistants — sits on top of the idea in that paper.

The paper is famous for being short and elegant. It is also famous for being hard on a first read, because it assumes you already know a lot of maths.

This post makes a promise: if you're in Class 9 and you know basic algebra (what a variable is, how to multiply and add), you can understand the whole thing — including the maths. We'll build every piece from zero. No hand-waving, no "just trust me." Bring a pen; we'll do real numbers by hand.

Here's the map:

1. The problem the Transformer solved
2. The three maths tools we need (vectors, the dot product, softmax) — built from scratch
3. The one big idea: attention
4. The core formula, assembled piece by piece — with a full worked example
5. Where Q, K, and V come from
6. Multi-head attention
7. Positional encoding (teaching the model about word order)
8. The full Transformer block
9. Encoder, decoder, and translation
10. Why this changed everything
11. Glossary

Let's go.

---

1. The problem before Transformers

Computers don't read words. They read numbers. So the first job of any language AI is to turn words into numbers, and then do maths on those numbers to predict the next word, or translate a sentence, or answer a question.

Before 2017, the best language models read a sentence one word at a time, in order, like reading a book through a keyhole — you see one word, remember a little, slide to the next word, remember a little more. These were called RNNs (Recurrent Neural Networks).

Two big problems:

• They were slow. Word 10 could not be processed until words 1–9 were done. You can't split the work across many processors if each step waits for the one before it.
• They forgot. By the time the model reached the end of a long sentence, the beginning had faded. In "The cat that the dog chased was scared," who was scared? The cat — but that word is far away from "scared." Old models struggled to connect distant words.

The Transformer's promise: let every word look at every other word directly and all at once. No keyhole. No waiting in line. The whole page, seen together.

The mechanism that lets a word "look at" other words is called attention. That's the heart of the paper.

---

2. Three maths tools, built from scratch

To understand attention, we need exactly three ideas. If you already know them, skim. If not, we'll build each one carefully.

2.1 A vector is just a list of numbers

Suppose you wanted to describe a person with numbers. You might write:

Riya = [height 155, age 14, marks 88]

That ordered list [155, 14, 88] is a vector. Each slot means something fixed (slot 1 = height, slot 2 = age, slot 3 = marks). A vector with 3 numbers is called 3-dimensional.

In language AI, we do the same thing to words. Each word is turned into a vector — a list of numbers that captures its "meaning." This is called an embedding.

You don't pick these numbers by hand. The model learns them from reading huge amounts of text, and it arranges them so that words with similar meanings get similar vectors. "King" ends up near "queen"; "banana" ends up far away.

Key idea: meaning becomes position in space. Similar meaning → nearby vectors.

For our examples we'll use tiny 2-dimensional vectors like [1, 0] so we can picture them as arrows on graph paper (go 1 step right, 0 steps up). Real models use vectors with 512 numbers or more — same idea, just more slots.

2.2 The dot product measures "how aligned two vectors are"

Here is the single most important operation in the whole paper. Take two vectors of the same length. Multiply matching slots, then add up the results. That one number is the dot product.

$a \cdot b = a_1 b_1 + a_2 b_2 + \cdots + a_n b_n$

a = [1, 2, 3]
b = [4, 0, 5]
a · b = (1×4) + (2×0) + (3×5) = 4 + 0 + 15 = 19

Why should you care about one number? Because the dot product tells you how much two vectors point the same way:

• If they point in the same direction → large positive dot product.
• If they're at right angles (unrelated) → dot product is 0.
• If they point opposite → negative.

Let's see it. Take q = [1, 0] (an arrow pointing right) and compare it with three others:

vector	picture	q · vector	meaning
k1 = [1, 0]	same direction (→)	1×1 + 0×0 = 1	perfectly aligned
k2 = [0, 1]	right angle (↑)	1×0 + 0×1 = 0	unrelated
k3 = [0.7, 0.7]	45° between (↗)	1×0.7 + 0×0.7 = 0.7	partly aligned

So the dot product is a similarity score. Big score = "these two match." This is exactly how one word will decide how much to care about another word. Remember this.

2.3 Softmax turns scores into percentages

Say attention gives you three raw scores: [1, 0, 0.7]. These aren't easy to use directly — we'd like them as weights that add up to 100%, so we can say "pay 47% attention here, 17% there, 35% there."

The function that does this is softmax. It does two things:

1. It makes every number positive (using the exponential, explained below).
2. It divides each by the total, so they add up to 1 (i.e., 100%).

The exponential (eˣ). e is a special constant, about 2.718. Raising e to a power (eˣ) is a smooth way to (a) make any number positive, and (b) exaggerate differences — a slightly bigger score becomes a much bigger weight. That's useful: we want the model to be able to focus strongly on the best matches.

The softmax recipe for scores [s1, s2, s3] — first as one formula:

$\text{softmax}(s)_i = \frac{e^{s_i}}{\sum_{j} e^{s_j}}$

The same thing, step by step:

step 1: raise e to each score →  e^s1, e^s2, e^s3
step 2: add them up          →  total = e^s1 + e^s2 + e^s3
step 3: divide each by total →  weight_i = e^s_i / total

Worked example with scores [1, 0, 0.7]:

e^1   ≈ 2.718
e^0   = 1.000
e^0.7 ≈ 2.014
total ≈ 2.718 + 1.000 + 2.014 = 5.732

weight1 = 2.718 / 5.732 ≈ 0.474   (≈ 47%)
weight2 = 1.000 / 5.732 ≈ 0.174   (≈ 17%)
weight3 = 2.014 / 5.732 ≈ 0.351   (≈ 35%)

check: 0.474 + 0.174 + 0.351 ≈ 1.00  ✓

Notice how the biggest raw score (1) got the biggest weight (47%), but the others still get some attention. Softmax focuses without completely ignoring anyone.

You now know the only three tools you need: vectors (lists of numbers for meaning), the dot product (a similarity score), and softmax (scores → percentages). Everything else is these three, assembled cleverly.

---

3. The one big idea: attention

Here's the intuition, then we'll make it exact.

When you read "The cat that the dog chased was scared," and you reach the word "scared," your brain automatically looks back and asks: "Scared — who? Which earlier word does this connect to?" You scan the sentence, decide "cat" is the answer, and pull the cat's meaning forward.

Attention is a machine that does exactly this, with maths. For each word, it:

1. Forms a question ("what am I looking for?"),
2. Compares that question against every word in the sentence (using the dot product!),
3. Turns those comparisons into percentages (using softmax!),
4. Builds a blend of the other words' meanings, weighted by those percentages.

The result: each word gets rewritten to include the context it needs. "It" learns it refers to "cat." "Bank" learns whether we mean a river bank or a money bank, by looking at the words around it.

The library analogy (Query, Key, Value)

The paper gives each word three different vectors, and this trips people up. The library analogy makes it click:

• Query (Q) — what you're searching for. ("I want books about volcanoes.")
• Key (K) — the label on each book's spine. You compare your query against every key to see which books match.
• Value (V) — what's actually inside the book. Once you know which books match, you read their contents.

So for every word:

• Its Query asks a question.
• Every word's Key advertises "here's what I'm about."
• You match Query against all Keys (dot products → softmax → percentages).
• You collect the Values of the words, blended by those percentages.

Three roles, because a word plays a different part depending on whether it's asking, being searched, or contributing content. Where these three vectors come from, we'll see in Section 5. First, the formula.

---

4. Scaled dot-product attention — the core formula, built step by step

The paper's central formula looks scary:

$\text{Attention}(Q, K, V) = \text{softmax}\!\left(\frac{Q K^{\top}}{\sqrt{d_k}}\right) V$

By the end of this section you'll read it as easily as a recipe. Let's assemble it.

We'll follow one word (call it word 1) as it attends to a 3-word sentence. We're given:

• Word 1's query: q = [1, 0]
• The three keys: k1 = [1, 0], k2 = [0, 1], k3 = [0.7, 0.7]
• The three values: v1 = [2, 0], v2 = [0, 2], v3 = [1, 1]

(Here dₖ = 2, because the keys have 2 numbers each. Hold that thought.)

Step A — score each word (dot product Q · Kᵀ)

"How well does my query match each key?" Dot product with every key:

score1 = q · k1 = 1×1 + 0×0 = 1
score2 = q · k2 = 1×0 + 0×1 = 0
score3 = q · k3 = 1×0.7 + 0×0.7 = 0.7

So scores = [1, 0, 0.7]. Word 1 matches word 1 best, then word 3, then word 2.

(The Kᵀ — "K transpose" — in the formula is just bookkeeping that lines up the numbers so this dot product happens for every word at once. Don't let the symbol scare you; it's the dot products you just did.)

Step B — scale down by √dₖ

Divide every score by the square root of the key length. Here dₖ = 2, and √2 ≈ 1.414:

score1 / 1.414 = 1   / 1.414 ≈ 0.707
score2 / 1.414 = 0   / 1.414 = 0
score3 / 1.414 = 0.7 / 1.414 ≈ 0.495

scaled scores ≈ [0.707, 0, 0.495]

Why divide? When vectors are long (imagine 512 numbers), dot products can become huge. If the scores are huge, softmax turns "spiky" — it hands ~100% to one word and ~0% to everything else. That sounds decisive, but it's bad for learning: the model can no longer make small adjustments, so training gets stuck. Dividing by √dₖ keeps the scores in a gentle range so softmax stays smooth and the model keeps learning. (The √dₖ is the exact amount that cancels out how much the numbers grow with length.)

Step C — softmax the scaled scores → attention weights

Now apply the softmax recipe from Section 2.3 to [0.707, 0, 0.495]:

e^0.707 ≈ 2.028
e^0     = 1.000
e^0.495 ≈ 1.640
total   ≈ 4.668

weight1 = 2.028 / 4.668 ≈ 0.434   (43%)
weight2 = 1.000 / 4.668 ≈ 0.214   (21%)
weight3 = 1.640 / 4.668 ≈ 0.351   (35%)

These are the attention weights: word 1 will pay 43% attention to word 1, 21% to word 2, and 35% to word 3.

Step D — blend the Values, weighted by attention (... · V)

Finally, build word 1's new vector as a weighted mix of the value vectors:

output = weight1 × v1 + weight2 × v2 + weight3 × v3

Do it slot by slot.

Slot 1 (x):

0.434 × 2  +  0.214 × 0  +  0.351 × 1
= 0.868    +  0          +  0.351
= 1.219

Slot 2 (y):

0.434 × 0  +  0.214 × 2  +  0.351 × 1
= 0        +  0.428      +  0.351
= 0.779

output ≈ [1.22, 0.78]

That's it. You just did attention by hand. Word 1 started as some vector and came out as [1.22, 0.78] — a new vector that blends in the words it decided were relevant. Do this for every word in the sentence and you've run one attention layer.

Re-read the formula now: $\text{softmax}\!\left(QK^{\top}/\sqrt{d_k}\right) V$.

• Q·Kᵀ = score every word against every word (Step A)
• /√dₖ = keep the scores gentle (Step B)
• softmax(...) = turn scores into percentages (Step C)
• ... · V = blend the values by those percentages (Step D)

It's a recipe you can now follow.

Try it yourself: redo the example with q = [0, 1] (a query pointing up). You should find word 2 now gets the most attention. Working it out by hand is the fastest way to make this permanent.

---

5. Where do Q, K, and V come from?

We assumed the query, keys, and values were handed to us. In reality, each word starts with just one vector — its embedding — and the model creates Q, K, and V from it using three learned tables of numbers.

A matrix is a table of numbers, and multiplying by one makes a new vector

A matrix is a rectangle of numbers. Multiplying a vector by a matrix means: take the dot product of the vector with each column of the matrix, and the answers become your new vector. (Dot products again — they're everywhere.)

Example — turn the 2-number vector x = [1, 0] into a new 2-number vector using a matrix W:

        | 3  1 |
W  =    | 2  4 |

x · W:
  new slot 1 = (1×3) + (0×2) = 3
  new slot 2 = (1×1) + (0×4) = 1
  result = [3, 1]

So a matrix is a machine that transforms vectors into other vectors. The numbers inside the matrix are what the model learns.

Three matrices: W_Q, W_K, W_V

Each word's embedding is multiplied by three different learned matrices to produce its three vectors:

query  = embedding · W_Q
key    = embedding · W_K
value  = embedding · W_V

• W_Q learns how a word should "ask its question."
• W_K learns how a word should "advertise itself."
• W_V learns what content a word should "contribute" when attended to.

These three matrices (plus a few others) are the knobs the model tunes during training, using the huge text it reads, until the attention it produces is useful. Training itself (how those knobs get adjusted, via gradients and backpropagation) is a story for another post — here, just know: the matrices are learned, not hand-set.

---

6. Multi-head attention: several experts, different lenses

One round of attention captures one kind of relationship. But language has many at once:

• grammar — which noun does this verb belong to?
• meaning — which words are about the same topic?
• reference — what does "it" point to?

So the Transformer runs attention many times in parallel, with different W_Q, W_K, W_V matrices each time. Each parallel copy is called a head. The original paper used 8 heads.

Analogy: eight students each read the same sentence with a different highlighter — one marks grammar links, one marks topic links, one marks pronoun links, and so on. Then they combine notes.

Mechanically:

1. Run 8 attentions in parallel → 8 output vectors per word.
2. Stick them side by side (concatenate) into one long vector.
3. Pass that through one more learned matrix (W_O) to mix the heads back into the normal size.

More heads = more kinds of relationships captured at the same time, and it's still fully parallel (fast on GPUs).

---

7. Positional encoding: teaching the model word order

Here's a subtle problem. Look again at Steps A–D. Attention treats the words like a bag — it compares every word to every word, but nothing in the maths knows which word came first. "Dog bites man" and "Man bites dog" would look identical. That's clearly wrong.

The fix: before attention, add a "position vector" to each word's embedding — a unique numeric fingerprint for position 1, position 2, position 3, and so on. Now "dog at position 1" and "dog at position 3" have different vectors, so order is baked in.

The paper builds these fingerprints from sine and cosine waves (the same sin/cos from your trigonometry, which gently wiggle between −1 and +1). The formula for the number in slot i at position pos is:

$PE_{(pos,\,2i)} = \sin\!\left(\frac{pos}{10000^{\,2i/d}}\right)$

$PE_{(pos,\,2i+1)} = \cos\!\left(\frac{pos}{10000^{\,2i/d}}\right)$

Don't panic at the formula — here's all it's really doing:

• Each slot uses a wave of a different wavelength: early slots wiggle fast, later slots wiggle slowly.
• So each position gets a unique pattern of wave-values across the slots — like a barcode. Position 1's barcode differs from position 2's, which differs from position 3's.
• Sine/cosine are chosen because they're smooth and repeat predictably, which makes it easy for the model to compute "how far apart are these two positions?" — it can get relative distance just by doing maths on the encodings.

Mini-picture (imagine 4 slots; numbers illustrative):

position 1 → [0.84, 0.54, 0.10, 1.00]
position 2 → [0.91, -0.42, 0.20, 0.98]
position 3 → [0.14, -0.99, 0.30, 0.96]

Each row is a different "barcode." Add the matching barcode to each word's embedding, and attention now has order information to work with.

(Newer models use variants like learned or rotary positional encodings — RoPE — but the job is always the same: give the model a sense of where each word sits.)

---

8. The full Transformer block

We now have all the parts. A single Transformer block stacks them like this, applied to every word:

1. Multi-head self-attention   (mix information between words)
2. Add & Normalize             (residual connection + layer norm)
3. Feed-forward network         (each word "thinks" on its own)
4. Add & Normalize             (again)

Two of these pieces are new, and both are simple:

Add & Normalize (residual connection + layer norm)

• Add (residual connection): keep a photocopy of what went into the step and add it back to what came out: output = input + step(input). Why? So useful information is never lost, and the signal can travel cleanly through dozens of stacked blocks without fading. It's a safety wire: at worst, a layer can do nothing and pass the input straight through.
• Normalize (layer normalization): rescale the numbers in each vector so they sit in a comfortable range (not exploding to huge values, not shrinking to nothing) — like adjusting the volume to a steady level so later layers get a clean signal.

Feed-forward network

After attention has mixed information between words, each word passes through its own tiny 2-step network to process what it gathered:

$\text{FFN}(x) = \max\!\left(0,\; x W_1 + b_1\right) W_2 + b_2$

In words: multiply by a learned matrix, apply ReLU (a rule that just replaces negatives with 0 — max(0, ...)), then multiply by another learned matrix. This is the per-word "thinking" step. (Attention shares information across words; the feed-forward network works on each word alone.)

Stack them

The paper stacks N = 6 such blocks. Each block's output feeds the next. Early blocks catch simple patterns; deeper blocks build up to grammar, meaning, and reasoning — much like early layers of a vision network see edges and later layers see faces.

---

9. Encoder, decoder, and translation

The original paper was built for translation (English → German), so it has two stacks:

• Encoder (6 blocks): reads the whole input sentence and turns it into a set of rich, context-filled vectors — one per input word. It uses the plain self-attention we've described (every word sees every word).

• Decoder (6 blocks): writes the output translation one word at a time. Each decoder block has two attention steps:

  1. Masked self-attention over the words written so far.
  2. Cross-attention, where the Queries come from the decoder (what it's trying to write) and the Keys/Values come from the encoder (the input sentence). This is how the translation "looks back at" the source text.

Masking: no peeking at the future

When the model is learning to produce word 5, it must not be allowed to see words 6, 7, 8 (they haven't been written yet — that would be cheating). So in the decoder's self-attention we mask future positions: we set their scores to −∞ before softmax. Since e^(−∞) = 0, those future words get 0% attention. Simple trick, exactly right.

This decoder-only, masked, "predict the next word" idea is precisely what GPT-style models do. Chatbots are, at heart, a stack of Transformer decoder blocks trained to guess the next token, over and over.

---

10. Why this changed everything

Three reasons the Transformer took over:

1. Parallel, so fast. Every word is processed at the same time (no waiting in line like RNNs). That means you can train on enormous amounts of text using thousands of GPUs — which is exactly what made huge models possible.
2. Long-range memory. Because every word attends directly to every other word, connecting word 1 to word 100 is just as easy as connecting word 1 to word 2. No fading.
3. It scales. Add more blocks, more heads, more data, more compute — and it keeps getting better. This "it just keeps improving as you scale it" property is the engine behind GPT-2 → GPT-3 → GPT-4 and beyond.

The title was a flex, and it was right: for sequence problems, you could throw away the old recurrent machinery. Attention really was (almost) all you needed.

---

Recap — the whole thing in ten lines

1. Words become vectors (embeddings): meaning as position in space.
2. Add a positional encoding so the model knows word order.
3. From each word, make a Query, Key, Value using three learned matrices.
4. Score every word against every word with dot products (Q·Kᵀ).
5. Scale by √dₖ to keep scores gentle.
6. Softmax the scores into attention percentages.
7. Blend the Values by those percentages → each word is rewritten with context.
8. Do this in 8 parallel heads to capture many relationships at once.
9. Wrap it in residual + layer norm, add a per-word feed-forward network, and stack 6 of these.
10. An encoder reads; a decoder writes the next word using masked attention. Scale it up → ChatGPT.

If you followed the worked example in Section 4 with a pen, you understand the core of modern AI better than most people who use it every day.

Don't just read it — build it. Open an attention visualizer, click a word, and watch the weights light up exactly as we computed them. Then try a lesson and compute one by hand yourself. Understanding sticks when it comes from your own pencil, not a video.

---

Glossary

• Vector — an ordered list of numbers. A word's vector is its embedding.
• Embedding — the vector that represents a word's meaning; learned from data.
• Dot product — multiply matching slots of two vectors and add; a similarity score.
• Softmax — turns a list of scores into positive percentages that add to 1.
• eˣ (exponential) — e ≈ 2.718; raising it to a power makes numbers positive and exaggerates differences.
• Query / Key / Value (Q/K/V) — a word's three roles: what it's searching for / how it advertises itself / what content it contributes.
• Attention — for each word, score all words (Q·Kᵀ), scale, softmax, then blend their Values.
• dₖ — the length of the key vectors; we divide scores by √dₖ.
• Head / multi-head attention — running attention several times in parallel with different learned matrices, then combining.
• Positional encoding — a per-position "barcode" (built from sine/cosine) added to embeddings so the model knows word order.
• Matrix — a table of numbers; multiplying a vector by it produces a new vector (via dot products).
• W_Q, W_K, W_V, W_O — the learned matrices that make Q, K, V and combine heads.
• Residual connection — add a copy of a step's input to its output, so information isn't lost across deep stacks.
• Layer normalization — rescale a vector's numbers to a steady range for stable training.
• Feed-forward network — a small per-word 2-layer network (with ReLU) that processes what attention gathered.
• ReLU — max(0, x): replace negatives with 0.
• Encoder / Decoder — the stack that reads the input / the stack that writes the output.
• Masked attention — hide future words (set their scores to −∞ → 0% after softmax) so the model can't peek ahead.
• Transformer — the whole architecture from the paper; the foundation of modern LLMs.