The Transformer, Explained

This post is the on-ramp for the whole day. Before we touch attention math or write any code, you need a clean skeleton of what a Transformer actually is — a single architecture that quietly replaced recurrent and convolutional networks across most of deep learning, and that powers nearly every model you have heard of by name.

The goal here is to lock in the big pieces and the vocabulary so the later posts have somewhere to stand. The mechanics, the from-scratch build, and the common traps are all just consequences of the handful of ideas introduced on this cover.

The defining move of a Transformer is to drop step-by-step processing entirely. Instead of a loop that reads one token after another, it stacks identical blocks, and inside each block every token gets to look at every other token at once through self-attention, followed by a small feed-forward network that refines each token on its own.

The key thing to hold onto is the division of labor: attention mixes information across tokens, and the feed-forward network processes each token independently. There is no recurrence and no convolution anywhere — just attention plus pointwise math, repeated many times. Everything else in the architecture exists to make that core stack trainable.

Attention replaced recurrence because RNNs had two structural ceilings. First, an RNN must finish token t before it can start token t+1, so the computation is inherently sequential and cannot use the parallel muscle of a GPU. Second, information has to survive being rewritten at every step to travel a long distance, so RNNs systematically forget long-range context.

Transformers sidestep both. Because every position attends directly to every other position, distance no longer costs information, and because there is no sequential dependency, the entire sequence is processed in parallel as large matrix multiplications. That single combination — direct long-range access plus parallelism — is what made training models with billions of parameters practical at all.

Before any attention happens, raw text has to become numbers. The input is first split into tokens, which are usually sub-word pieces rather than whole words, and each token is mapped through a learned embedding table to a vector. From that point on the model never sees text again — only vectors.

Concretely, a sequence of n tokens becomes an n-by-d matrix, where d is the model's hidden width. That matrix is what flows through every layer, being reshaped and refined but never leaving vector space. Holding this picture — a sequence is just a stack of vectors moving through the network — makes the later shape-juggling in attention far easier to follow.

The pipeline diagram gives you the entire architecture in four stages. Text is tokenized into integer ids, those ids are embedded into vectors, the vectors pass through N identical blocks that each apply attention and a feed-forward network, and finally an output head maps the refined vectors to whatever the task needs — next-token probabilities, a classification, a translation.

If you remember only one visual from this post, remember this one. Everything technical that follows lives inside the 'N x Block' stage; the tokenize, embed, and head stages are the relatively simple bookends that get the data in and the predictions out.

A single block is smaller than its reputation suggests. It contains multi-head self-attention, an Add & LayerNorm step that wraps it in a residual connection, a position-wise feed-forward network, and a second Add & LayerNorm. That is the whole recipe, and a Transformer is simply this block repeated N times.

The reason this matters is that depth in a Transformer comes from stacking copies of one simple unit, not from designing many bespoke layers. Once you understand a single block — which is exactly what the mechanics and code posts will dissect — you understand the entire model, because every block is identical in structure.

Transformers come in two flavors, and knowing the split early prevents endless confusion. An encoder sees the whole input at once and uses bidirectional attention, so every token can look both left and right; this is ideal for understanding tasks and is what BERT uses. A decoder generates left to right and uses masked, causal attention so a token can only see what came before it; this is what GPT uses for generation.

The original Transformer combined both for translation, but most modern systems pick one side. The practical takeaway: if the job is understanding a fixed input, think encoder; if the job is generating a sequence, think decoder. The only structural difference is the attention mask, a detail the mechanics and mistakes posts return to.

This snippet shows just how little code a Transformer takes in a modern framework. A single nn.TransformerEncoderLayer captures the entire block — attention, feed-forward, residuals, and normalization — parameterized by the vector width d_model, the number of attention heads, and the feed-forward dimension. Wrapping it in nn.TransformerEncoder with num_layers stacks N identical copies.

The point of showing this now is to demystify the architecture before we build it by hand. What looks like a formidable model is, at the library level, a few lines. The from-scratch post will open up exactly what this layer does internally so the abstraction stops being a black box.

The flow diagram makes the residual stream concrete — arguably the most important mental model for how a Transformer 'thinks.' Token embeddings enter as an n-by-d tensor, attention adds information mixed from other tokens, the feed-forward network adds per-token refinement, and this repeats across N blocks, with the tensor keeping the same shape the whole way.

The crucial idea is that each block reads from and writes back to a shared running representation — the residual stream — rather than replacing it. Information accumulates additively as the sequence moves deeper. This is why residual connections are not a minor detail but the backbone that lets gradients and information flow cleanly through dozens of layers.

These five terms are the working vocabulary you will see in every discussion of Transformers. A token is a sub-word unit of input; an embedding is the learned vector that represents a token; attention is the mechanism by which tokens assign weights to other tokens; a head is one parallel sub-channel of attention; and d_model is the width of the vectors carried through the entire network.

Get these straight now and the rest of the day reads smoothly. In particular, keep clear the distinction between a token (a discrete input piece) and its embedding (the continuous vector), and between d_model (the full width) and the per-head dimension d_k (a slice of it) — both pairs are routinely confused and both matter for the shape arithmetic ahead.

This recap consolidates the skeleton into five durable takeaways: a sequence goes in and comes out as vectors; attention mixes information across tokens; the feed-forward network refines each token in isolation; residual connections and LayerNorm keep the deep stack stable; and you build depth by stacking identical blocks.

If you can recite these five points, you are ready for the rest of the day. Each subsequent post assumes this foundation — what a Transformer is, what a block contains, and how the pieces connect — and builds on it rather than re-explaining it.

The teaser points forward to the 'why it matters' post. Having established what a Transformer is and what it is built from, the natural next question is why this one architecture took over not just NLP but vision, audio, and even biology. The next post makes the case with the concrete properties — parallelism, long-range access, and scaling — that drove its dominance.