PyTorch in 8 Slides

This is the cover for the whole day. The promise is simple: by the end of these five posts you will understand what PyTorch is, why it dominates, how its gradients flow, how to write a full training loop, and how to avoid the bugs that silently break training.

The '8 pieces' framing matters because newcomers experience PyTorch as a sprawling API with hundreds of functions. In reality it rests on a handful of ideas. This first post lays those ideas out as a map so the rest stops feeling random.

The one-sentence definition does real work: PyTorch is tensor math plus automatic differentiation, exposed as ordinary Python. Most confusion about PyTorch comes from skipping this and treating it as a 'neural network library' with magic model objects. It is more fundamental than that.

Because it is just array math with derivatives, you can use PyTorch for things that aren't neural networks at all — optimization, physics simulations, any problem where you need the gradient of a function. The deep-learning layers (nn) are convenience built on this foundation, not the foundation itself.

Holding this definition in mind explains the layering you'll see two slides later: tensors at the bottom, autograd in the middle, nn on top.

The tensor is the atom of PyTorch. If you know NumPy arrays, you already know 80% of tensors: same indexing, same broadcasting, same shape semantics. The two additions are what make deep learning possible.

First, a tensor can live on a GPU, where thousands of cores do matrix math in parallel — the operation that dominates neural network compute. Second, a tensor can carry a requires_grad flag, which tells autograd to track operations on it so gradients can flow back later.

The dimensionality vocabulary (0-D scalar, 1-D vector, 2-D matrix, 4-D image batch as batch x channels x height x width) recurs constantly, so it pays to internalize it early.

This snippet is the absolute starting point: create a tensor, inspect its three defining properties, and do a matrix multiply. shape tells you the dimensions, dtype the numeric type (float32 is the deep-learning default), and device whether it lives on CPU or GPU.

The @ operator is matrix multiplication, the workhorse of neural networks. Running this confirms your install works and grounds the abstract definition in something you can actually see printed. Note the trailing dots in 1. and 2. — they force float32, which matters because many operations and most models expect floats, not integers.

The stack diagram is the single most useful picture for orienting yourself in PyTorch. At the bottom sits the Tensor: raw n-dimensional arrays that can run on a GPU. In the middle sits autograd: the engine that records operations on tensors and computes their derivatives. On top sits nn.Module: the layer that packages common patterns — linear layers, activations, whole models, loss functions.

The key insight is that each layer only depends on the one below it. nn.Module is built from autograd-tracked tensors; autograd is built from tensor operations. You can drop down a layer any time you need control. This is why advanced users mix raw tensor ops with high-level modules freely — they're all the same substance.

'Define-by-run', also called eager execution or a dynamic computation graph, is PyTorch's defining design choice. The graph that autograd uses is constructed on the fly as your Python executes, then discarded after the backward pass. The next forward pass builds it fresh.

The practical payoff is enormous. Control flow is just Python control flow: an if that takes different branches per input, a for loop whose length depends on the data, recursion — all of it works with no special syntax. And because execution is line by line, you can drop a print or a debugger breakpoint anywhere and see real values.

This comparison clarifies what 'dynamic' buys you by contrast with the old static-graph model that early TensorFlow used. In the static world you first defined a symbolic graph of the entire computation, compiled it, then fed data through it in a session. The graph could be heavily optimized and run fast, but it was opaque: control flow needed special graph operations, and debugging meant inspecting symbolic placeholders rather than real numbers.

PyTorch's dynamic approach traded a little raw speed for a lot of developer velocity — and that trade is why research migrated. Notably the gap has since narrowed: torch.compile now gives PyTorch much of the static-graph speed while keeping the dynamic feel, and modern TensorFlow adopted eager mode to copy PyTorch.

This flow shows the lifecycle of a tensor relative to the GPU. By default tensors are created on the CPU. You explicitly move them to the GPU with .to('cuda') (or .cuda()), where subsequent operations run across thousands of cores in parallel. When you need to read a result back into normal Python or NumPy, you move it back with .to('cpu').

The load-bearing rule, which returns as a bug in post 5, is that operands of an operation must be on the same device. You cannot multiply a CPU tensor by a GPU tensor. Keeping model and data on one device is the whole discipline of GPU usage in PyTorch.

Naming what PyTorch is NOT prevents a category of frustration. Beginners coming from scikit-learn expect a .fit() method that trains a model in one call. PyTorch deliberately doesn't give you that — you write the training loop yourself, which is more code but total control.

It's also not a curated model zoo on its own; pretrained models live in companion libraries like torchvision and Hugging Face. And it's not an autoML system that picks architectures for you. Higher-level frameworks — PyTorch Lightning, fastai — wrap PyTorch to provide the .fit() experience. Knowing where PyTorch ends and these wrappers begin stops you from hunting for buttons that were never there.

This is the compression of the whole post into five lines you can recite. Tensor equals a GPU-capable NumPy array. autograd equals free derivatives of anything you compute. nn.Module equals reusable building blocks for models. You still write the training loop yourself. And underneath, it's all ordinary Python objects you can inspect and print.

If you hold only these five facts, the rest of the API becomes discoverable rather than mysterious. Every function you encounter is doing one of these jobs.

The CTA closes the concept post and points forward. We've established the what; the next post tackles the why — the specific reasons PyTorch overtook the field and where it still falls short. Saving the post gives readers the map to return to as the later posts go deeper.