What is Artificial Intelligence?

We've covered what AI is and why it matters. Today we open the hood. The goal of this post is to demystify the phrase 'the model learns' so thoroughly that it stops sounding like magic and starts sounding like the simple, repetitive loop it actually is.

We'll walk the whole pipeline: turning messy reality into numbers, making a guess, measuring how wrong that guess is, and nudging the system to be slightly less wrong — millions of times over. You don't need calculus to follow it; you need the right pictures. By the end, 'training', 'loss', 'gradient descent', and 'inference' will all be words you can explain to someone else, which is the real test of understanding.

Before any detail, hold the whole thing on a napkin. Collect data. Turn it into numbers. Push those numbers through a model that has millions of adjustable knobs (called weights). Compare the model's guess to the known correct answer. Nudge every knob a tiny amount in the direction that would have made the guess better. Repeat, millions of times, until the guesses are good.

That loop — guess, measure, adjust — IS learning. Everything else in deep learning, however intimidating the paper, is a variation or an optimisation of those four steps. If you can hold this napkin sketch in your head, you have the scaffolding to hang every later detail on. Most people who find AI confusing never got this simple loop straight; you now have it.

This diagram is the training loop made concrete. Data (examples paired with their correct answers) flows into the model. The model produces a guess. A loss function scores how far that guess was from the truth. And an update step adjusts the weights to shrink that error next time. Then the cycle repeats with the next batch of data.

The single most important thing to notice is that it's a loop, not a one-shot process. No individual pass teaches the model much; the magic is in the repetition across enormous amounts of data. It's less like a flash of insight and more like erosion — millions of tiny adjustments that gradually carve the weights into a configuration that works. Keep this four-box cycle in mind and the rest of the post is just zooming into each box.

Step one, and the step beginners most underrate: everything has to become numbers. A model can't read the word 'cat' or see a photo — it only does arithmetic. So text is split into tokens and mapped to vectors of numbers; images become grids of pixel values; audio becomes sequences of waveform samples. This translation from reality to numbers is called creating an 'embedding'.

This step deserves respect because it's where a surprising amount of both the power and the bugs live. A good representation makes the downstream learning easy; a bad one makes it nearly impossible no matter how fancy the model. Mismatched formats, inconsistent scaling, and lost information at this stage quietly sabotage everything after it. When a model behaves strangely, an experienced engineer often looks here first — at how the inputs were turned into numbers — before touching the model itself.

Step two: the model makes a guess, and we measure exactly how wrong it is. Internally, the input numbers get multiplied and transformed through the model's weights to produce an output. Early on those weights are random, so the first guesses are essentially noise — and that's completely expected. We're not hoping for a good first guess; we're setting up a signal to improve.

That signal is the 'loss function', a single number that scores how far the guess is from the correct answer. High loss means very wrong; low loss means close. The loss is the model's compass — without a precise measure of wrongness, there's no direction to improve in. Choosing the right loss for the task is one of the quiet, important decisions in machine learning, because the model will relentlessly optimise exactly what you measure, for better or worse.

Step three is where the actual learning happens, and it's the part that sounds scary but isn't. Using calculus — specifically the gradient — the model works out, for every single weight, which direction to nudge it to make the loss a little smaller. Then it takes a tiny step in that direction. Do this across millions of examples and the weights gradually settle into values that produce good answers.

This process is called gradient descent, and it's the engine underneath essentially all of deep learning, from a tiny classifier to GPT-scale models. The 'gradient' is just the slope — which way is downhill on the error landscape — and 'descent' is taking steps in that downhill direction. You don't need to do the calculus by hand; frameworks like PyTorch compute it automatically. But knowing that this is all that's happening removes the mystique entirely.

If the math still feels abstract, here's the picture that makes gradient descent intuitive forever. Imagine you're standing on a foggy mountainside, blindfolded, trying to reach the valley. You can't see the bottom, but you can feel the slope under your feet, so you step in the steepest downhill direction. Then you feel again, and step again. Eventually you reach a low point.

Map that onto training: your altitude is the error (loss), 'downhill' is the negative gradient, and the size of each step is the 'learning rate'. Take steps too big and you bound right over the valley and bounce around chaotically; take them too small and you'll be descending until next week. Tuning that step size is one of the everyday crafts of training models. Hold this blindfolded-hiker image and gradient descent will never intimidate you again.

There's a distinction that clears up a lot of confusion: training versus inference. Training is the slow, expensive, one-time process we've been describing — running the loop over huge data to learn the weights. It can take thousands of GPUs and weeks for a frontier model. Inference is what happens afterwards: you feed in a new input and the trained model just runs forward to produce an output. Inference is fast and cheap by comparison.

The ChatGPT you interact with is in inference mode — its weights are frozen. This explains something that puzzles many newcomers: why the model doesn't 'remember' your previous conversation unless the app explicitly feeds that history back in. The model isn't learning from you in real time; it learned once, during training, and is now simply applying that fixed knowledge. Memory in chat apps is an engineering trick layered on top, not the model quietly updating itself.

The whole point of all this isn't to memorise the training data — it's to generalise to data the model has never seen. A model that perfectly reproduces its training examples but fails on anything new is useless, like a student who memorised last year's exam answers and is helpless when the questions change. This failure has a name — overfitting — and it's the single most common pitfall in all of machine learning.

That's why practitioners obsess over holding out a separate 'test set' the model never trains on: it's the only honest measure of whether real learning happened. Good performance on training data is necessary but meaningless on its own; good performance on unseen data is the real prize. Every technique you'll meet later — regularisation, dropout, cross-validation — exists to fight overfitting and push the model toward genuine generalisation rather than rote memorisation.

This flow shows the journey from raw data to a model running in production, and it's worth seeing as a whole. You start with messy, real-world data and clean it. You train a model on most of that data, learning the weights. You evaluate honestly on a held-out test set the model never saw. And only then do you deploy it to serve real users — where the work continues, because you have to monitor it.

The arrows that loop back are the important part. Real systems aren't 'train once and forget'; the world shifts, data drifts, and yesterday's accurate model slowly becomes today's stale one. Production ML is a cycle of deploy, monitor, detect drift, retrain. Beginners imagine the job ends at 'it works in the notebook'. Practitioners know that's roughly the halfway point — getting it to keep working in the real world is the other half.

Here's the part beginners reliably skip, and it costs them: they obsess over the model and ignore the data. They tune architectures and hyperparameters for days while the actual problem is sitting in the inputs — wrong labels, data leaking from the test set into training, mismatched formats, or simply not enough good examples. The uncomfortable rule of thumb is that roughly 80% of 'the AI is broken' turns out to be 'the data is broken'.

This is why senior engineers spend most of their time on data and evaluation, not on clever model tweaks. It's less glamorous than designing a novel network, but it's where the real wins live. If you take one practical lesson from this post into your own projects, make it this: when something's wrong, look at your data first, your evaluation second, and your model architecture last. You'll fix problems far faster than the people who do it in the opposite order.

That's how AI actually works, demystified: turn reality into numbers, guess, measure the error, nudge the weights downhill, and repeat until it generalises — then deploy and keep watching it. The vocabulary that once sounded intimidating (loss, gradient descent, inference, generalisation) is now yours to use and explain.

Tomorrow we make all of this tangible with actual code. In Day 1, Post 4 we'll train a real model in about fifteen lines — no rules written by hand, just examples and a single .fit() call — and watch it classify things it has never seen. Seeing the loop run on your own machine is what turns this conceptual understanding into something you truly own. Save this, and bring your laptop.