A Brief History of AI

Welcome to Day 4, post 3. We've covered the cycle and why it matters; now we look under the hood of the history itself. AI's whole story is really three different bets on one question: how do you make a machine smart?

Each bet defined an era, and each era ended when it hit a ceiling the next bet was designed to break through. Seeing the history as a sequence of strategies — rather than a blur of dates — turns it into something you can reason with. By the end you'll be able to name the three paradigms, explain why each gave way, and spot which one any given system belongs to.

Strip away the dates and AI's history is three bets on how to create intelligence. Bet one: write the rules yourself — encode human expertise as explicit logic. Bet two: don't write the rules, learn patterns from data using statistics and hand-picked features. Bet three: learn everything end-to-end, letting deep networks discover even the features from raw input.

The elegant part is that each bet was a direct response to the previous one's ceiling. Rules couldn't handle messy reality, so we learned from data. Learning from data was bottlenecked by hand-designed features, so we learned the features too. Framed this way, the history isn't a list to memorise — it's a logical progression where each era exists to fix the last one's fatal limitation.

This diagram lays the three paradigms out as an arrow, left to right: Symbolic AI, then Machine Learning, then Deep Learning, with the new world of foundation models at the leading edge. The arrow matters — it's not three separate boxes, it's a directed progression where each stage builds on and reacts to the one before.

Reading it, you can trace how responsibility shifts steadily from human to machine across the arrow. On the left, humans supply the rules; in the middle, humans supply the features; on the right, the machine supplies even those. The picture compresses the next several slides into one motion: the long migration of intelligence-work from the programmer's keyboard into the model itself.

Era one, Symbolic AI (1950s–80s), made the most intuitive bet: just encode human knowledge as explicit logic and rules. If experts know how to play chess or prove theorems, write those rules down and the machine can do it too. And for those clean, well-defined domains, it genuinely worked — symbolic systems played strong chess and proved real mathematics.

But it slammed into a wall at messy, real-world perception. You cannot hand-write a rule for every way a cat can look — every angle, lighting, breed, and pose. The world has infinite edge cases, and rules don't scale to infinity. That ceiling — brittleness in the face of messy reality — is exactly what the next era was built to overcome.

Era two, Machine Learning (1990s–2010s), changed the bet: instead of writing rules, let algorithms learn patterns from data. This era married statistics with hand-picked features and produced workhorse models like support vector machines and random forests that still run in production today.

It was a real leap — systems could now improve from examples rather than waiting for a human to anticipate every case. But it carried its own ceiling: it was bottlenecked by manually-designed features. A human still had to look at the raw data and invent the signals the model would learn from, and on truly messy data like images that's brutally hard. Modest dataset sizes capped it further. Powerful, but waiting for the breakthrough that would let machines design their own features.

Era three, Deep Learning (2012–now), broke the feature ceiling. Stack many layers in a neural network and it learns the features itself, straight from raw data — no human feature engineering required. The dam broke in 2012 when AlexNet crushed image recognition; in 2017 the Transformer unlocked language the same way; and from there, sheer scale did the rest.

This is the era we're living in, and its signature is that capability grows with data and compute. The catch, as always, is the cost of the breakthrough: deep learning is ravenous for data and compute, and it's hard to interpret. But for the first time, a paradigm could handle the messy perception that defeated symbolic AI and skip the manual features that bottlenecked classic ML — solving both prior ceilings at once.

Summarised, each era ended (or evolved) at a specific limit. Symbolic AI couldn't handle messy perception — too many edge cases to hand-code. Classic ML was bottlenecked by manual feature design — humans couldn't engineer good features for the hardest data. Deep learning solved features but became hungry for data and compute.

And the emerging answer to deep learning's hunger is foundation models: pretrain one enormous model on everything once, then fine-tune it cheaply for thousands of downstream tasks. Read top to bottom, this is a chain of ceilings and breakthroughs — every era's solution became the next era's problem to optimise. That's the real shape of progress: not a straight climb, but a relay of limitations being handed forward and overcome.

This chart plots the training data each era consumed, and like the compute chart from post 1, it climbs explosively. Symbolic AI used essentially none — knowledge came from human experts, not datasets. Classic ML used modest, curated datasets. Deep learning uses staggering volumes, up to and including large fractions of the public internet.

The visual makes a subtle but important point: the eras aren't just different techniques, they're different appetites for data. Each paradigm shift was unlocked partly by having more data available to feed it. So the curve isn't only describing the past — it hints at the constraint on the future. When you run out of high-quality data to scale on, you face a new ceiling, which is one of the live questions hanging over the current boom.

Foundation models flip the entire economics of the field. The old way trained one model per task — one for spam, one for translation, one for image tagging. The new way pretrains a single giant model on essentially everything, then adapts it cheaply to thousands of tasks through prompting or light fine-tuning.

This is the most general thing we've ever built — one model that can write, summarise, translate, and code — which makes it tempting to call it general intelligence. But underneath, as Day 2 taught, each model is still narrow: it's pattern-matching over its training distribution, not understanding. The leap is in reusability and breadth of application, not in a new kind of mind. Holding that distinction — astonishingly general in use, still narrow in nature — is the mark of someone who actually gets where we are.

The recurring mistake, era after era, is that each generation's experts assumed their paradigm was the final one — that symbolic logic, or classic ML, or deep learning, was the destination rather than a stage. They were wrong every time, and there's no reason to think we're the exception now.

The meta-lesson, distilled from seventy years, is this: the method that wins is usually the one that best converts more data and more compute into more capability. Symbolic AI couldn't ride that curve; deep learning can, which is why it won. So when you evaluate any new approach, the sharpest question isn't 'is it clever?' — it's 'does it scale?'. Bet on scalability, because history shows scalability is what survives.

So AI's history is three bets — write the rules, learn from data, learn everything end-to-end — each fixing the last one's ceiling, now converging on foundation models. The through-line: bet on what scales. Save this; the three-eras frame is a clean way to place any AI technique in context.

Tomorrow we make the eras tangible. Day 4 post 4 solves one task — spam detection — in the style of each era, in real code, so you can watch sixty years of AI history play out line by line.