For most of us who use AI tools daily, the experience of November 2022 was the same. ChatGPT appeared. We tried it. It was uncanny. We could not, in that moment, explain why something that had not existed last week now existed and was actually useful.

The honest answer is that ChatGPT had been there for a long time. The wrapper was new. The substrate was a decade old. The reason the perception was so jarring is that the substrate had been forming in places most of us didn't read.

Here is what was actually happening, in short.

The embarrassing era (2010-2012)

If you said "AI" in 2011, you got eye-rolls. The state of the art was IBM Watson, which beat Ken Jennings at Jeopardy! in February 2011 using DeepQA, a system of about 100 classical language-analysis techniques running in parallel. Watson was impressive. Watson did not use deep neural networks. Watson was sold as the future of medicine and ended up sold for parts at a quarter of what IBM had spent building it.

Google was running on PageRank and the freshly-launched Knowledge Graph. Siri had shipped. Both felt like advanced search bars with personality. "AI" meant a brittle expert system that worked for the demo and broke in production.

The inflection point (September 2012)

On 30 September 2012, three University of Toronto researchers, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, submitted AlexNet to the ImageNet image classification competition. The runner-up scored a 26.2% top-5 error rate. AlexNet scored 15.3%. The previous year's gains had been measured in fractions of a percent.

Yann LeCun called it "an unequivocal turning point in the history of computer vision." It was. The recipe (big labelled dataset, big convolutional neural network, GPUs to train it on) became the template for the next decade. Nothing in the public felt this. ImageNet was an academic competition. The press wrote about Watson.

The Cambrian period (2013-2016)

This is where the substrate built up.

In January 2013, Tomáš Mikolov and a Google team published word2vec. For the first time, words became vectors in high-dimensional space, and the vectors had structure: king minus man plus woman approximately equalled queen. This is the seed of every embedding-based system since.

In September 2014, Sutskever, Vinyals and Le published seq2seq. Neural networks could now generate, not just classify. In 2015, generative adversarial networks. In 2016, AlphaGo beat Lee Sedol. This was the only moment in this period the general public saw deep learning at work, and most people concluded that AI meant board games.

The unlock (June 2017)

On 12 June 2017, a Google Brain team published "Attention Is All You Need". The Transformer architecture replaced the sequential processing of RNNs and LSTMs with self-attention. The headline result was a translation improvement. The actual breakthrough was that attention could be computed in parallel, which meant you could throw GPUs at it in a way you couldn't with recurrence.

This is what made the next five years possible. Without parallel training, you couldn't reach the scales that made language models useful. The paper was recognised inside ML research immediately. It was framed as a translation result. Almost nobody outside NLP read it as civilisational.

The scaling era (2018-2020)

BERT launched in October 2018 and was deployed across nearly every English Google Search query within two years. Most users of Google Search have been using a Transformer model daily since 2020 without knowing it. GPT-1 (June 2018), GPT-2 (February 2019, with OpenAI staging the release citing "misuse" concerns), GPT-3 (June 2020) followed in sequence, each one bigger than the last.

The hidden cause, in retrospect, was a paper most people haven't read. In January 2020, Jared Kaplan and an OpenAI team published "Scaling Laws for Neural Language Models". It showed that loss decreases as a power law with model size, dataset size, and compute, across more than seven orders of magnitude. This converted "make it bigger" from a hunch into an investment thesis. The capital allocation that followed is what made GPT-3, GPT-4, and everything since financially possible.

GPT-3 was available to the public in June 2020 via API. You needed a key, Python, and patience. The tech press wrote breathless coverage. The general public did not notice.

The moment the iceberg surfaced (November 2022)

ChatGPT was released on 30 November 2022. The underlying model (GPT-3.5) was, in essence, two years old. What was new was reinforcement learning from human feedback, the layer that made the model behave like a polite assistant instead of a probability machine, and the chat interface. One million users in five days. One hundred million in two months. The fastest consumer product adoption in history at the time. OpenAI's own engineers were surprised it worked.

This is the moment most of us mark as the start. It was actually the moment the iceberg surfaced. The work below the waterline had been forming since 2012.

What actually mattered

If you rank causal weight by what was necessary for the outcome: The Transformer architecture (2017) — parallel training. Scaling laws (2020) — the discovery that capital would convert to capability. RLHF (~2022) — the wrapper that turned a text completer into something that answers questions. The chat interface (November 2022) — the UX innovation that made five years of latent capability legible to non-engineers.

The "model got bigger" story is the simplest and the least true. GPT-3 had the size. It lacked the wrapper. Without the chat interface, GPT-4 would have been a developer tool. The breakthrough that the general public felt in November 2022 was a product decision, not a research one.

The thing that's actually weird

The strangeness of the November 2022 experience is real. But it isn't strange because the technology appeared overnight. It is strange because for ten years the work was buried in arXiv preprints and Google research blogs, visible to maybe ten thousand people in the world, and then OpenAI built a website around it and suddenly seven hundred million people knew. The acceleration was in the distribution, not the science.

What this implies for the next big surprise: the substrate is forming now, in places we are not reading. Whatever the next ChatGPT moment is, the science behind it is probably already on arXiv. We will, in retrospect, decide it was obvious. It is not obvious yet because we are not the ten thousand people watching the right venues.