Building Ghosts, Not Summoning God
Inside Andrej Karpathy’s worldview: AI isn’t summoning higher powers, it’s finishing the Industrial Revolution.
The most interesting thing about this moment in AI is the sheer diversity of conviction among the people closest to it.
You can find Nobel laureates warning that we are on the brink of unleashing something malevolent, and others insisting that progress has already peaked and the field is coasting on diminishing returns. One of the co-founders of a leading model lab urges caution, arguing that we still don’t fully understand what we’re building; another dismisses the anxiety and accelerates forward, convinced that the only way out is through.
That dispersion of belief - among the people shaping the technology - is what makes this debate so absorbing. The stakes are existential, but the compass points in every direction.
I’ve spent time reading every side - the optimists, the doomers, the skeptics - and the only honest conclusion is that no one really knows. But of all the voices in the noise, I’ve found Andrej Karpathy’s perspective particularly grounding. He’s curious without being evangelical, optimistic without being utopian, realistic without cynicism. He doesn’t sound like he’s selling anything or defending a missed bet - just trying to see the technology as it is, not as he wishes it to be.
His two-hour conversation with Dwarkesh Patel this weekend is well worth a listen. Here are the ideas that stayed with me:
(1) The Decade of Agents, Not the Year.
Karpathy shares that he is triggered by some of the over-prediction going on in the industry and begins with a simple correction to the zeitgeist: this isn’t the year of AI agents; it’s the decade.
Today’s agents are brittle. They forget what you tell them, can’t learn across sessions, and struggle to reliably use tools. Each interaction starts from scratch. What’s missing is the connective tissue of cognition - memory, feedback, multimodal awareness, persistence. Building those layers will take years of engineering, not quarters of product iteration.
The challenges are solvable but slow. The potential of the technology is obvious, but integrating it seamlessly into daily life will take patient systems work.
(2) From Animals to Ghosts
“We’re not building animals,” Karpathy says. “We’re building ghosts.”
Evolution builds hardwared brains through billions of years of trial and error. AI models, by contrast, are trained through imitation - absorbing human language and behavior encoded on the internet. They are “crappy evolution,” as he puts it: disembodied minds that learn patterns from culture, not from the world itself.
The key idea is that AI will not grow to resemble us through evolution but through simulation. Its advantages - reasoning speed, scale, recall - will come precisely from its disembodiment. But its blind spots will too: a lack of grounding in lived experience.
(3) The Cognitive Core Hypothesis
Karpathy argues that modern AI models mix two very different things: knowledge and intelligence. In technical terms, pre-training gives models vast stores of memorized data (“weights”) and the ability to generalize patterns from it. His claim is that the two shouldn’t be bundled together. “We need to remove some of the knowledge,” he says, “and keep what I call the cognitive core.”
Right now, models behave like overeducated parrots - they know everything but think poorly. The goal, he says, should be to isolate the cognitive core: a smaller, leaner system that doesn’t remember everything but knows how to reason.
Think of the difference between an encyclopedia and a mathematician. The encyclopedia has every fact but no understanding; the mathematician may forget formulas but can rebuild them from first principles. Karpathy’s point is that intelligence lies in structure, not storage.
If we can decouple long-term memory (facts retrievable from external databases) from a compact reasoning kernel, we could get models that are faster, cheaper, and more capable - machines that think first and recall later.
(4) In-Context Learning as Gradient Descent in Disguise
Karpathy notes that “it’s possible in-context learning runs a small gradient-descent loop internally.” That’s a dense, technical statement, but it’s worth unpacking.
Simply put, he’s describing “in-context learning”, or how AI models can seem to learn within a single conversation - even though, technically, they aren’t being retrained. The model isn’t changing its permanent memory (its weights). Instead, it’s rearranging its short-term thoughts - the equivalent of working memory - to recognize patterns and act smarter within the current exchange.
Karpathy’s claim goes one step further. He suspects that the process by which the model adjusts itself inside a conversation resembles a tiny version of learning itself - a small, temporary loop of improvement happening under the hood.
If that’s true, every chat session is a tiny act of self-teaching. The model isn’t just recalling what it knows; it’s actively reasoning in real time. And if we ever find a way to capture and retain those micro-learnings between sessions, AI could move from being forgetful to genuinely self-improving.
(5) Reinforcement Learning Is “Terrible but Miraculous”
In today’s AI playbook, reinforcement learning is how models are fine-tuned after training: reward good behavior, penalize bad, and let the system adjust. Karpathy calls this both “miraculous” and “terrible.”
Why terrible? Because it’s astonishingly inefficient. A model might generate a thousand words of reasoning and then receive a single bit of feedback - “good” or “bad.” It’s like grading a student only once at the end of the semester. The system learns, but slowly and noisily, “sucking supervision through a straw,” as he puts it. Miraculous because, despite that absurd simplicity, it works.
Humans, by contrast, learn through process feedback: we notice when a line of thought goes wrong, we revise mid-stream, we reflect. Karpathy believes future AI training will move in that direction - models that critique their own answers step by step, refine them, and learn not from rewards at the end but from reasoning about reasoning.
If that shift happens, it would replace today’s brute-force reward hacks with something closer to introspection - machines that get better not because we score them, but because they’ve learned how to review themselves.
(6) Collapse, Dreaming & Entropy
As models begin to train on their own output - the flood of AI-generated text on the Internet - they risk collapsing. Each generation imitates the last until creativity and diversity vanish.
Karpathy draws an analogy to the human brain. Sleep and dreaming, he suggests, may serve as a natural defense against this kind of collapse, injecting randomness and novelty back into the system. Dreams keep the mind from overfitting to yesterday’s reality.
Future AI systems may need their own equivalent: periods of synthetic “dreaming” that mix noise, imagination, and exploration to maintain diversity of thought. Without this built-in entropy, intelligence eventually becomes stale - a feedback loop of its own clichés.
(7) The March of the Nines
From his years leading self-driving at Tesla, Karpathy learned a humbling rule: each extra “nine” of reliability - 90%, 99%, 99.9%, and so on - demands roughly the same mountain of effort.
It’s an exponential law of engineering: the closer you get to perfection, the harder every incremental step becomes. This, he says, will apply equally to AI agents. Demos that work 90% of the time look magical but are miles away from deployment-ready systems that must work 99.999% of the time. Closing the final 10% is where the real engineering - and the real years - begin.
(8) AGI and the GDP Curve
Karpathy’s view of the economic impact of AI is strikingly unsensational. “You can’t find computers in the GDP,” he notes. Despite transforming every industry, the invention of the microchip barely shows up in macroeconomic data.
His point: technology diffuses slowly. Its effects are real but absorbed into the steady rhythm of growth. AI will likely follow the same pattern - an accelerant to the existing exponential rather than a sudden spike.
That doesn’t mean it won’t change everything; it means change will seep in invisibly, system by system, until one day the world simply operates on a new baseline of intelligence.
(9) Education After Utility
Karpathy’s new venture, Eureka, aims to rebuild education for a world where machines can already do most of the learning for us. “Pre-AGI education is useful,” he says. “Post-AGI education is fun.”
What he means is that once human labor is no longer the bottleneck, learning will stop being a means to an end. It will become a form of self-cultivation - the intellectual equivalent of going to the gym. You don’t lift weights to move heavy objects; you do it to stay strong.
In the same way, education in the AI era could become about keeping the mind in shape - a lifelong practice of curiosity, reflection, and growth.
Threaded through Karpathy’s thinking is a broader vision of intelligence as infrastructure - something we are gradually weaving into the fabric of society, as real and as invisible as electricity.
He doesn’t see an apocalypse or a singularity, just the slow construction of a cognitive layer across the world: humans and machines co-building systems that think, remember, and reflect together.
The next decade, in his telling, won’t be about the arrival of AGI. It will be about learning how to live with - and build - our ghosts.
Very nicely written!
nicely distilled