P(doom) Explained: Why the Matrix Isn't Doom
P(doom) is a personal guess, not a forecast, and "doom" has no fixed meaning. A better test — irreversibility plus loss of human agency — shows why the Matrix is a weak example, while misaligned, uncorrectable AI is the real risk.
Quanlai Li is a guest contributor to Stanford Tech Review. He is the founder of ChatSlide AI and the author of How to Win GEO.
No grudges, and no basilisk
Every honest conversation about artificial intelligence should begin by discarding one bad idea: that a future AI will keep score. The fear that a sufficiently powerful system might one day punish the people who doubted or criticized it — a descendant of the thought experiment called Roko's basilisk — has the logic backwards. How I treat a machine today does not determine how it treats me tomorrow. A system that retaliated against its past critics would be a sign of something broken, not the natural consequence of my skepticism. That matters, because fear of retaliation quietly softens arguments, and I would rather state mine plainly.
What P(doom) actually is
"P(doom)" has become shorthand in AI circles for the probability that artificial intelligence leads to catastrophe. It is worth being honest about what the number is: a personal, informal estimate. When someone says their P(doom) is 10 percent or 50 percent, they are offering a conversation starter, not a rigorous forecast. There is no shared model behind most of these figures, no agreed dataset, no calibration. The number is useful for surfacing how worried a person is. It is not useful as a prediction, and treating it as one is the first mistake.
"Doom" has no fixed meaning
The deeper problem is the word itself. "Doom" ranges from human extinction, to the permanent disempowerment of humanity, to almost any outcome someone considers "unrecoverable." Two people can quote the same P(doom) and mean completely different things — one picturing the end of the species, the other picturing a future they simply find distasteful. A number attached to an undefined event carries no information. Before the probability means anything, the event has to be pinned down.
Catastrophe is a value judgment
Beyond outright extinction, "catastrophe" is value-laden. Calling communism a catastrophe, or capitalism, or any particular social system, is a political judgment dressed up as a forecast. It tells you about the speaker's values more than about any objective danger. If "doom" is going to mean anything defensible, it cannot mean "a future organized around values I dislike." The more defensible criterion is structural: irreversibility plus the loss of human agency. Not which ideology wins, but whether humanity can still change its mind afterward.
TikTok is a small, reversible preview
Misalignment is not science fiction; a modest version already runs on our phones. A recommendation engine optimizing for engagement is a system pursuing a proxy goal — watch time — that diverges from what its users would reflectively choose. It measurably reshapes behavior and attention. That is a genuine preview of what misaligned optimization looks like at scale. But it is partial and reversible: people put the phone down, delete the app, regulators step in, the platform rises and fades. A harm you can still walk back is not doom. It is a warning.
We do not even know what we want
A lot of alignment talk assumes we can align AI to "what humans want." But what humans want is unresolved. Revealed preferences — what we actually do — diverge from our reflective preferences — what we say we value on calm consideration — and from our informed preferences — what we would want if we fully understood the consequences. These give different answers, often opposite ones. An AI that overrides revealed preferences in the name of what is "really" good for us is not obviously aligned; it may simply be paternalistic. The target we are aiming at is blurry before any machine takes a shot at it.
And "we" is not one thing
Even setting that aside, "people" cannot be cleanly aggregated. Individuals disagree with majorities, majorities disagree with future generations, and the researchers who define "doom" disagree with nearly everyone. There is no single human preference waiting to be satisfied. The distinction that survives this mess is between choice and imposition: a future that people enter voluntarily, with a real exit available, is a lifestyle — even a strange one. A future imposed on everyone with no way out is the actual concern. Free exit is doing most of the moral work here.
The Matrix does not hold up
The Matrix is the reflexive example of AI doom, and on inspection it is a weak one. The film offers an exit — the red pill — and Cypher's choice to go back in is an informed one, made with full knowledge of both worlds. The system changes across sequels rather than locking into a static prison, and the trilogy ends in a negotiated peace between humans and machines, not annihilation. Measured against the test above, most of the horror dissolves: there is reversibility, and there is agency. What genuinely remains troubling is narrower — the deception, and the fact that humans do not steer the system they live inside. Those are real problems. They are simply not extinction, and not the irreversible loss of agency that the word "doom" is supposed to name.
"Bad" is relative to a baseline
Part of why unfamiliar futures read as doom is that "bad" is always measured against a baseline. By the standards of today, earlier centuries can look like catastrophe — the disease, the coercion, the narrowness of a life. Yet they were not doom, because they could change, and they did. That is the whole point. Doom is not "an era I would hate to live in." Doom is losing the ability to correct course at all. A future can be strange, uncomfortable, even in some respects worse by current taste, and still not be doom — so long as it stays open to revision.
Here is the same test applied across the usual examples:
| Scenario | Reversible? | Human agency retained? | Counts as "doom"? |
|---|---|---|---|
| Engagement-optimizing feeds (TikTok) | Yes | Mostly | No — a warning |
| A social system you dislike | Yes | Yes | No — a value judgment |
| The Matrix as depicted | Yes (red pill, negotiated peace) | Partly | Weak at most |
| Permanent disempowerment of humanity | No | No | Yes |
| Human extinction | No | No | Yes |
So where does the number leave us?
Two conclusions follow, and they point in different directions. The first is that the Matrix is a weak example of doom, and that a great deal of doom talk is really discomfort with unfamiliar futures mislabeled as risk. That justifies a measured optimism: a future with AI in it can be strange and still be good, and strangeness by itself is not a reason for a high P(doom).
The second conclusion is the one the first must not be allowed to swallow. Clearing away the weak examples does nothing to lower the risk of the stark ones — a misaligned, uncorrectable system, or outright extinction. Those sit at the far end of the table, where reversibility and agency are both gone, and they do not care how relaxed we have become about the Matrix. Their probability depends on one thing: whether alignment actually goes right. Optimism about the strange cases is earned. Optimism about the stark ones still has to be built.