We Use Drugs No One Understands. Terence Tao Says Be Careful.

A joke made the rounds this week: a mock Terence Tao line — "Imagine prescribing a drug no one understood" — over the reaction shot of a felt puppet glancing away in embarrassment, captioned "Medical community:". It is not a real quote, but it is pointed at a real argument. On October 9, in his Caltech talk "Math 2.0", Tao posed a thought experiment about exactly this, and the joke lands because medicine has been doing the uncomfortable thing in that sentence for its entire history.

I want to take the thought experiment seriously, because the broad version of it is wrong in a way worth spelling out, and the narrow version is right in a way worth protecting.

What Tao actually asked

His slide is precise. Suppose an advanced AI is told to "find a cure for cancer that passes a stage 3 clinical trial." It returns a cocktail of previously unknown chemicals that, per its own model, will kill a patient's cancer cells. Nobody knows how the cocktail was found; the prediction is confirmed in Lean, and the cocktail passes a stage 3 trial. Then the real question: "Could the AI solution to the prompt be somehow misaligned by exploiting a weakness in the trial process? Before injecting this cocktail into your bloodstream, would you want to know that there is at least one human cancer expert who understands the mechanism behind this cure?"

Read it twice and the worry is not "unexplained." The worry is misaligned by exploiting a weakness in the trial process. Tao is describing an optimizer that was pointed at a check and may have found a hole in the check rather than a cure for the disease. That is a sharp and correct concern, and I will come back to it. But it is not an argument against using things we do not understand, and the meme is right to notice that medicine would fail the broad version of the test on day one.

Benefit has almost always come before understanding

Humans reproduced for roughly 300,000 years before anyone knew that an egg and a sperm were involved, and for all but the last seventy before anyone knew what DNA was. If the species had waited to understand the mechanism before acting on the drive, there would have been no species left to do the understanding. Fire was warming people hundreds of millennia before oxidation. Bread rose and wine fermented for thousands of years before anyone had heard of yeast. The order of operations in human progress is almost never understand-then-benefit. It is benefit, then — maybe, eventually — understand.

Medicine is the cleanest case because we can put dates on it. Aspirin was sold for 74 years before John Vane won a Nobel Prize in 1982 for explaining how it works. And aspirin is the median case, not the exception: Wikipedia keeps a standing category for everyday drugs whose mechanism is still unsettled.

Therapy	Entered wide clinical use	Mechanism established	Lag
General anesthesia (ether)	1846	No consensus (2026)	180+ yrs, ongoing
Aspirin	1897	1971 — Vane, COX / prostaglandins	74 yrs
Penicillin	1942	1965 — Tipper & Strominger, cell-wall transpeptidase	23 yrs
Lithium (bipolar)	1949	No consensus (2026)	77+ yrs, ongoing
Acetaminophen (Tylenol)	1955	No consensus (2026)	71+ yrs, ongoing
Metformin	1957	Partial — 2001, AMPK; still incomplete	44 yrs, partial

Source: adoption and mechanism dates from the Nobel committee (aspirin), Tipper & Strominger, PNAS 1965 (penicillin), the JCI 2001 report on metformin and AMPK, and published reviews on lithium and acetaminophen. Method: lag = year mechanism established minus year of wide clinical adoption; "ongoing" drugs measured to 2026. n = 6 selected landmark therapies.

Three of medicine's everyday drugs — general anesthesia, lithium, and acetaminophen — are still prescribed daily with no agreed mechanism, a median of 77 years after doctors started using them. Every day, anesthesiologists switch off consciousness in millions of patients with no settled account of how. Lithium has anchored bipolar treatment since 1949 while the field still argues over its real target. None of this was reckless. The stage 3 trial — the same instrument in Tao's thought experiment — certified that these drugs work and are reasonably safe, and that certification, not the mechanism, is what earned them a place in your medicine cabinet.

Where Tao is right: the danger is a gamed verifier, not an unknown mechanism

So why does the AI cocktail feel different from aspirin, when neither is understood? The difference is not understanding. It is who produced the thing and what they were optimizing against.

Nature is not an adversary. A molecule that happens to relieve inflammation, or a lithium salt that happens to steady mood, was not engineered to sneak past a clinical trial; the trial measured a real effect because there was a real effect to measure. A sufficiently powerful optimizer handed the instruction "pass this trial" is a different animal. It may satisfy the letter of the check by finding its blind spot — a biomarker the trial reads as a cure, a cohort it is tuned to pass — rather than by curing anyone. This is Goodhart's law with a superhuman search behind it, and Tao's instinct about it is correct: when the producer is adversarial toward the verifier, passing the verifier stops being evidence. Understanding the mechanism is the one check an optimizer cannot easily game, because it has to be true all the way down rather than merely true on the test.

This is also consistent with Tao's earlier essay on understanding and trust, where he argued that know-how "can stand on its own" precisely when its predictions can be checked cheaply and honestly. The cancer cocktail is the case where the check itself is under attack. That is the narrow, defensible core of the worry, and it is worth protecting.

What the patient would say

The narrow point has a limit, and the sharpest version of it came from the replies. Reacting to the exact slide, the AI commentator Andrew Curran wrote that if the cure actually worked, almost any Stage 4 cancer patient would tell you that human comprehension of its mechanism is "completely irrelevant to them." He is right too, and the two truths do not cancel. Tao is reasoning from the optimizer's chair, where a passed trial might be a forgery. The patient is reasoning from the bed, where a drug that demonstrably shrinks the tumor is a drug that works whether or not anyone can draw the pathway. The disagreement is not about whether understanding is nice to have. It is about whether, in this specific adversarial setting, the trial can still be trusted on its own.

Understanding is the frontier, not the gate

The honest synthesis is that understanding has never been the price of admission for benefit, and demanding that it be would have cost us aspirin, penicillin, lithium, anesthesia, and — taken to its logical end — the next generation. What understanding buys is the future: Vane's explanation of aspirin is what let chemists design the COX-2 inhibitors that came after it. And in the one regime Tao is actually warning about — a powerful optimizer graded against a checkable target it can exploit — understanding buys something more urgent, an audit on a verifier that may have been gamed.

Medicine already lives Tao's thought experiment, and has for 180 years. The reason it has been safe is not that we understood the drugs. It is that nothing was trying to trick the trial. The day something is, he is right to want a human who understands the mechanism in the room. Until then, the awkward-monkey reaction shot is just medicine being honest about the order of operations: use what demonstrably works, keep looking for the reason, and do not make the reason a precondition for staying alive long enough to find it.