Dario’s Treebeard Problem

Why frontier AI needs testing, taxes, civil-liberties guardrails, and state capacity before the exponential outruns the state.

Dario Amodei has published another long essay about AI policy, which by now is its own subgenre. This one, Policy on the AI Exponential, is about the mismatch between AI, which is moving on an exponential curve, and politics, which often moves like Treebeard.

Amodei opens with the Lord of the Rings analogy. Democratic institutions are like Treebeard, the ancient sentient tree, which moves slowly. They deliberate, compromise, consult stakeholders, and produce reports before they move. That slowness is often a virtue, but AI progress has created a timing problem. The feedback loop for capability improvement is measured in weeks. The feedback loop for policy is measured in election cycles, agency processes, and committee calendars. The worry is not just that policy will be late, but that it will arrive to a different world.

Amodei’s other analogy is more familiar: cars, airplanes, and drugs. New technologies create enormous benefits, but we do not let companies sell planes without safety checks, drugs without trials, or cars without basic standards. That does not mean we ban planes, drugs, or cars. It means that when a product has a meaningful blast radius, society eventually builds rules around testing, liability, certification, and recall.

I’ve said this before, at the risk of sounding naive: I believe Dario is earnest.

Yes, he is the CEO of Anthropic. Yes, Anthropic benefits from a world where frontier AI is treated as powerful, dangerous, and too complex for two people in a garage to govern. Yes, “regulate us” is often the mating call of the scaled incumbent. But sincerity and self-interest can share a bunk bed.

The obvious critique is that Dario is warning about the frontier while personally moving it forward.

His defense is that progress is inevitable, and that it is better for a safety-oriented lab to help set norms near the frontier than to abandon it to less cautious actors.

What makes Dario harder to dismiss is consistency. He has occupied this strange position in AI for years: building the thing, warning about the thing, and asking governments to regulate the thing. He made these arguments when OpenAI was clearly ahead. He kept making them as Anthropic caught up. And he is still making them now, when Anthropic is leading. The line has not conveniently appeared only after the moat was dug.

Dario is the person selling rocket fuel while begging for a fire code. Easy to mock. Harder to ignore.

The essay’s core argument is simple: AI policy has to graduate from transparency to control. For the last few years, AI policy has mostly meant transparency: publish evaluations, disclose safety practices, report incidents, give governments visibility. Fine. Useful. But Amodei argues that some risks are now specific enough to test. And if they can be tested, they can be regulated.

The essay then walks through five policy areas.

1. Public Safety

Amodei wants mandatory third-party testing for the most powerful models before deployment, focused on catastrophic risks: cyber, bio, loss of control, and AI systems that accelerate AI R&D itself. If a model fails those tests, the government should have the authority to block, deter, or revoke deployment.

This is the airplane analogy applied to model releases. You do not ask Boeing to publish a thoughtful essay about safety culture and then let passengers board. You test the plane, and someone has the authority to ground it.

The concern is obvious: safety regulation can become an incumbent moat. Big labs can hire lawyers, auditors, policy teams, and former regulators. Smaller labs get the bill. Still, the alternative is strange. If a model can materially increase cyber or bio risk, should release depend entirely on the company’s internal judgment? “Trust us, we ran evals” is not a regulatory regime.

2. Labor & Tax Policy

Amodei argues that AI could create labor-market disruption that is larger than prior automation waves. The usual optimistic story is that tech displaces some jobs, creates new ones, and raises overall productivity. That story may still be true, but the timeline matters.

If AI can substitute for large categories of cognitive work overnight, then workers may not have time to retrain, relocate, or move into newly created occupations. Economists may eventually be right in the long run. The problem, as always, is living through the footnote.

He discusses wage insurance, retraining, job matching, retention incentives, and better labor-market measurement. He also leaves the door open to more structural redistribution if AI causes a sustained decline in demand for human labor: UBI, higher taxes on AI-driven companies, capital gains taxes, or universal capital accounts.

3. Accelerating the Positive

On frontier model deployment, Amodei wants stricter review but on AI-enabled science, he wants faster review. His point is that if AI accelerates drug discovery, biology, materials science, and clinical research, then the bottleneck may shift from invention to approval.

The upside of AI will not arrive just because models get better. It has to pass through hospitals, regulators, insurers, procurement systems, clinical trial networks, universities, and the rest of the institutional plumbing. A miracle drug stuck in ordinary paperwork is, from the patient’s perspective, not yet a miracle.

This is the underrated AI policy question: not just how do we slow down dangerous capabilities, but how do we speed up beneficial deployment? Otherwise we get the worst possible combination: risks that compound quickly and benefits that don’t.

4. Civil Liberties

A lot of AI risk discourse focuses on rogue systems. Amodei is also focused on obedient systems. An AI that perfectly follows instructions can still be dangerous if the instructions come from a government conducting mass surveillance, a military delegating lethal force too freely, or a bureaucracy using automation to make high-stakes decisions with limited recourse.

AI will change the balance between citizens, companies, and governments. That can happen through surveillance, autonomous weapons, persuasion, cyber capabilities, and administrative automation. The concern is not only that AI systems might behave badly on their own. It is that they might behave very efficiently on behalf of institutions that already have coercive power.

His policy suggestions include limits on domestic autonomous weapons, stronger privacy protections, restrictions on data brokers, safeguards around government use of AI, and rights for citizens to use AI systems when navigating state action.

5. Geopolitics

Amodei argues that advanced AI will become a source of national power: economic, military, scientific, intelligence, and diplomatic. His answer is a democratic coalition that coordinates on AI development, chip access, export controls, safety standards, and defensive use of AI.

If frontier AI becomes a general-purpose engine of power, then AI policy cannot remain a domestic consumer-protection debate. It becomes part of alliance management, export strategy, military doctrine, and economic statecraft.

---

Amodei’s essay is a warning that AI policy cannot remain a series of after-the-fact hearings about things that have already happened. If frontier AI is going to be regulated like a serious technology, the institutions around it need to move earlier in the cycle: before release, before labor markets fully absorb the shock, before scientific bottlenecks waste the upside, before governments normalize AI-enabled coercion, and before geopolitics hardens into an uncontrolled race.

The hard part is that democratic institutions are not designed for exponential time. They are designed for legitimacy, deliberation, and constraint. Those are features, not bugs. But they become dangerous if the only choice is between moving slowly and moving blindly.

Treebeard eventually does act. The question is whether AI policy can do the same before the forest has already changed shape.

---