The uk treasury announced an extra £120 million for the ai security institute

Frontier models are tested before release, but no test proves that a model is safe in any absolute sense. What the results of these AI security tests are worth depends on who runs them and who pays the person running them.

On October 24, the UK Treasury announced an extra £120 million for the AI Security Institute, the UK’s AI safety body. It is the public organization that tests the most advanced models before they are made available to the public. This top-up alone is more than ten times the annual budget of its US counterpart.

The figure is striking. More importantly, it raises a question an executive can use starting tomorrow: when a vendor claims its model is safe, who checked, and who paid for the check?

Three Ways to Verify AI Security, Ranked by Who Pays

I looked at how models are checked today and sorted the approaches by a single criterion: who pays the person doing the test. The criterion is simple. It tells you more than the tester’s reputation does.

The Lab Checks Itself

Every lab runs its own safety tests before a release. This is necessary, because no one knows the model better than the people who built it. But a lab grading its own work will always be the least convincing of the three cases, however honest the people involved. Incentives pull toward the launch date.

Last month, The Information reported that Google, OpenAI and Anthropic are working on a joint body to set safety standards. The initiative came after a White House oversight plan stalled. I read this as a signal. The industry is looking for a shared framework, and that framework does not exist yet.

The Lab Pays an Outside Evaluator

Anthropic embedded six METR employees, with access to its internal tools. Anthropic says it cannot veto what METR publishes. It can, however, require that certain details be redacted.

This is a step above self-checking. A fresh pair of eyes sees things the internal team no longer sees. The weak point is access. A team paid by the lab depends on the lab to get into its tools. If access closes, the test stops. That dependence weighs on candor, even when neither side acts in bad faith.

The Public Pays

The UK institute answers to the government, not to a lab. It is not a regulator and it cannot block a launch. It does, however, see the largest models before the public does.

Grading other people’s papers strikes me as a solid first step. Incentives are aligned with finding problems, rather than with shipping the model on schedule. A tester paid by the public answers to taxpayers. It does not answer to a release calendar.

What the UK Treasury Announcement Changes

The UK is paying to verify models it does not build. That position gives its approach a more regulatory look than that of the United States, even though the institute has no power to block anything. The gap between image and real power deserves to be noted. The institute produces findings. Other players then decide what to do with them.

I do not know how the £120 million will be allocated in detail, so I refrain from saying what it will fund. The signal itself is clear: part of the verification of frontier models is becoming a recurring public expense, separate from the labs’ budgets.

For an SMB executive, this changes nothing about what they can control. It changes how to read vendors’ safety promises. A safety statement does not carry the same semantic authority depending on whether it comes from a vendor’s marketing, an outside evaluator’s report or a public body. The text can be identical. The source changes its weight.

A Tester’s Independence Rests on Three Levers

To judge the AI security a vendor claims, I look at three levers.

The first is funding. Whoever pays can stop paying. The second is access. Whoever opens the internal tools decides what the tester can see. The third is publication. In the case of Anthropic and METR, the lab cannot block what is published, but it can ask for certain details to be redacted. This means the reader of the report does not necessarily see everything the tester saw.

A publicly funded body has an advantage on all three levers. Its funding does not depend on a commercial relationship with the lab. Its power remains limited, since it cannot block a launch. None of the three models is perfect. Each has its place, and they are not equal.

What an AI Security Test Does Not Tell You

A frontier model test covers the general model. It does not cover your process, your data, or the way you will plug the tool into your business. That is where most of an SMB’s risks lie.

AI proposes, recommends, drafts, sorts. The decision stays with the person who acts on the output. A model rated well by an independent evaluator, plugged into a poorly defined process, speeds up the process’s flaw. The vendor’s security controls do not replace human validation in your workflow.

The difference is made by integration, far more than by the model. Two companies buying the same model get very different results depending on who reviews the outputs, at what stage, and with what clear instruction.

Five Questions to Ask Before Believing a Model Is Safe

I ask every vendor these. They fit on half a page and they go straight to the point.

1. Who ran the test? A name, a team, not a generic phrase.
2. Who paid for the test? The lab, a third party chosen by the lab, or a public body.
3. What access did the tester have? Only the published model, or also internal tools and preliminary versions.
4. Who controls publication? Can the vendor demand removals, and on which points?
5. Does the test cover my use case? A general result says nothing about your specific case.

I then give each safety claim a different weight depending on the answers. It is a weighted commitment. A guarantee verified by an independent third party carries more weight than an internal one, and I give it more room in my purchasing decision.

Putting It into Practice in an SMB

Here is how I would proceed in a company of a few dozen or a few hundred people.

1. Open a one-page file for each AI tool used or under consideration. Record the answers to the five questions in it.
2. Name one person responsible for each tool. They track the vendor’s version changes.
3. Define which outputs require human review before action. Start with anything that touches customers, money and legal commitments.
4. Review the file when the vendor changes models. A new version may have been tested differently from the previous one.
5. Keep the decision with a named human. The tool proposes. That person decides.

This work takes little time and it protects EBITDA in two ways. It avoids costly downstream errors. It also avoids buying a safety promise that rests on nothing verifiable.

What I Take Away

All three verification approaches have a use. Self-checking is indispensable and insufficient. The evaluator paid by the lab brings an outside view under access constraints. The public body best aligns its incentives with finding problems, without any power to block.

When a company tells you its model is safe, look at who ran the test and who paid for it before taking the claim at face value.

FAQ

Are frontier AI models safe?

No test shows that a model is safe in any absolute sense. Evaluations reduce uncertainty about certain risks before release. Their value depends on who runs the test, who pays for it and the access granted to the tester.

How are AI models tested before release?

There are three routes. The lab tests its own model. It can also bring in an outside evaluator it pays, like the METR employees embedded at Anthropic. Finally, a public body such as the UK’s AI Security Institute evaluates the largest models before the public sees them.

Can the UK's AI Security Institute block a model launch?

No. The institute is not a regulator and cannot prevent a release. It answers to the government rather than to a lab, and it sees the largest models before the public does.

Who pays independent AI model evaluators?

It depends on the model. The lab sometimes pays an outside evaluator, who remains dependent on it for access. The state funds public bodies: the UK Treasury announced an extra £120 million for the AI Security Institute on October 24.

What should I check before deploying an AI model in my company?

Ask who ran the safety test, who paid for it, what access the tester had and who controls publication. Then check that the test covers your use case. Finally, keep human validation on outputs that touch customers, money or legal matters.