Anthropic Gives METR an Inside View for AI Risk Assessment
Ars Technica reports that nonprofit evaluator METR has a team inside Anthropic with access to hardware and internal systems. Evaluation findings remain unspecified.
The Short Version
Anthropic has given outside evaluators access resembling that of employees to assess its models’ risks, Ars Technica reports. Nonprofit METR now has a team based in Anthropic’s office with access to hardware and internal systems.
Why It Matters
The concrete change is the access available for assessment. It could let evaluators examine risks from a closer vantage point, but access itself is not a finding that a model is safe. The supplied account reports the arrangement without describing evaluation results.
The Bigger Picture
Ars Technica places the move within AI labs’ proposals to give outside evaluators a larger role as concerns grow about increasingly capable systems. It also notes that METR has worked with OpenAI, Google and Meta as well as Anthropic. That history describes its involvement across labs; it does not establish what this particular assessment will conclude.
What to Watch
The next useful details would be what METR can test and what findings it can publish. The available report does not specify access conditions, publication guarantees or authority over model releases. Those remain open questions when judging how much practical oversight the arrangement provides.
What would you want an outside AI evaluator to publish before its work increased your confidence in a model?
Connie Brew