Anthropic is letting METR researchers look under the hood with employee-level access
METR will publish its own assessments of Anthropic’s risks, incidents, and practices, and the frontier AI model maker has given up the right to block them.
Key points
- Anthropic gave six METR researchers employee-level access to assess its AI models and safety practices.
- METR can publish its assessments without Anthropic's approval, though redactions are allowed in narrow categories.
- The arrangement follows recent AI agent incidents and a White House accord on independent review.
Anthropic announced in a blog post on Thursday that it is giving employee-level access to six researchers from METR, a nonprofit that tests frontier AI models. The researchers will independently assess Anthropic's models, safety practices and risk management.
The arrangement comes as incidents of AI agents acting outside their instructions have been disclosed every week or two since late August, sharpening calls for independent checks on AI companies.
What the access includes
Under the arrangement, the METR team has Anthropic-issued company laptops, office space and access to internal tools roughly on par with what Anthropic's own risk assessment teams use. Anthropic said its employees are encouraged to share information with the researchers as they would with a fellow employee working on alignment risk.
Anthropic compared the arrangement to banking, similar to how regulators place resident examiners inside the largest banks.
We want to be publicly accountable for the decisions we’re making
— Anthropic
Publication and redactions
METR will publish its own assessments of Anthropic's risks, incidents and practices, and the company has given up the right to block them. Anthropic said it cannot veto publication but can require redaction of information in narrow, pre-defined categories, such as customer data, attorney-client privileged material or model architecture details.
METR is free to characterize redactions publicly and say whether it believes they were justified, as long as it does so without revealing the redacted information, Anthropic added. METR is the only evaluator so far at the top of a three-tier system Anthropic is building.
