Anthropic pitches AI safety bill that may block a model if 50+ people could be killed
The proposed policy seems to offer protections against AI running amok but actually forbids federal agencies from blocking a model's release without the permission of a judge.
Key points
- Anthropic has drafted an AI safety bill it hopes Congress will consider, according to a Politico report.
- The draft would allow a model's release to be blocked when there is a foreseeable, imminent risk of over 50 deaths or over a billion dollars in property damage.
- The bill would require the Commerce Department to ask a Washington, D.C., judge to halt a model's release, rather than letting agencies act on their own.
Anthropic has drafted a bill on AI safety that it hopes Congress might adopt, Politico reported. The proposal sets out scenarios in which the release of an AI model could be blocked from distribution.
Under the draft, a block could apply when there is a foreseeable, imminent risk that more than 50 people could die or that property damage could exceed a billion dollars. Politico reported that the risk must stem from the creation or release of a biological weapon, the model evading its developer's controls or a cyberattack.
How evaluations would work
The bill proposes using independent evaluators registered with the U.S. Commerce Department to assess a model's potential for disaster. Those evaluators could be paid by the AI labs themselves, provided the payment is not contingent on the evaluation results, according to Politico.
It is not clear how the proposed system would keep payments from influencing an evaluation. Politico reported that the legislation would bar federal agencies from using their own discretion to halt a model's distribution.
This bill represents the proposals outlined in Anthropic's Advanced AI Framework
— an Anthropic spokesperson
Timeline for reporting risk
If evaluators find an imminent catastrophic risk, they would have 72 hours to notify the Commerce Department. The department would then have to ask a judge in Washington, D.C., to stop the model's release.
An Anthropic spokesperson confirmed the draft to Politico and said it reflects proposals in the company's Advanced AI Framework, which it has advocated with federal and state lawmakers since June. Anthropic did not immediately respond to TechCrunch's request for comment.
