Anthropic CEO defends safety record against prominent critic

Thank you for this, @_NathanCalvin. I appreciate and am a fan of the investigative work you’ve done on behalf of AI safety and believe your comments are coming from a constructive, honest place, and that it benefits the public to have people whose job is to scrutinize the actions of frontier AI companies. I think something would be wrong if companies of the size of Anthropic didn’t get harshly criticized basically every day; strong antibodies against corporate power are a healthy feature of a free society, and especially so when issues of the gravity of the one we’re facing are at play. I also do agree that some of the examples you cite were mistakes by Anthropic. That said, I disagree with the thrust of your post. At the core, a good fraction of the things you criticize are consequences…

A lot of what Amodei is saying here is true. That's why I and many other people who are concerned about AI safety have historically given Anthropic the benefit of the doubt; they really have sent some costly signals indicating that at least some of their decision makers care about mitigating catastrophic risks. But the object-level issue here is "is Anthropic's bill good, or even acceptable?" Amodei isn't really engaging with that question. That's fine; he's a very busy and important guy, and even among DC AI policy professionals only a small handful of people have actually read and understood Anthropic's bill. It's good that he cares enough to make the general case for "Anthropic cares about AI safety" in response to a Nathan Calvin tweet. But someone who works at Anthropic does…

I think it’s a good idea to take concentration of power risks quite seriously. I am actually more concerned, overall, about concentration of power than I am about loss of control (though tbc I think both are very real and important concerns). But I still think this kind of defense of Anthropic’s bill misses the mark. This person is citing the Lieu and Moran “Kill Switch” bill as an example of a law that doesn’t take concentration of power seriously enough. That bill only gives the executive the authority to shut down deployment *after* the model has caused a catastrophe (IIRC they use a similar 50 deaths or serious injuries / billion in damages threshold). It’s true that emergency shutdown provisions have to strike a balance between granting necessary emergency response authorities and…