The report was written by Gladstone AI, a company that runs technical briefings on AI for government employees, and was delivered to the State Department in February. It identifies two categories of risk: “weaponization risk,” where AI systems could be used to execute catastrophic attacks, and “loss of control” risk, where advanced AI systems may outmaneuver their creators. The authors argue that these risks are exacerbated by the race dynamics in the AI industry, where the first company to achieve artificial general intelligence (AGI) will reap the majority of economic rewards, incentivizing companies to prioritize speed over safety.
Key takeaways:
- A report commissioned by the U.S. government warns of substantial national security risks from artificial intelligence (AI), including a potential “extinction-level threat to the human species.”
- The report recommends a set of policy actions that would disrupt the AI industry, including making it illegal to train AI models using more than a certain level of computing power and outlawing the publication of the “weights,” or inner workings, of powerful AI models.
- The report was written by Gladstone AI, a company that runs technical briefings on AI for government employees, and was commissioned by the State Department as part of a federal contract worth $250,000.
- The report's recommendations include the creation of a new federal AI agency, tightening controls on the manufacture and export of AI chips, and channeling federal funding toward “alignment” research that seeks to make advanced AI safer.