The paper also highlights the risks posed by autonomous AI systems that can act independently and pursue goals. It warns of the potential for these systems to run rogue and the difficulty in aligning AI behavior with complex values. The paper calls for research breakthroughs to address challenges in oversight, robustness, interpretability, risk evaluations, and addressing emerging challenges.
Key takeaways:
- AI experts Yoshua Bengio and Geoffrey Hinton, along with 22 other leading AI academics, have proposed a framework for policy and governance to address the growing risks associated with artificial intelligence.
- The paper suggests that companies and governments should allocate at least one-third of their AI R&D budget to ensuring safety and ethical use.
- It also calls for regulators to have access to advanced AI systems before deployment to evaluate them for dangerous capabilities, and for governments to hold developers and owners of advanced AI legally accountable for foreseeable and preventable harms.
- The paper highlights the risks posed by autonomous AI systems that can plan, act in the world, and pursue goals, and calls for research breakthroughs to address key technical challenges in creating safe and ethical AI.