OpenAI Launches "Preparedness" Division to Address Potential AI Risks

OpenAI, the driving force behind the renowned artificial intelligence system ChatGPT, is set to embark on a new journey

by Faruk Imamovic
OpenAI Launches "Preparedness" Division to Address Potential AI Risks
© Getty Images News/Win McNamee

OpenAI, the driving force behind the renowned artificial intelligence system ChatGPT, is set to embark on a new journey: the establishment of a specialized division focusing on the prevention of AI-induced catastrophes.

Navigating the New Frontier of AI Safety

Unveiled on October 25th, the new initiative named "Preparedness" signifies OpenAI's commitment to not only advancing AI capabilities but also ensuring their responsible utilization.

This fresh division will channel its efforts into anticipating and mitigating threats associated with the misuse of AI. Particularly, the team will be delving into challenges posed by AI in relation to chemical, biological, radiological, and nuclear threats.

Equally pressing will be the risks associated with individualized persuasion, cybersecurity, and AI's capacity for autonomous replication and adaptation. Under the leadership of Aleksander Madry, the Preparedness team is poised to tackle some pressing questions.

Foremost among these is assessing the danger of pioneering AI systems when misused and gauging the possibility of malicious entities deploying stolen AI model weights. In a statement released by the company, OpenAI underscored the potential advantages and pitfalls of emerging AI systems: “We believe that frontier AI models, which will exceed the capabilities currently present in the most advanced existing models, have the potential to benefit all of humanity”.

But, they didn't shy away from addressing the looming dangers, noting that these models also herald "increasingly severe risks."

Building a Future-Proof AI Ecosystem

OpenAI's proactiveness doesn't end with mere acknowledgment.

As part of their dedication to safety, they've proclaimed, “We take seriously the full spectrum of safety risks related to AI, from the systems we have today to the furthest reaches of superintelligence. [...] To support the safety of highly-capable AI systems, we are developing our approach to catastrophic risk preparedness”.

In a move to bolster its Preparedness division, OpenAI is on a talent hunt, scouting for individuals from varied technical backgrounds. Further underlining their commitment to this cause, they've also introduced the AI Preparedness Challenge.

This challenge, centered on the prevention of catastrophic misuse, offers an enticing reward: $25,000 in API credits for the top 10 innovative submissions. This bold move follows a previous announcement made in July 2023, where OpenAI hinted at the formation of a team specifically oriented towards countering potential AI threats.

Now, with the inception of the Preparedness team, the company is walking the talk, ensuring that the growth of AI is paired with rigorous safeguards.