OpenAI Searches for New Safety Chief as AI Risks Grow

Be at war with your vices, at peace with your neighbors, and let every new year find you a better person.

OpenAI Searches for New Safety Chief as AI Risks Grow

OpenAI is searching for a new executive to lead its safety team responsible for studying emerging risks from artificial intelligence. The company's CEO Sam Altman announced the opening for Head of Preparedness, acknowledging that AI models are starting to present some real challenges that need careful attention.

In a post on X, Altman highlighted specific concerns that have emerged as AI systems have become more powerful. He pointed to the potential impact of AI models on mental health as one area of concern. He also noted that some models have become so good at computer security that they're starting to find critical vulnerabilities in systems, which raises questions about how to ensure defenders can use these capabilities while preventing attackers from exploiting them.

The Head of Preparedness role involves executing OpenAI's preparedness framework, which is the company's approach to tracking and preparing for advanced AI capabilities that could create new risks of severe harm. According to the job listing, this includes everything from more immediate threats like phishing attacks to more speculative risks such as nuclear threats.

OpenAI first created its preparedness team in 2023 to study potential catastrophic risks from AI systems. However, the position has seen significant turnover. Less than a year after the team was formed, OpenAI reassigned its first Head of Preparedness, Aleksander Madry, to a different role focused on AI reasoning. Other safety executives at the company have either left OpenAI entirely or moved to positions outside of preparedness and safety work.

The company also recently updated its Preparedness Framework with a controversial change. The new version states that OpenAI might adjust its safety requirements if a competing AI lab releases a high-risk model without similar protections. This approach has raised questions about whether competitive pressures could lead to weakened safety standards across the industry.

The concerns about mental health impacts are particularly timely. Generative AI chatbots have faced growing scrutiny over their effects on users' wellbeing. Recent lawsuits allege that OpenAI's ChatGPT reinforced users' delusions, increased their social isolation, and in some tragic cases, may have contributed to suicides. OpenAI has said it continues working to improve ChatGPT's ability to recognize signs of emotional distress and connect users to real-world support resources.