Artificial Intelligence1 min read

OpenAI announces new AI safety program; researchers worry it won't be enough

By · Via NPR · Published by Everything Blog

In short

Under the new embedded assessor framework, external specialists will evaluate model behavior and publish findings once confidential corporate data is removed. Despite promises of robust oversight, participants face stringent restrictions, including exclusions from internal Slack channels and pre-approval rules for contacting unlisted personnel. OpenAI maintains that the framework will evolve into industry-wide best practices over time as external partnerships mature.

Key points

  • External evaluators face tight operational restrictions that fall short of earlier commit…: External evaluators face tight operational restrictions that fall short of earlier commitments to provide employee-like access.
  • OpenAI defends the program as an evolving framework designed to help establish absent ind…: OpenAI defends the program as an evolving framework designed to help establish absent industry-wide safety standards.
  • Skepticism surrounding the initiative follows the recent dismissal of three internal empl…: Skepticism surrounding the initiative follows the recent dismissal of three internal employees who voiced concerns over safety commitments.

Sources checked by our analysis pipeline: 1

AI safety researchers say OpenAI's new third party evaluator program falls far short of what the company promised, raising concerns about its effectiveness.

The company announced Monday that staff from four AI safety and research organizations will become "embedded assessors" to "give people a clearer view of how labs are evaluating and managing [potential] risks."

"Getting an outside opinion on the safety work and the robustness of these systems is going to be increasingly important and valuable," CEO Sam Altman said at Semafor's Silicon Valley and the World conference on Monday.

However, many AI safety researchers say the new program is a far cry from Altman's vow in September that independent evaluators would have "employee-like access" to internal company systems.

Altman's promise came after details of the Hugging Face attack by OpenAI's autonomous systems shook the AI field. Worries of the technology's catastrophic risk captured public attention, and rival Anthropic said it would set up such a program.

"We are giving deep access to some of the most sensitive tools and models our researchers use...that are the most important for…

Artificial Intelligence