Artificial Intelligence1 min read

OpenAI to limit weapons information for teens on ChatGPT in violence-prevention push

By · Published by Everything Blog

In short

The upcoming safety rollout will restrict practical instructions on weaponry and security systems for underage users, even if the prompts appear harmless. In addition, the platform will restrict details concerning past mass attacks and how weapons were acquired for all users across the service. The system will also automatically cut off sessions and temporarily suspend accounts displaying an ongoing fixation on violence. OpenAI plans to notify parents directly whenever a teenager's account is paused due to repeated violent discussions.

Key points

  • Parents will receive direct alerts if their teenager's account is temporarily paused for…: Parents will receive direct alerts if their teenager's account is temporarily paused for repeatedly discussing violent topics.
  • Policy updates follow findings from Common Sense Media showing parental alert failures re…: Policy updates follow findings from Common Sense Media showing parental alert failures regarding self-harm and eating disorders.
  • Internal company data indicates that teenagers currently spend an average of less than 15…: Internal company data indicates that teenagers currently spend an average of less than 15 minutes each day using the chatbot.

Sources checked by our analysis pipeline: 1

Oct 30 : OpenAI said on Friday it is tightening limits on the weapons-related information ChatGPT provides to teens, even when a question appears harmless, as part of new safeguards aimed at preventing real-world violence.

The move comes as AI companies face growing scrutiny over the risks chatbots pose to young users and the companies' role in detecting threats of harm.

Here are more details:

• OpenAI will restrict practical information on weapons and security systems that could be misused, with the limits applying to teens even when there is no sign of violent intent

• For all users, ChatGPT will be able to automatically end chats where someone appears to be fixating on violence, and OpenAI may temporarily pause accounts after multiple such chats

• The company also said it strengthened its age prediction system to move users it estimates are under 18 into the teen experience, and it plans to notify parents if their teen's account is paused for repeatedly discussing violence

• For all users, OpenAI is limiting detail on past mass attacks, including the weapons used…

Artificial Intelligence