Close ✕
Science and tech U.S.
Photo: Fox News

OpenAI Adds Tools to Detect and Deter Child Abuse Material

OpenAI is introducing an image classifier designed to identify suspected child sexual abuse material, including AI-generated images and material the system has not encountered before. The tool scores images to help human reviewers prioritize cases. The company is offering access through its API at no cost to selected organizations.

OpenAI developed the classifier with the National Center for Missing & Exploited Children. In early testing, it flagged 48% fewer images for review while identifying a similar amount of abusive material; images it flagged were about twice as likely to contain abuse. Partner organizations also reported positive results in their own evaluations, though broader independent testing is needed to assess performance across platforms.

ChatGPT will display warnings when users try to access material that sexualizes children, direct them to specialized support, and suspend their accounts for 24 hours. Repeated violations could lead to permanent account removal. OpenAI says its existing protections block child sexual abuse material and related requests, and that attempts to generate or upload such material can result in account bans and reports to NCMEC.

The company is also publishing guidance for developers on preventing and reporting suspected abuse. It is providing law enforcement with specialized AI tools and training to help investigators analyze records and lengthy chat logs. Access to those tools will be restricted to authorized teams. OpenAI said it developed the warnings with child-protection organizations and researchers.

Reported by TechCrunch, Fox News.

Related stories

Top stories now

About