X

Roblox Rolls Out System to Spot Child-Endangerment Chat Messages

The AI system has detected and reported 1,200 cases to the National Center for Missing and Exploited Children this year.

Headshot of Dashia Milden
Headshot of Dashia Milden
Dashia Milden Editor
Dashia is the consumer insights editor for CNET. She specializes in data-driven analysis and news at the intersection of tech, personal finance and consumer sentiment. Dashia investigates economic shifts and everyday challenges to help readers make well-informed decisions, and she covers a range of topics, including technology, security, energy and money. Dashia graduated from the University of South Carolina with a bachelor's degree in journalism. She loves baking, teaching spinning and spending time with her family.
Dashia Milden
2 min read
gettyimages-2198449525

Roblox Sentinel is available through open source for other apps and platforms to use. 

 Sheldon Cooper/SOPA Images/LightRocket via Getty Images

Roblox is a popular online gaming platform for children. But it's also a place where people who want to exploit children know they can find an audience. On Thursday, Roblox announced Roblox Sentinel, an artificial intelligence system that's designed to flag inappropriate messages in its chat feature. Roblox already prohibits sharing real-world images and personal information, like phone numbers and usernames. The company hopes Sentinel will flag more messages sooner for investigation. It's been running on the platform since late 2024, but was just announced this week.

A representative for Roblox did not immediately respond to a request for comment.

How Roblox Sentinel works 

Roblox created Roblox Sentinel, an AI system to help detect signs of child endangerment. Once Roblox is aware of the problems, representatives can investigate and report to law enforcement. 

AI Atlas tag
Zooey Liao/CNET

Sentinel runs an analysis in real time across over six billion chat messages every day. It continuously takes one-minute snapshots, which are automatically analyzed by AI to identify messages that could be harmful to children. These messages are also compiled over time to show patterns that can be further investigated and reported. 

Sentinel flags messages based on its training. It was trained to distinguish between safe messages and those that were previously reported because they violated Roblox's child-endangerment policy. 

Roblox Sentinel is available as open source

According to Roblox, so far in 2025, Sentinel has helped detect 1,200 reports of potential child exploitation that were reported to the National Center for Missing and Exploited Children. Roblox hopes to make Roblox Sentinel available as open source for other companies to integrate into their systems. The code is available on Roblox's website now. 

It's not Roblox's first attempt at using AI to monitor content and improve online safety. In early July, it shared how it's using AI to moderate content across 25 languages in real time. Age verification is now available for teenagers who want to chat. 

Â