Stanford researchers find Mastodon has a massive child abuse material problem

Mastodon, the decentralized network viewed as a viable alternative to Twitter, is rife with child sexual abuse material (CSAM), according to a new study from Stanford’s Internet Observatory (via The Washington Post). In just two days, researchers found 112 instances of known CSAM across 325,000 posts on the platform — with the first instance showing up after just five minutes of searching.

To conduct its research, the Internet Observatory scanned the 25 most popular Mastodon instances for CSAM. Researchers also employed Google’s SafeSearch API to identify explicit images, along with PhotoDNA, a tool that helps find flagged CSAM. During its search, the team found 554 pieces of content that matched hashtags or keywords often used by child sexual abuse groups online, all of which were identified as explicit in the “highest confidence” by Google SafeSearch.

There were also 713 uses of the top 20 CSAM-related hashtags across the Fediverse on posts that contained media, as well as 1,217 text-only posts that pointed to “off-site CSAM trading or grooming of minors.” The study notes that the open posting of CSAM is “disturbingly prevalent.”

One example referenced the extended mastodon.xyz server outage we noted earlier this month, which was an incident that occurred due to CSAM posted to Mastodon. In a post about the incident, the sole maintainer of the server stated they were alerted to content containing CSAM but notes that moderation is done in his spare time and can take up to a few days to happen — this isn’t a giant operation like Meta with a worldwide team of contractors, it’s just one person.

While they said they took action against the content in question, the host of the mastodon.xyz domain had suspended it anyway, making the server inaccessible to users until they were able to reach someone to restore its listing. After the issue was resolved, mastodon.xyz’s administrator says the registrar added the domain to a “false positive” list to prevent future takedowns. However, as the researchers point out, “what caused the action was not a false positive.”

“We got more photoDNA hits in a two-day period than we’ve probably had in the entire history of our organization of doing any kind of social media analysis, and it’s not even close,” David Thiel, one of the report’s researchers, said in a statement to The Washington Post. “A lot of it is just a result of what seems to be a lack of tooling that centralized social media platforms use to address child safety concerns.”

As decentralized networks like Mastodon grow in popularity, so have concerns about safety. Decentralized networks don’t use the same approach to moderation as mainstream sites like Facebook, Instagram, and Reddit. Instead, each decentralized instance is given control over moderation, which can create inconsistency across the Fediverse. That’s why the researchers suggest that networks like Mastodon employ more robust tools for moderators, along with PhotoDNA integration and CyberTipline reporting.