Briefing: Grok brings nonconsensual image abuse to the masses

This piece was updated to reflect X’s partial restriction on Grok image generation.

We are well into the second straight week of X users using Grok to flood Elon Musk’s platform with nonconsensual sexualized images.

Researcher Genevieve Oh found that Grok produced about 6,700 sexually suggestive images on X in a 24-hour period. In what it described as a “conservative” estimate, AI detection company Copyleaks identified “roughly one nonconsensual sexualized image per minute in the observed image stream” on December 31. Another researcher cataloged almost 500 cases.

Many of the images aren’t “just” nonconsensual. They are abusive, macabre, and offensive. Grok put a bikini on the corpse of Renée Good, the woman killed by an ICE agent in Minneapolis on January 7. It accepted a prompt to add a swastika bikini to the photo of a Holocaust survivor. It engaged with requests to represent women in scenarios that evoked physical assault.

What’s visible on X is just one part of it. Grok’s standalone app has also been used to generate sexual imagery of children aged 11 to 13, according to the Internet Watch Foundation. As Emanuel Maiberg wrote on 404 Media, Telegram users exchange tips on how to generate “convincing and lurid” sexual deepfakes and share their creations. One abuser grumbled that “too many people using grok under girls post are gonna destroy grok fakes. Should be done in private groups.”

The man responsible for all this, Elon Musk, tweeted once about the fiasco, writing that “anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.” He did not take responsibility for Grok’s failures. A statement from X’s Safety team focused on child sexual abuse material, even though synthetic nonconsensual intimate imagery is also illegal in many countries. Only on Friday, January 9, did X take a minor step towards stemming the abuse by restricting some of the image generation features to paying users.

Musk has spent considerably more time boasting about Grok’s recent growth, which presumably has been driven in part by the surge in abusive prompts. He published or reshared at least 30 celebratory posts about Grok’s user base and product features between January 7 and January 8. He hasn’t appeared to express remorse for the widespread abuse unleashed on X users. This is the same man who, in 2024, called biased responses from Google’s Gemini chatbot “extremely alarming” and posted about them nonstop.

Governments around the world are paying attention. Australia’s eSafety commissioner Julie Inman-Grant said that “we will use this range of regulatory tools at our disposal to investigate and take appropriate action.” British, French, German, and Indian government officials all threatened consequences. The European Commission called Grok’s outputs “illegal” and requested that X retain relevant documents through the end of 2026. None of these regulators were impressed by the limited step of restricting image generation to paying subscribers.

All of this is likely of little concern to Musk for as long as he maintains the support of the American government. Disseminating nonconsensual deepfake intimate imagery is a criminal offense in the United States under the 2025 Take It Down Act. The law’s co-sponsor Ted Cruz wrote that “many of the recent AI-generated posts are unacceptable and a clear violation of my legislation,” adding that “guardrails should be put in place.” Yet Cruz also claimed to be “encouraged that X has announced that they're taking these violations seriously.”

Take It Down requires platforms to take down nonconsensual deepfake imagery within 48 hours of being notified, but this provision only goes into effect in May. Based on an audit that showed X consistently removed nonconsensual content when requested under copyright laws, the law will likely help reduce the presence of nonconsensual nudes on the platform. But it’s not clear to me that bikini pictures would be eligible for removal. Plus, this system places the burden on victims to report content, rather than requiring generators to prevent it from being created in the first place.

Safety guardrails don’t appear to be top of mind for Musk. CNN reported that the latest bout of Grok-enabled abuse followed a meeting where the xAI CEO expressed frustration about restrictions that had been placed on the chatbot’s image and video generation capabilities. Ashley St Clair, the mother of one of Musk’s many children, was also targeted by Grok’s undressing spree. She claims to have lost her monetization privileges for speaking up about it.

Grok’s bikini requests on X will probably peter out as the trend dissipates and the removal requests catch up. Already a sizable part of the visible feed appears to be driven by adult content creators [NSFW] – including AI-generated ones [NSFW!] – trying to ride the surge in attention by requesting Grok to nudify their own pictures.

But it’s not just Grok. Wired found that Google and OpenAI’s chatbots have also been used to generate nonconsensual sexualized images. Dedicated AI nudifiers make millions and reach millions. Mass generation of nonconsensual sexualized harassment is a fixture of the current online ecosystem, and it will take a lot of work to change that. – Alexios

A recent graphic from the US National Weather Service left people in Camas Prairie, Idaho puzzled. The wind forecast contained typical information about expected gusts and impacts. But it also included details for “Orangeotild” and “Whata Bod” — two places that don’t exist.

The NWS, which experienced significant layoffs last year thanks to Elon Musk’s Department of Government Efficiency, later admitted that the graphic was created with AI, according to the Washington Post.

“The blunder — not the first of its kind to be posted by the NWS in the past year — comes as the agency experiments with a wide range of AI uses, from advanced forecasting to graphic design,” the paper reported.

Futurism also amusingly noted that the hallucinated town of “Whata Bod” sounds “more like an old timey dirty joke than an actual place where people live.”

Indicator is a reader-funded publication.