Starting today, US law requires tech platforms to have reporting flows that allow for the rapid removal of synthetic non-consensual intimate imagery (SYNCII). The obligations include:
A clear and accessible process for users to report SYNCII
A requirement to remove verified SYNCII within 48-hours of a user’s report
The removal of duplicates or identical copies.
These obligations, enshrined in the Take It Down Act, were reiterated last week by the chair of the Federal Trade Commission in a letter to CEOs of twelve companies subject to the law.
To test platform compliance, Indicator reviewed the reporting flows and other online forms for 16 apps and digital media platforms three times between May 12 and May 19.
By the legal deadline, we were able to locate some form of reporting mechanism for SYNCII in all but one of the platforms reviewed. We could not locate a dedicated form for Apple, nor did we receive a response to our request for information from the company.
Here’s the overview of our findings, which we will update if platforms roll out new features shortly after publication.
Platform | Logged out link | Description |
|---|---|---|
Apple (App Store) | Not found | Logged-in flow to report an app includes a “pornography” option but no specific dropdown for SYNCII. |
Bluesky | Not found | Must be logged in to report. Select “Adult content,” then “Adult deepfake content” |
Discord | Reporting intimate images includes content that is “digitally manipulated depicting you” | |
Google Search | Request removal under “nudity or sexual material,” second step allows selection of “Content falsely portrays me in a sexual act, or in an intimate state.” | |
In-app flow is available at the bottom of the reporting screen. Direct link also available. | ||
Not found | In-app flow allows flagging of non consensual intimate imagery. Platform told Indicator that logged-out users can submit a report through the Help Center starting May 19. | |
In-app flow mentions that NCII can be “real and AI generated”. Online form doesn’t mention deepfakes. | ||
In-app NCII reporting option includes “fake or lookalike pornography”. | ||
Roblox | Not found | Platform told indicator that starting May 19 it will “provide a direct tool both within the platform and on our website,” to submit NCII removal requests. |
Snap | No in-app option found. Tried under “Nudity & sexual content,” “They leaked/are threatening to leak my nudes” and “Other” reporting flows but none included deepfake-specific option as of May 19. | |
Tinder | Selecting “I have a safety or privacy concern” in the dropdown allows nested option “Intimate images shared without consent (including deepfakes)” | |
Tumblr | In-app flow allows reporting for non consensual intimate imagery specifically under TIDA. No mention of SYNCII. | |
TikTok | Couldn’t locate a dedicated option in the reporting flow inside the app. Online form allows for removal of explicit deepfakes “regardless of whether the individual is believed to have provided consent”. | |
Twitch | Not found | The in-app flow only calls out non-consensual exploitative images, but the company has indicated in the past this covers SYNCII as well. |
X | In-app flow allows flagging “Report content under the US Take It Down Act” after selecting “Private or non consensual content.” | |
YouTube | No in-app flow detected, but the privacy complaint process allows for content to be reported because it is an AI-generated deepfake; there is also a dedicated form tied to the Take It Down Act. |
Reporting flows vary significantly across platforms and aren’t always easy to find. Some platforms, like Bluesky, nest deepfake nude removals within the “Adult content” reporting flow and do not appear to offer a way to report SYNCII without having a profile on the platform.
Others, like Reddit, represent SYNCII as “fake” pornography that can be reported in the same flow as real non-consensual intimate media.
Instagram’s in-app reporting flow currently includes a small note at the bottom of the screen that says “in the US, you can create a detailed report for something that contains intimate imagery.” That leads to a form outside the app.
Several platforms told Indicator that they were in favor of the law when it was proposed and are committed to enforcing it.
“We continue to support the Take It Down Act,” said Cindy Southworth, head of women’s safety at Meta in an emailed statement. She called TIDA “an important step in addressing [intimate image abuse] across the internet, and we’ve already been compliant for several months.”
Other obligations
Providing a reporting form is only the first step towards reducing the prevalence of deepfake abuse online.
Victims need to be able to find the reporting flows in search results, which isn’t necessarily straightforward. A review by the Cornell Tech Master’s student Rachel Keels found that such forms can be pushed lower in results by ads for paid services that in some cases don’t specialize in SYNCII.
The content then needs to be handled correctly. When researchers tested X’s reporting mechanism in 2024, they found that the platform failed to remove the deepfake nudes that were flagged. The new legal obligations will hopefully change that.
Platforms also have to identify and remove duplicates. Southworth told Indicator that “Meta was the first company to use hashing technology to help find and remove identical copies of known adult NCII, and prevent new copies from being shared.”
In addition, a lot of SYNCII circulates in private groups via disappearing messages that can’t be accessed by victims, or on platforms that are unlikely to seriously commit to removing it.
Potential drawbacks
Like any law, Take It Down is an imperfect tool. And some researchers worry that it may be counterproductive.
Riana Pfefferkorn, a policy fellow at the Stanford Institute for Human-Centered Artificial Intelligence (HAI) said that the law’s short turnaround process, threat of substantial fines, and good-faith immunity for wrongly removed content create a “shoot first, ask questions never” dynamic. The incentive structure “is in the direction of just remove it, remove it, remove it,” she said.
Pfefferkorn is particularly concerned about weaponization against trans people, sex workers whose content is consensual, and political speech that the administration disagrees with. (In a speech to Congress last year, Trump said “I’m going to use that bill for myself too, if you don’t mind, because nobody gets treated worse than I do online.”)
These concerns led a range of internet rights organizations to speak out against the law when it was being debated.
Pfefferkorn also expects enforcement to be politically uneven: platforms close to the administration, particularly X, may face little consequence regardless of compliance, while more politically vulnerable platforms scramble to be seen as trying.
On the plus side, the Take It Down Act enters into full force with rare bipartisan backing and a clear victim-centered purpose: to give people a federal mechanism to remove real or synthetic intimate imagery they never consented to, and/or that they never agreed could be shared. Whether it delivers on that promise will depend less on the statute than on how platforms and the US government choose to enforce it.
“It’s really hard to predict,” Pfefferkorn said, “and I’m interested as anybody else to see how it plays out.”
