San Francisco Presses Apple and Google to Purge AI Nudify Apps From Their Stores

San Francisco’s city attorney has told Apple and Google to remove 13 apps from their stores that can generate nonconsensual AI nude images, escalating a fight over how the biggest mobile platforms police sexual deepfakes. The legal action matters because officials say the companies are not just hosting the tools, but profiting from them through app sales and in-app fees.

City Attorney David Chiu sent cease-and-desist letters on Thursday demanding the companies delete the apps, cut off developers, and tighten review systems so similar products do not reach consumers again. The move is part of a broader crackdown on the rapidly growing market for so-called nudify tools, which can turn a normal photo into an explicit deepfake in seconds.

Chiu’s office says the apps exploit women and girls, fuel harassment, and may violate California law. Apple and Google, meanwhile, have both said they prohibit pornography and abuse on their platforms and have taken down many such apps before, even as new ones keep appearing.

What San Francisco is demanding from Apple and Google

San Francisco wants more than a few app removals. The city attorney is pressing Apple and Google to stop what his office describes as a business model built around sexual abuse software.

In the cease-and-desist letters, Chiu’s office says the companies should remove the 13 targeted apps, end relationships with their developers, and improve moderation so the products cannot be reintroduced under slightly different names. The letters also argue that the platforms are helping distribute and monetize deepfake pornography in ways that should no longer be tolerated.

“Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu told WIRED.

According to the city attorney, the companies have likely earned substantial revenue from apps that charge users for nudification features. Because Apple and Google take a cut of transactions processed through their stores, the office says they should be treated as financial participants rather than passive hosts.

Chiu’s office further argues that California law bans services that support the production of deepfake pornography. That legal framing is important because it shifts the issue from content moderation to potential liability for enabling a harmful marketplace.

Why nudify apps have become such a persistent problem

Nudify apps keep resurfacing because many of them are designed to look harmless at first glance. They often present themselves as face-swapping, image-editing, or entertainment tools, while offering explicit capabilities after installation or inside the app.

The underlying technology has also become easier to use. In many cases, all a user needs is a single photograph and a few taps to produce a sexualized fake image, sometimes within seconds. As generative AI has improved, the outputs have become more realistic and therefore more damaging.

What makes the problem particularly alarming is the target. The overwhelming majority of these tools are used to create sexual images of women and girls without consent. Advocates and researchers say the result is a form of digital sexual abuse that can humiliate victims, damage reputations, and cause severe psychological harm.

Chiu said the images are used to bully, degrade, and threaten women and girls, adding that victims have described suicidal thoughts after being targeted.

The scale of the issue has become harder to ignore. WIRED and other outlets have documented cases involving minors, including incidents in schools where deepfake sexual abuse images were created of students. The technology is also increasingly available through web apps, bots, and mainstream mobile storefronts, which widens access far beyond niche corners of the internet.

How did Apple and Google respond?

Google said it has already removed hundreds of apps with nudifying capabilities, including the five Android apps singled out by San Francisco. Apple did not comment before publication.

Google’s public response emphasized that the company’s store rules already ban sexual content and that it takes action when violations are reported. It also said it has been working proactively to find and remove harmful apps, including by restricting search terms such as “nudify.”

A Google spokesperson said the company has suspended hundreds of violating apps and blocked related search queries after reviewing reports and taking enforcement action.

Those statements show the tension at the heart of the debate. Both companies can point to policies that theoretically prohibit pornography, abuse, and harassment. But critics say enforcement has been inconsistent, reactive, and too slow to prevent harmful tools from reaching large audiences.

Apple’s silence in this instance is notable, but it is not the first time the company has faced pressure over store moderation. Researchers and watchdogs have repeatedly shown that apps can slide through review systems, change branding, and reappear after removal. The result is a cycle in which platforms respond to public reporting only after the damage has already spread.

What do researchers say about the app-store ecosystem?

Researchers have been warning for months that nudify tools remain easy to find on major app stores, despite repeated takedowns. Their findings suggest the problem is not isolated to a handful of bad actors, but embedded in the design of broad, dual-use image tools.

In January and April, the Tech Transparency Project found roughly 100 apps and advertisements across Apple’s App Store and Google Play that promoted nudifying technology. According to the group, those apps were downloaded hundreds of millions of times and may have generated substantial revenue.

Katie Paul, who leads the Tech Transparency Project, said the repeated discovery of these apps showed that the stores’ safety promises do not match reality.

The group’s findings were especially striking because many of the apps had already been reported before, only to surface again in new forms. That suggests moderation gaps are not just about speed; they are also about the systems used to assess whether a product is likely to be repurposed for abuse.

In May, researchers from Cornell University and Georgetown University published a preprint paper examining face-swapping apps on both platforms. They identified 420 apps that offered general face-swapping features and tested 155 of them for whether they could be used to create nude-image swaps.

In 70 percent of those cases, the researchers found that the apps could indeed be used to make sexual deepfakes, often without any built-in safeguards to prevent abuse. Their conclusion was that many of these products are effectively dual-use: they are sold as benign tools, but they can easily be used to generate harmful content.

How the apps hide in plain sight

The most effective nudify apps rarely advertise themselves directly as sexual abuse tools. Instead, they lean on broad language about face swapping, photo editing, or AI creativity, which helps them pass under platform review systems.

That strategy also makes consumer-facing moderation difficult. A store reviewer may see a generic description and a harmless-looking icon, while the real functionality becomes obvious only after install or after the user begins interacting with hidden menus.

One app described in the city’s investigation has more than 1 million downloads and displays a menu of image styles with names that clearly sexualize the output. Another app markets itself with language promising “free and uncensored” video generation. WIRED did not name the apps to avoid directing traffic toward them.

These examples point to a larger moderation challenge. The platforms are not merely trying to identify explicit material; they are also trying to determine intent, potential misuse, and whether a product’s advertised purpose matches its actual behavior. That is a far harder task than basic keyword filtering.

Why this matters beyond a single app purge

The San Francisco action is significant because it treats nudify apps as a platform responsibility, not just a user misconduct problem. If app stores can host tools that help generate nonconsensual sexual images, the argument goes, then they are part of the supply chain that enables abuse.

That theory could carry broad implications. Mobile marketplaces are central distribution points for software, payments, and customer discovery. If legal or regulatory pressure grows, companies may need to tighten review standards, impose more aggressive developer vetting, and monitor recurring categories of abuse more closely.

The stakes are especially high because explicit deepfakes are becoming easier to produce, harder to trace, and more damaging to victims. A person does not have to publish the image widely to inflict harm; even a single fake nude shared privately can be used for extortion, coercion, or humiliation.

Chiu’s office has already used legal tools against deepfake websites, targeting 16 popular services in earlier enforcement efforts. The new notices against Apple and Google extend that campaign into the mobile ecosystem, where millions of users can discover and pay for these tools through ordinary app-store searches.

Who is most at risk?

Women and girls are the primary targets, according to researchers, advocates, and city officials. Minors are also exposed, which is why reports of school-related deepfake abuse have intensified public concern and sharpened the urgency around app-store enforcement.

The harm is not limited to the creation of an image. Victims can face intimidation, blackmail, school discipline problems, family conflict, and long-term emotional distress. In some cases, the image may remain online indefinitely, making the abuse difficult to fully erase.

Table: Key facts in the San Francisco app-store crackdown

Timeline: How the pressure on nudify apps has built

What happens next?

The immediate question is whether Apple and Google will comply quickly or resist the city’s demands. If they remove the apps, the dispute may fade from the headlines but the underlying problem will remain. If they do not, San Francisco is signaling that it is prepared to consider further legal options.

Chiu said his office plans to continue pursuing the issue and wants stronger screening before apps ever reach consumers. His message is that platform moderation must move earlier in the chain, not only after media reports or watchdog complaints expose a problem.

Chiu said he hopes the companies will act immediately to remove the apps and improve screening, but warned that the city will examine “all of our legal options” if they fail to do so.

For Apple and Google, the episode is another reminder that trust in app stores is not just a marketing message. It is a public promise about safety, and that promise is being tested by software that can inflict intimate, lasting harm with remarkable speed.

The broader challenge is likely to outlast this particular case. As AI-generated imagery becomes more powerful and easier to distribute, the line between harmless creative tools and abuse-enabling software will remain difficult to police. For regulators, researchers, and platform operators alike, the question now is whether that line can be enforced before victims are harmed.

For San Francisco, the answer is straightforward: if the biggest technology companies want the benefits of app-store dominance, they must also accept responsibility for the damage their marketplaces can help cause.