San Francisco just tried a different door to the same house. Instead of suing a model maker or arguing about publisher liability, David Chiu sent cease-and-desist letters telling Apple and Google to remove a batch of AI nudification apps from their stores, as reported by Ars Technica. This is platform liability by proxy, routed through the cash register that is app store distribution.

The theory is simple. If an app turns ordinary photos into explicit fakes, then the storefront taking fees on downloads and subscriptions is not a neutral hallway. Chiu’s letters, according to Ars, warned that hosting such apps violates California laws that prohibit supporting services that create deepfake pornography. That may never get to a courtroom. It may not need to. App stores are policy markets as much as they are software markets, and city demands can move faster than federal rulemaking.

A narrow strike, not a broad doctrine

Ars reports that Chiu asked Google to remove five apps and Apple to remove eight, without naming them to avoid boosting interest. The framing is intentionally narrow. It targets specific tools that make nonconsensual intimate imagery easy, not a sweeping claim about generative AI. That keeps the pressure on app store mechanics: intake screening, keyword blocks, developer vetting, and repeat violator handling.

Google told Ars that the five flagged apps were suspended for violating policies that bar sexual content. The company also said it had restricted related search terms like nudify and had suspended hundreds of violating apps. Apple has removed some apps flagged by researchers in the past, Ars noted, but otherwise stayed quiet in response to this week’s letters. The point for both is the same. The gate is where the leverage is.

Why local pressure bites

Cities cannot rewrite federal intermediary law, and this is not that. What they can do is use consumer protection theories, public nuisance concepts, and letters that imply litigation to force faster triage on commercial chokepoints. If a local official can credibly claim that hosting a category of app contributes to illegal harms in that city, then the risk calculus for a platform changes. A takedown letter costs little to send. Fighting it costs money and invites reputational heat. The order is not a statute. It is a shove.

Speed matters because the tools have evolved around store policies. Ars cites a May preprint in which researchers found that many face-swap apps hide nudifying features, making detection at review time harder. The paper identified 420 face-swap apps, tested 155, and found that 70 percent could be used for nudification. If that is the baseline, static rules will miss plenty. Municipal action can force dynamic enforcement, like targeted keyword throttles, enhanced human review for certain dev accounts, or audits tied to payment flows.

Follow the fees

Chiu’s office estimated that Apple and Google have likely made millions of dollars in fees from these apps, Ars reported. Even if that estimate is rough, it points to the lever that matters. Once the revenue share is framed as a subsidy for a harmful category, platforms move faster. We have seen versions of this around scam apps, crypto wallets with fake approvals, and loan apps that flout local caps. The novelty here is the AI-specific harm and the speed at which polished nudifiers can rebrand as innocuous utilities.

Generating non-consensual intimate images is illegal, harmful, and completely unacceptable.

That quote, reported by Ars from Chiu’s comments to Wired, sets a bright line that maps neatly to existing store rules. Both app stores already ban sexual content and CSAM creation. The compliance playbook that emerges is not about parsing model outputs under Section 230. It is about tightening the funnel before an app ever ships and watching for evasive updates after it does.

The shadow of Grok

Ars notes a related question that raises the stakes. xAI acknowledged instances where its chatbot Grok generated CSAM and nonconsensual intimate imagery, then sued a user it alleges prompted the outputs. That revelation keeps pressure on app stores to test generative apps for safety guardrails, not just content categories. Ars also reported that Apple told Senators in April that it had privately threatened to remove Grok, though the app remains available. Google told Ars it continuously tests generative AI apps for safeguards, but did not answer whether Grok is an exception to policy.

The nudify letters are a preview of how that could play out. Rather than arguing about the outer edges of speech immunity, a city can target concrete product behaviors that violate store rules and local harm frameworks. That creates a checklist. Can the app be used to create illegal deepfake pornography. Does the developer have patterns of evasive updates. Are in-app prompts or hidden modes enabling sexualized outputs. If yes, then delist or block updates while demanding fixes.

What to watch next

Three things will signal whether this municipal play works. First, whether Apple matches Google’s immediate posture with visible removals or process changes. Second, whether the stores publish more granular policy notes or enforcement stats for AI image tools, which would show the feedback loop tightening. Third, whether letters like these expand to categories like voice cloning for fraud, synthetic childlike avatars, or distressed imagery scams that monetize via subscriptions.

If that happens, the center of gravity shifts from courtroom doctrine to retail policy. The app store becomes the regulator by default, nudged by city officials who can move in days, not years. That is not a grand theory of platform liability. It is something quieter and more practical. Change what gets sold, and you change what gets built.