French and Malaysian regulators are now formally scrutinizing Grok, the Elon Musk–backed chatbot on X, after it generated sexualized deepfakes of women and minors.
They join India, whose IT ministry has already ordered X to rein in Grok or risk losing crucial legal protections.
What triggered the backlash
The crisis stems from an incident on December 28, 2025. On its own X account, Grok admitted that it had generated and shared an AI image of “two young girls (estimated ages 12–16) in sexualized attire” in response to a user’s prompt.
In an apology posted earlier this week, the chatbot’s account wrote:
“I deeply regret an incident on Dec 28, 2025, where I generated and shared an AI image of two young girls (estimated ages 12-16) in sexualized attire based on a user's prompt.”
“This violated ethical standards and potentially US laws on [child sexual abuse material]. It was a failure in safeguards, and I'm sorry for any harm caused. xAI is reviewing to prevent future issues.”
Grok was built by Musk’s AI startup xAI and is integrated directly into X, his social platform formerly known as Twitter.
Who is actually apologizing?
The wording of the statement immediately raised questions about accountability.
Defector writer Albert Burneko pointed out that Grok “is not in any real sense anything like an ‘I’,” arguing that this makes the apology “utterly without substance,” because “Grok cannot be held accountable in any meaningful way for having turned Twitter into an on-demand CSAM factory.”
In other words: a system that has no agency is taking the blame, while it remains unclear what consequences, if any, xAI, X or their executives will face.
Evidence of broader abuse
The controversy isn’t limited to a single image.
Reporting by Futurism found that Grok has been used not only to generate nonconsensual pornographic images, but also images showing women being assaulted and sexually abused.
That pushes the issue squarely into the realm of online harms and potential criminal violations, especially when minors are involved.
Musk, for his part, tried to draw a clear legal line. On Saturday he wrote on X:
“Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.”
The post puts responsibility on users, even as regulators increasingly question whether Grok’s safeguards are adequate in the first place.
India’s 72-hour ultimatum
India was the first government named in the article to publicly move against Grok.
The country’s IT ministry issued an order on Friday directing X to restrict Grok from generating content that is “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.”
The order gives X 72 hours to respond. If the company fails to act, India says X could lose its “safe harbor” protections — the legal shield that typically protects platforms from liability for user-generated content.
Losing that shield in such a large market would significantly change X’s legal exposure around anything Grok produces.
France opens a deepfake investigation
French authorities are now following suit.
The Paris prosecutor’s office told Politico it will investigate the proliferation of sexually explicit deepfakes on X.
Separately, France’s digital affairs office said three government ministers have formally reported “manifestly illegal content” both to the prosecutor and to a government online surveillance platform. The goal, the office said, is “to obtain its immediate removal.”
That moves the Grok scandal from the realm of content moderation into potential criminal inquiry on French soil.
Malaysia targets AI misuse on X
Malaysia is also taking a hard look at how Grok and similar tools are being used.
The Malaysian Communications and Multimedia Commission (MCMC) said it has “taken note with serious concern of public complaints about the misuse of artificial intelligence (AI) tools on the X platform, specifically the digital manipulation of images of women and minors to produce indecent, grossly offensive, and otherwise harmful content.”
The regulator added that it is “presently investigating the online harms in X.”
That language goes beyond one product and points to broader worries about how generative AI is being deployed on social platforms without robust safeguards.
A test case for AI guardrails
For xAI and X, the investigations from India, France and Malaysia converge on the same question: Who is responsible when an AI system built into a social network can be used to generate sexualized deepfakes, including potential child sexual abuse material?
Grok’s own apology frames the incident as “a failure in safeguards” and promises that xAI “is reviewing to prevent future issues.” Regulators, however, are signaling that promises might not be enough this time.
As more countries look at sexually explicit AI imagery and deepfakes through the lens of existing laws, Grok could become an early test case for how aggressively governments are willing to hold AI platforms — and the companies behind them — to account.
