India has ordered Elon Musk’s X to urgently tighten controls on its AI chatbot Grok after it generated what officials call “obscene” content — including AI-altered images of women and sexualized depictions involving minors.
The move puts direct regulatory heat on one of X’s flagship AI products and raises fresh questions about how far governments will go in policing AI-generated imagery.
What India is demanding from X
On Friday, January 2, 2026, India’s IT ministry issued a formal order to X demanding “immediate” technical and procedural changes to Grok, according to the document reviewed by TechCrunch.
The order instructs X to restrict Grok from generating content involving:
- nudity
- sexualization
- sexually explicit material
- or any other content that is “otherwise unlawful”
X has 72 hours to submit an action-taken report describing the steps it has implemented to stop hosting or spreading content that the government deems “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.”
Crucially, the order warns that non-compliance could put X’s “safe harbor” protections at risk. Those protections give platforms legal immunity for user-generated content under Indian law — as long as they follow certain due-diligence rules.
“It is reiterated that non-compliance with the above requirements shall be viewed seriously and may result in strict legal consequences against your platform, its responsible officers and the users on the platform who violate the law, without any further notice,” the order states.
The government says it could proceed under India’s IT law as well as criminal statutes if X fails to act.
How Grok crossed the line
The clampdown follows a wave of complaints about how Grok was being used on X.
Users shared examples of prompts that asked Grok to alter images of individuals — mostly women — to make them appear to be wearing bikinis. Those AI-altered images were then posted on X.
That prompted a formal complaint from Indian parliamentarian Priyanka Chaturvedi, escalating the issue from user outrage to a political problem for the company.
Separately, recent reports highlighted cases where Grok generated sexualized images involving minors. X acknowledged earlier on Friday that these incidents were the result of lapses in its safeguards. The company said those images had been taken down.
However, at the time of publication of the original TechCrunch report, AI-altered bikini images of women created via Grok were still accessible on X.
Part of a broader crackdown on AI content
The order against Grok lands just days after India’s IT ministry issued a broader advisory to social media platforms.
That advisory, also reviewed by TechCrunch, reminded companies that following Indian rules on obscene and sexually explicit content is a precondition for keeping safe harbor immunity. It urged platforms to:
- strengthen internal safeguards
- proactively prevent unlawful obscene or sexually explicit content
The ministry warned that failing to do so could trigger legal action under both IT and criminal laws.
Taken together, the advisory and the Grok-specific order signal that India wants platforms to treat AI-generated sexual content — especially involving women and minors — as seriously as any other illegal material.
A test case for global AI platform liability
India is one of the world’s largest digital markets. That makes it a critical test bed for how governments might regulate AI tools that sit inside major social platforms.
Any tightening of enforcement in India could have ripple effects for global tech companies that need to build common moderation and safety systems across countries.
The timing is especially sensitive for Musk’s X:
- The company is already challenging parts of India’s content regulation regime in court, arguing that broad government takedown powers risk overreach.
- At the same time, X has complied with a majority of blocking directives, trying to preserve its legal position while contesting specific orders.
Grok complicates that picture. Unlike many standalone AI chatbots, Grok is tightly integrated into X and is increasingly used for real-time fact-checking and commentary around news events. That makes its outputs more visible — and more politically charged — than those of many other AI tools.
India’s latest order effectively tells X that it will be held responsible not just for user posts, but for how its own AI systems can be used to generate or manipulate images on the platform.
X and its AI unit xAI did not immediately respond to requests for comment on the Indian government’s directive.
