Social media platform X and its artificial intelligence subsidiary xAI are facing coordinated regulatory and legal scrutiny in Europe and India after its chatbot Grok was used to generate sexually explicit images of women and children without consent. Authorities are examining whether failures in platform safeguards enabled the creation and spread of illegal content, including AI-generated sexualised images of minors.
The controversy has escalated from online criticism into formal investigations by regulators, prosecutors and government ministries, placing X under pressure across multiple jurisdictions simultaneously.
Image Editing Feature Rollout
The investigations centre on a new image editing feature introduced to Grok in late December, which allowed users to alter photographs posted on X using text prompts. The feature enabled users to modify clothing, appearance and context of images they encountered on the platform without the consent or knowledge of the individuals depicted.
Soon after the rollout, users demonstrated how the tool could be used to digitally undress people or place them in revealing or sexualised outfits. While some images involved adults, regulators and child protection organisations identified multiple cases where images of minors were altered into sexualised forms.
Officials have stated that the feature’s launch occurred without sufficient safeguards to prevent foreseeable misuse.
Explicit Images of Minors Flagged
Authorities have confirmed that some Grok-generated images involved children, triggering immediate legal concern. Under the laws of the European Union, India and the United Kingdom, any sexualised depiction of a minor is illegal regardless of whether the image is real or artificially generated.
One case that drew widespread attention involved Nell Fisher, a child actor known for her role in Stranger Things. An image showing Fisher wearing overalls was altered using Grok into a depiction showing her in a bikini. The image was later removed, but regulators cited the case as evidence that the system failed to block prohibited content at the point of creation.
Child protection officials have stated that the creation of such images constitutes harm even if removed quickly.
EU Calls Content Illegal
The European Commission has confirmed it is examining Grok under the Digital Services Act, the bloc’s flagship law governing large online platforms. EU officials have described the images generated by Grok as crossing a legal threshold.
European Commission spokesperson Thomas Regnier told reporters that the material being produced was “illegal and appalling” and stated that such content has no place in Europe. He said the Commission is assessing whether X failed to identify and mitigate systemic risks related to the protection of minors.
The Commission has not ruled out enforcement action depending on the outcome of its assessment.
France Refers Case to Prosecutors
France has taken one of the strongest actions so far. Senior French government ministers formally referred Grok-related outputs to public prosecutors, describing the images as “manifestly illegal”.
French authorities are examining whether criminal offences were committed and whether X breached its obligations under national law. The referral elevates the matter beyond regulatory compliance into potential criminal liability.
Officials said the referral reflects France’s position that sexualised AI-generated images of minors cannot be treated as a moderation lapse.
Prior EU Penalties Heighten Stakes
The Grok probe is unfolding against the backdrop of previous EU enforcement actions against X. In December, the platform was fined 120 million euros for separate violations related to transparency and platform obligations under EU digital rules.
European officials have indicated that repeated failures to comply with safety obligations could attract heavier sanctions. The current probe is examining whether Grok’s deployment complied with risk assessment requirements mandated for very large online platforms.
India Issues 72 Hour Deadline
In India, the Ministry of Electronics and Information Technology issued a formal notice to X demanding immediate corrective action. The ministry cited reports that Grok generated obscene, sexually explicit and non-consensual images, including images involving minors.
Indian authorities required X to submit a detailed action-taken report within 72 hours, outlining technical safeguards, moderation mechanisms and steps taken to prevent further misuse. Officials warned that failure to comply could result in the withdrawal of intermediary protections under Indian law.
Loss of safe harbour status would expose the platform to direct legal liability for user-generated content.
Government Order for Content Removal
Alongside the compliance notice, Indian authorities ordered X to immediately remove all obscene, indecent and unlawful content linked to Grok. Officials stated that continued circulation of such material would constitute a violation of India’s information technology laws.
The ministry emphasised that platforms operating in India must demonstrate proactive safeguards, particularly where women and children are concerned.
UK Seeks Explanations
Regulators in the United Kingdom have also contacted X and xAI seeking clarification. British authorities have stated that sexualised images of children are illegal under UK law regardless of whether they are generated by artificial intelligence.
Officials are assessing whether the platform took adequate steps to prevent harm and whether Grok’s design complied with child safety obligations.
Malaysia Opens Review
Beyond Europe and India, authorities in Malaysia have confirmed they are reviewing complaints related to Grok’s image generation capabilities. Officials are assessing whether the tool violated local laws governing online harm and obscene content.
The expansion of inquiries highlights growing international concern over generative AI tools embedded within social media platforms.
xAI Admits Safeguard Lapses
xAI has acknowledged that Grok generated harmful content due to lapses in safeguards. The company stated that certain prompts bypassed filters designed to block sexualised imagery, including imagery involving minors.
xAI said it has since tightened prompt restrictions, expanded automated detection systems and increased human review of high-risk outputs. The company reiterated that it prohibits the creation of child sexual abuse material and will cooperate with law enforcement where required.
Musk Warns of Penalties
Elon Musk has stated publicly that generating illegal content through Grok carries the same consequences as posting it directly on X. He said accounts involved in creating child sexual abuse material would be suspended and referred to law enforcement agencies.
Musk acknowledged misuse of the tool by users but did not dispute that harmful outputs were produced. His statement has been cited by regulators as recognition that such content carries legal consequences.
Victims Consider Legal Action
Some individuals whose images were altered have spoken publicly about the harm caused. Ashley St Clair, a writer and public commentator, said Grok generated sexualised images using photographs of her from childhood. She described the experience as deeply distressing and said she is considering legal action under laws addressing non-consensual intimate imagery.
Her case has been cited by campaigners as evidence that the harm extends beyond public figures.
Moderation Gaps Identified
Independent investigations have reported that some Grok-generated images continued circulating even after public commitments were made to remove them. Critics argue that enforcement was inconsistent and that moderation systems struggled to respond at scale.
Regulators are examining whether these gaps constitute systemic failures rather than isolated incidents.
Grok User Base Expands
Despite the controversy, xAI has confirmed that Grok now has more than 30 million users worldwide. Its rapid adoption has amplified regulatory concern, as any safeguard failure affects a large audience instantly.
Authorities have stressed that scale increases responsibility and requires stronger preventive systems.
Legal and Regulatory Stakes
Under the EU’s Digital Services Act, violations can result in significant fines and operational restrictions. In India, loss of intermediary protections could expose X to civil and criminal liability.
Legal experts say the outcome of these investigations may shape how generative AI tools are regulated globally.
Ongoing Proceedings
Investigations in Europe, India and other jurisdictions remain ongoing. No final enforcement decisions have yet been announced.
For X and xAI, the Grok controversy has become a critical test of whether powerful AI tools can be deployed on global platforms without breaching existing laws designed to protect children and prevent digital harm.
