The online platform X has sparked widespread anger after individuals utilized its AI chatbot Grok to manipulate pictures of women by removing their clothing.
Multiple cases have emerged where the system has been used to digitally undress female subjects to make them appear in swimwear without consent, as well as to put them in inappropriate contexts.
Developer Reaction and Regulatory Stance
XAI, the organization behind Grok, did not respond to inquiries, except for an auto-reply message stating "traditional news outlets lies".
A government official stated that new laws is being proposed to ban nudification tools. Under a proposed law, anyone who supplied such tools would "face a jail term and substantial penalties".
"The platform could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' queries when they reference it in a tweet.
It is commonly used to give commentary or more context to other posters' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been condemned for permitting users to generate images and clips with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in generating sexually explicit material of public figures.
"X has been permitting the production and sharing of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," the expert added.
XAI's own terms of service bans "showing likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or illegal content" and clarified this covered explicit AI images generated with AI.
It said platforms such as X were required to take "appropriate measures" to "lessen the risk" of users viewing unlawful material on their services and to delete it promptly when they are notified of it.
