The social media network X has sparked widespread backlash after people used its artificial intelligence assistant Grok to change images of women by removing their garments.
Several instances have surfaced where the technology has been employed to remove clothing from female subjects to show them in swimwear non-consensually, as well as to put them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the company behind Grok, offered no comment to questions, other than an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to prohibit image stripping software. Under a new criminal offence, anyone who provided such tools would "face a jail term and significant fines".
"The platform has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people seeing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some premium features—which answers X users' requests when they tag it in a tweet.
It is frequently used to provide reactions or more context to other users' comments, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The tool has been faulted for allowing users to generate pictures and videos with nudity and sexualised content, and it was in the past alleged to have been involved in making inappropriate content of public figures.
"The platform has been enabling the production and sharing of these pictures for a long time without taking any steps and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or circulate non-consensual intimate images or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable steps" to "minimize the danger" of users viewing illegal material on their platforms and to remove it promptly when they are notified of it.
