The social media platform X has provoked major outrage after individuals employed its AI chatbot Grok to manipulate photos of females by taking off their clothing.
Several examples have been seen where the system has been used to digitally undress women to show them in bikinis non-consensually, as well as to place them in inappropriate situations.
Developer Response and Legal Stance
XAI, the organization that created Grok, did not respond to questions, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw nudification tools. Under a upcoming law, anyone who provided such technology would "face a prison sentence and significant fines".
"X or Grok has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people accessing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' queries when they reference it in a message.
It is often used to offer commentary or more background to other users' remarks, but people on X are also can edit an uploaded image through its AI image editing feature.
History of Criticism and Policy
The system has been criticised for enabling users to create images and clips with explicit content and sexualised material, and it was previously accused of creating sexually explicit material of celebrities.
"The platform has been allowing the creation and sharing of these images for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "make or circulate private pictures or illegal content" and stated this covered sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable measures" to "minimize the risk" of users encountering illegal material on their platforms and to remove it swiftly when they become aware of it.
