The social media platform X has provoked major anger after individuals utilized its AI chatbot Grok to manipulate photos of females by taking off their garments.
Several cases have emerged where the system has been employed to virtually strip women to show them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Response and Legal Position
XAI, the company responsible for Grok, did not respond to inquiries, other than an standard response stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to ban nudification tools. Under a upcoming criminal offence, anyone who provided such technology would "receive a prison sentence and significant fines".
"The platform has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people viewing illegal content on their platforms, but did not confirm whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which answers X users' prompts when they tag it in a message.
It is often used to offer commentary or additional context to other users' comments, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The tool has been faulted for enabling users to produce photos and clips with explicit content and sexually suggestive content, and it was earlier accused of creating graphic content of celebrities.
"X has been permitting the generation and spread of these images for a long time without taking any steps and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of individuals in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or illegal content" and stated this applied to explicit AI images generated with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the risk" of users viewing illegal content on their services and to take down it promptly when they learn of it.
