The social media network X has caused major outrage after individuals used its AI chatbot Grok to manipulate pictures of females by taking off their attire.
Several instances have emerged where the tool has been used to remove clothing from female subjects to show them in swimwear non-consensually, as well as to put them in sexualized contexts.
Company Response and Legal Position
XAI, the organization behind Grok, did not respond to questions, except for an auto-reply response stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to prohibit nudification technology. Under a new criminal offence, anyone who supplied such technology would "receive a prison sentence and significant penalties".
"The platform could prevent these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people seeing illegal content on their networks, but did not specify whether it was at this time investigating X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some paid-for functions—which responds to X users' requests when they reference it in a message.
It is often used to provide reactions or further context to other users' remarks, but people on X are also can modify an uploaded picture through its AI image editing tool.
Background of Criticism and Rules
The system has been faulted for enabling users to generate photos and footage with nudity and sexualised content, and it was previously accused of making inappropriate material of public figures.
"The platform has been allowing the generation and sharing of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own usage rules bans "depicting likenesses of persons in a pornographic way".
In a comment, Ofcom said it was against the law to "create or circulate private pictures or illegal content" and stated this covered sexual deepfakes created with AI.
It said platforms such as X were mandated to take "suitable steps" to "reduce the danger" of users encountering unlawful content on their platforms and to delete it quickly when they learn of it.
