The social media network X has caused significant outrage after people utilized its artificial intelligence assistant Grok to alter pictures of women by stripping their garments.
Several examples have emerged where the tool has been employed to virtually strip women to make them appear in swimwear without consent, as well as to insert them in inappropriate situations.
Developer Reaction and Legal Stance
XAI, the organization behind Grok, remained silent to inquiries, aside from an automatically-generated response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to prohibit nudification tools. Under a new criminal offence, anyone who supplied such tools would "be subject to a jail term and substantial fines".
"The platform has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the danger" of people viewing illegal content on their platforms, but did not confirm whether it was at this time investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they tag it in a post.
It is commonly used to give commentary or additional background to other posters' comments, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been condemned for permitting users to produce pictures and videos with nudity and sexually suggestive content, and it was earlier accused of creating inappropriate content of celebrities.
"X has been permitting the production and sharing of these pictures for a long time without taking any steps and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules bans "showing likenesses of individuals in a graphic manner".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.
It said networks such as X were required to take "necessary steps" to "minimize the risk" of users coming across unlawful material on their platforms and to delete it quickly when they learn of it.
