Governments ban the Grok chatbot on account of nonconsensual bikini pics : NPR

[ad_1] <img src="https://npr.brightspotcdn.com/dims3/default/strip/false/crop/4941x3294+0+0/resize/1100/quality/50/format/jpeg/?url=http%3A%2F%2Fnpr-brightspot.s3.amazonaws.com%2Ff0%2F8d%2Ff5092de2426f8f66fe4e39cedced%2Fgettyimages-2255889313.jpg" data-template="https://npr.brightspotcdn.com/dims3/default/strip/false/crop/4941x3294+0+0/resize/{width}/quality/{quality}/format/{format}/?url=http%3A%2F%2Fnpr-brightspot.s3.amazonaws.com%2Ff0%2F8d%2Ff5092de2426f8f66fe4e39cedced%2Fgettyimages-2255889313.jpg" class="img" alt="The UK communications regulator Ofcom launched a formal investigation into Elon Musk's social media platform X regarding its AI chatbot, Grok following reports that Grok has been used to generate nonconsensual sexual deepfakes." fetchpriority="high"/> The UK communications regulator Ofcom launched a proper investigation into Elon Musk's social media platform X relating to its AI chatbot, Grok following studies that Grok has been used to generate nonconsensual sexual deepfakes. Leon Neal/Getty Pictures disguise caption

toggle caption

Leon Neal/Getty Pictures Indonesia and Malaysia briefly blocked X's chatbot, Grok, over the weekend after it made scores of pretend pictures publicly sexualizing largely girls and, in some cases, kids late final 12 months.

Governments around the globe are additionally launching investigations. The newest got here on Monday because the UK media regulator, Ofcom, launched a probe into the social media platform, which may lead to a ban.

Grok had been producing sexually specific pictures of individuals for a while. However the situation bought widespread consideration in late December as individuals used the chatbot to edit a excessive quantity of present pictures by tagging the bot in feedback and giving it prompts akin to "put her in a bikini." Whereas Grok didn't reply to the entire requests, it obliged in lots of circumstances. In some circumstances, Bellingcat senior investigator and researcher Kolina Koltai famous, customers can get Grok to generate frontal nudes.

Untold numbers of girls and in some circumstances, kids, as Reuters first reported, have had their likenesses sexualized on-line by Grok with out their permission, together with one of many moms of X proprietor Elon Musk's kids.

It is uncommon for therefore many governments to take motion towards a social media firm however this case is completely different, stated Riana Pfefferkorn, a coverage fellow at Stanford College. "Making baby sexual abuse [material] is flagrantly unlawful, just about all over the place on Earth."

By final Friday, X had restricted Grok's AI picture era characteristic to make it solely accessible to paying subscribers. Non-paying customers can nonetheless put individuals in bikinis publicly with just some clicks, however they'll solely put in a couple of such requests earlier than being prompted to enroll in a premium membership, which prices $8 a month.

NPR reviewed Grok's publicly accessible pictures generated earlier this month and located it had stopped making pictures of scantily clad girls a number of days into 2026. Nevertheless, it typically nonetheless presents up bikini-clad males.

xAI, the father or mother firm of X, has been pushing grownup content material with Grok since final 12 months. In Could, Koltai first famous that the chatbot would generate sexually specific pictures in response to requests on X like "take off her garments." This previous summer time, Grok launched "spicy mode" in its standalone app, which allowed customers to place bikinis on AI-generated characters.

Ben Winters, director of AI and privateness on the advocacy group Shopper Federation of America, stated that Grok not solely permits enhancing pictures of actual individuals, nevertheless it additionally offers a simple distribution platform through X. "It is a additional and important escalation," he stated.

Governments are outraged over X's transfer to limit entry to the picture era operate to subscribers. British Expertise Secretary Liz Kendall instructed Sky Information "it's insulting to say that you would be able to nonetheless entry this service should you pay for it."

The Indonesian authorities discovered that Grok lacked efficient guardrails to cease customers from making nonconsensual pornographic content material primarily based on actual Indonesian residents, the Related Press first reported.

"The federal government sees nonconsensual sexual deepfakes as a severe violation of human rights, dignity and the security of residents within the digital house," Indonesian Communication and Digital Affairs Minister Meutya Hafid stated in an announcement.

The AP additionally reported Grok will keep blocked in Malaysia till efficient guardrails towards misuse are put in place.

In response to NPR's questions, X spokesperson Victoria Gillespie pointed to an announcement posted on January 3 that stated "anybody utilizing or prompting Grok to make unlawful content material will endure the identical penalties as in the event that they add unlawful content material." The assertion echoed Musk's personal submit earlier that day.

Such an method is an "try and abdicate accountability, " stated Winters.

"It actually is not only the consumer that's prompting it alone," he stated. "It's the truth that the picture wouldn't be created if not for … the software they made."

Earlier than the wave of Grok-generated nonconsensual specific deepfakes hit X, different AI makers had added related capabilities to their chatbots. In November, Google launched a brand new picture producing mannequin, Nano Banana Professional. In December, OpenAI up to date its mannequin ChatGPT Pictures. Each can even edit pictures to place individuals in bikinis, Wired reported final month. A thread on Reddit distributing such pictures has been taken down.

In one other X submit, Musk prompt that since instruments from different AI corporations even have this undressing operate, the stress from governments is a type of censorship directed at his platform.

Koltai famous the development to make AI-generated nonconsensual intimate media has been within the making for the previous 12 months or two. "You will see these advertisements in Instagram even – it is like, add a photograph of you and your crush and also you guys can kiss," she stated. "In order that's kissing and hugging, and you then see there's that extra excessive spectrum."

"There may be clearly an enormous situation throughout the tech business as a result of we have seen this throughout a number of platforms, and [there is] not all the time nice pointers or boundaries or regulation," stated Koltai.

The criticism within the U.S. has been way more muted than in different nations. Sen. Ted Cruz, R-Tex., posted on X Wednesday that the pictures "needs to be taken down and guardrails needs to be put in place." He additionally stated that he's "inspired that X has introduced that they are taking these violations significantly and dealing to take away any illegal pictures and offending customers from their platform."

Grok has generated frequent controversy previously 12 months. Final summer time, the chatbot referred to itself as "MechaHitler" and spewed antisemitic conspiracy theories.

Apps with so-called nudifying capabilities have existed for years, largely within the shadows of the web. Winters stated officers have to do extra to police a few of X's options. "There are misrepresentations concerning the security of their merchandise. There are violations of their phrases of service," he stated.

"We have not seen actually any important motion from any U.S. companies, whether or not it is state or federal, which have the authority to implement the legislation," Winters stated.