Indonesia becomes the first country to suspend Elon Musk's Grok for generating 'non-consensual sexual deepfakes'

• Indonesia suspended access to Grok over "non-consensual sexual deepfakes."
• The minister of communications said she suspended Grok to protect women and children.
• Following criticism, Grok's AI image generator is now limited to paying subscribers.

Indonesia, home to the world's fourth-largest population, has barred access to Grok after the AI chatbot generated sexualized images of real women and children.

Indonesia's Ministry of Communications said it is temporarily suspending access to Grok to "protect women, children, and the entire community from the risk of fake pornographic content generated using artificial intelligence technology."

"The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," Meutya Hafid, the minister of communications, said in a statement.

Elon Musk's xAI makes Grok and also owns X, which integrates the bot into its platform. Indonesia is home to the world's third-largest number of X users.

World governments have taken aim at Grok after some used it to digitally undress real people in photos, and then post them to X.

French authorities told Politico this month that they will investigate sexually explicit deepfakes generated by Grok. And the Indian government said in a letter to X's chief compliance representative in the country that Grok is being "misused by users" to generate "images or videos of women in a derogatory or vulgar manner in order to indecently denigrate them."

The ministry asked X to conduct a "comprehensive technical, procedural, and governance-level review" and remove content violating the country's laws.

Ofcom, the UK's communications regulator, told Business Insider on Monday that it has "made urgent contact with X and xAI to understand what steps they have taken to comply with their legal duties to protect users in the UK."

Several US senators, meanwhile, have asked Apple and Google to remove X and Grok from their app stores over the images.

"Turning a blind eye to X's egregious behavior would make a mockery of your moderation practices," the senators write in a press release. "Indeed, not taking action would undermine your claims in public and in court that your app stores offer a safer user experience than letting users download apps directly to their phones."

Musk responded to the criticism this month on X, telling one user, "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content."

In a separate post, X's safety account wrote the company takes action "against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary."

Following the backlash, Grok's AI image generator is now limited to paying subscribers.