The European Union has launched a formal investigation into X’s Grok chatbot over concerns that it generates non-consensual sexual images, potentially violating the Digital Services Act and exposing EU citizens to serious harm. In response to mounting global regulatory scrutiny, X has restricted Grok’s image generation features and implemented new safety measures, but the EU will determine if these actions are sufficient to meet legal obligations.
The European Union has initiated a formal investigation into X’s Grok chatbot, following similar regulatory actions by at least ten countries worldwide. The EU Commission’s concerns focus on Grok’s ability to generate non-consensual sexual images, which has exposed EU citizens to what it describes as “serious harm.” The investigation centers on the risks associated with illegal content, including manipulated sexually explicit images, child sexual abuse material, and gender-based violence.
A key issue for the EU Commission is whether X, the company behind Grok, properly assessed and mitigated the risks before deploying Grok’s functionalities within the EU. According to a New York Times analysis, Grok generated 1.88 million sexualized images of women in just nine days starting in late December. The Commission will examine if X failed to conduct and submit a required ad hoc risk assessment for Grok, as mandated by the Digital Services Act (DSA), before introducing features that significantly altered X’s risk profile.
Under the DSA, companies that do not comply with its regulations can face fines of up to 6% of their global annual turnover. In addition to the EU, countries such as the UK, Australia, Canada, Japan, India, Malaysia, Indonesia, the Philippines, Ireland, and France have all taken some form of regulatory action against Grok in the past month, highlighting the global concern over the chatbot’s misuse.
In response to mounting criticism, Elon Musk and X have implemented measures to address these issues. On January 8th, Grok limited its image generation feature to paid X accounts, although the feature remained accessible on Grok’s standalone website. Shortly after, X’s Safety team announced new rules and technology to globally prevent the Grok account on X from editing real people’s images into revealing attire, such as bikinis, restricting this capability for all users, including paid subscribers.
European Commission Executive Vice President Hanna Verkunen condemned the creation of sexual deepfakes, calling them a violent and unacceptable form of degradation. She emphasized that the investigation will determine whether X has fulfilled its legal obligations under the DSA or if it has treated the rights of European citizens, particularly women and children, as collateral damage. Earlier this month, the European Commission ordered X to preserve all internal documents and data related to Grok, describing the generated images as appalling and disgusting.