India concluded its four-day AI Impact Summit 2026 in February 2026, aiming to mark India’s position in the global AI economy and make it the first AI Summit in the Global South. However, the country’s push towards AI poses dangers and harmful effects for women, with non-consensual sexual AI deepfakes being a major problem.
With the influx of AI-generated content, concerns about the creation, reproduction and distribution of deepfakes have increased. Non-consensual sexual deepfakes have been generated on a large scale using various AI chatbots. According to the New York Times and the Center for Countering Digital Hate DataWith Grok, the AI chatbot available on “X” (formerly Twitter), around 1.8 million sexualized images of women were generated and publicly shared by January 2026.
AI deepfakes are synthetic videos and images created using artificial intelligence. This is highly realistic, fabricated content that uses deep learning models to recreate people’s features, mimic their facial features, and adapt them to any given prompt.
To address the problem of non-consensual sexual deepfakes through AI, experts advocate a multi-tiered accountability process that works on regulations related to the company providing the AI service, individual accountability, and the legal framework.
“In the world of AI-powered deep fakes, concepts like consent and individual identity are greatly compromised,” said Sruthi Muraleedharan, a faculty member at Shiv Nadar University who specializes in gender studies, identity politics and symbolism.
According to them, AI deepfakes contribute to two forms of violations. Firstly, the dilution of the principle of the data subject’s consent regarding the content and its distribution. Secondly, correspondingly, the violation of a person’s right to privacy, dignity and reputation.
The chatbot has been in use since the end of December
In India, actress Malavika Mohanan was one of the many celebrities who received lewd comments and asks Grok to “put a bikini on her.”
Several women have expressed their concerns about the misuse of AI. In January 2026, Seema Anand, a London-based author and storyteller, filed an FIR regarding the creation and distribution of her non-consensual, AI-generated sexual content. The 63-year-old posted one video On Jan. 16, she said the incident made her feel “physically ill.”
Legal framework and its disadvantages
The prevalence of this issue prompted the Indian government to call on Elon Musk to take action in January 2026.
In the Indian legal framework, AI-related issues are addressed in the IT (Information Technology) Act, 2000 and the Digital Data Protection Act, 2023, as well as several other Indecent Representation of Women (Prohibition) Act, 1986, the Protection of Children from Sexual Offenses Act, 2012 in cases involving minors, and the Young Persons (Harmful Publications) Act, 1956.
As per the latest update, Grok image creation and editing is limited to Premium “X” subscribers. Apar Gupta, advocate and founding director of the Internet Freedom Foundation, said: “The reason chatbots continue to produce non-consensual content despite bans is a structural failure in the way these models are built. We cannot solve a systemic problem with reactive bans.” He added that Elon Musk’s comments follow a tired pattern of “whack-a-mole” moderation.
During the summit, Union Minister Ashwini Vaishnav spoke about the amendment to the Information Technology (Intermediary Guidelines and Code of Ethics for Digital Media) Rules laid down by a Three-hour time limit for social media platforms to remove AI-generated or deepfake content as soon as it is reported by the government or ordered by a court. Vaishnav also spoke about the commitment to technical traceability and redefining intermediary duty; However, the question of generation remains unanswered.
Under the Digital Data Protection Act of 2023, distribution of non-consensual sexual content is prohibited, but generation is not. Gupta further added, “The legal obsession with ‘circulation’ exonerates the developers of these tools. Accountability is a common ladder.” He explained: “We need to evolve our laws, particularly IT rules, to hold platforms liable if they fail to proactively prevent the generation of such content rather than waiting for a takedown notice.” Sruthi Muraleedharan said: “The nature of legal intervention and critical feminist legal analysis has been not to simply view them (non-consensual AI sexual deepfakes) as ‘offensive’ but to bring them into the realm of sexual abuse through the use of digital forums.”
Digital footprint and gendered public access
Premium membership at X is based on a subscription model, with a basic subscription available at INR 170 per month. As of January 2025, they had around 2 million subscribers, and Elon Musk said he hopes to increase that number by giving them access to Grok’s full range of features.
But the problem of non-consensual sexual deepfakes by AI is not new. A similar outrage erupted in October 2023 when a deepfake video of actress Rashmika Mandana went viral. She went to “X” and said: “AI is a force for progress, but its abuse to create vulgarity and target women signals a deep moral decay among certain people. Remember, the Internet is no longer a mirror of truth. It is a canvas on which anything can be invented.”
Many public figures, such as Keerthy Suresh and Girija Oak Godbole, have released similar statements raising concerns about the misuse of AI, prompting the government to take action.
Responding to queries to clarify the challenges clouding the investigation of deepfake, SP Cybercrime’s Lucknow office said it is a challenge to identify and trace the original source of deepfake generation. “Burner emails, fake accounts and VPNs make it extremely difficult to identify the real person behind the content,” said a written statement from the SP Cybercrime Office in response to this reporter’s query.
Rati Foundation, an NGO/organization that provides social legal support for women and children, and Tattle, a community of technologists, researchers and artists, advocate for responsible AI, media literacy and web scraping. published a report on November 3, 2025 about the role of AI in online harassment. It suggests a shift in the nature of agency when AI-generated content is used for online harassment. The report states: “In almost all cases, the AI-generated content relies on photos posted by the victim on their public profiles – the only ‘agency’ the victim had to initiate the attack. This means that the victims had little to no agency in the content that contributed to their harassment.”
The idea of digital footprint and gendered public access are used as tools to cyber-harass women. According to Muraleedharan, there is a recognition that even a basic sense of control over one’s digital presence can be compromised.
The question of problem solving
There remains a debate about how best to address the problem of AI deepfakes of non-consensual sexual content. Regardless, countries around the world have taken measures to curb the spread of the AI deepfake crisis. South Korea had criminalized the consumption of deepfake pornography, a step against digital sex crimes, and imposed stricter penalties for production, distribution and viewing. Denmark has prepared changes to national copyright rules to give people more control over their voices and images in AI-generated deepfakes.
The issue of deepfakes and its gendered impact deepens the digital divide, as women’s presence in physical public spaces is threatened by the use of gender-based violence, such as the misuse of AI to create deepfakes.
Sruthi Muraleedharan, a gender studies expert, explains: “Technology is not neutral to existing social hierarchies. Therefore, patriarchal and heteronormative assumptions, along with intersectional dimensions of majoritarian politics, can definitely influence the way in which marginalized groups are targeted, leading to a more comprehensive understanding of gender-based violence.” She gives the example from Sulli offersa GitHub-hosted website that had taken publicly available images of Indian Muslim women and created profiles describing the women as the “deal of the day.”
Gupta says: “If a society is deeply patriarchal, its users will naturally use new tools such as AI to reinforce this hierarchy through ‘image-based sexual abuse’.” If we don’t build human rights and gender sensitivity into the code from day one, technology will inherently favor the powerful and target the marginalized.”
Ishita Yadav is a freelance journalist and photographer based in Delhi. She is currently pursuing her Masters in Convergent Journalism from AJK MCRC, Jamia Millia Islamia. Her work focuses on human rights and gender.
