Lately, everyone on Instagram and other social media platforms is into the 80s trend, where users post AI-generated pictures of themselves with an 80s look and feel. However, beyond the environmental impact of generative AI, this seemingly harmless trend has worrying implications, especially when it comes to feeding children’s images with generative AI.
Meta’s announcement only addresses CSAM after it is uploaded to their social media platforms. This brings us to the question: What happens before Meta identifies and reports CSAM content on its platforms?
In recent months, Meta has come under scrutiny in India for its presence Child sexual exploitation and abuse material (CSEAM/CSAM) on its platforms. A BBC Eye investigation noted that advertising was reportedly used to promote such material and direct users to Telegram channels where CSAM was allegedly sold. Sometimes meta AI suggestions for videos featuring children redirected users to adult websites.
On September 15, 2026, Meta agreed with the reporting Directly refer matters related to child safety to the Indian Cybercrime Portal managed by the Indian Cybercrime Coordination Center (I4C). The company said the new agreement will strengthen its coordination with Indian authorities in combating online child exploitation. However, Meta’s announcement only addresses CSAM after it is uploaded to their social media platforms. This brings us to the question: What happens before Meta identifies and reports CSAM content on its platforms?
The scope of CSAM online
In 2025 the Internet Watch Foundation (IWF) evaluated 8,029 images and videos showing realistic sexual abuse of children. Of these, 3,443 videos were AI-generated, in contrast to 2024 when the IWF had identified only 13 such AI-generated videos. In 2025, there was a 264-fold increase in AI-generated CSAM. And this data represents only the material that the IMF found and assessed; The scope of such AI-generated CSAM across the world could be much larger.
The IWF found that 65 percent of the AI-generated videos it assessed were classified as Category A – depicting penetrative sexual assault, sadism or sodomy – the most serious category in the UK’s CSAM classification system. And 97 percent of the AI-generated images evaluated showed girls.
In 2025, there was a 264-fold increase in AI-generated CSAM. And this data represents only the material that the IMF found and assessed; The scope of such AI-generated CSAM across the world could be much larger.
A Report 2025 from the National Center for Missing & Exploited Children (NCMEC), a US non-profit organization, sheds further light on the problem. In 2025, the organization received more than 400,000 CyberTipline reports with a genAI nexus. More than 182,000 reports involved offenders who had AI-generated CSAM or were generating or attempting to generate CSAM using AI.
Since NCMEC began tracking AI-generated video content in 2023, it said more than 275 victims of AI-generated CSAM have been identified and more than 158,000 images and videos have been categorized as AI-generated CSAM. While the IWF and NCMEC data are not directly comparable because their samples vary and the studies are conducted in different countries, when considered together, the data points to the growing child safety concerns surrounding generative AI.
Social media is the last stop
Videos often circulate on Instagram showing very young girls in sexualized situations with adults. The videos appear to be AI-generated, with some even bearing Meta’s “AI Content” label. And these videos are publicly available to a global audience.
Given recent developments, it appears that some of these AI-generated CSAM videos have been removed from the platform. Since these videos appeared on Instagram, Meta is responsible for detecting, removing and reporting them. But platform responsibility doesn’t just start and end with Meta’s platforms. Somewhere, someone had to generate or modify the material on another platform before it could be uploaded to a meta platform.
This chain can start much earlier. For example, a photo of a child that was uploaded to an AI system without any malicious intent on the part of the child’s parents could be used for this purpose. India’s Digital Personal Data Protection Framework (DPDP) has separate provisions in Section 9 that focus on the safety of children online. However, these provisions only apply if the child actually exists. Once a photo containing facial biometric data enters the AI ecosystem, it can serve as a data set for fine-tuning the AI model’s results. The concern should therefore include both scenarios: what happens to the original biometric data and what can the technology generate from the child’s image?
One AI model can generate a completely synthetic image of a child, and another model can modify it and turn it into a video. When the final file is uploaded to a social media platform, only the final output is visible and the systems used to create it may not be traceable.
The BBC Eye Investigation revealed how an account that had not initially sought such material could be led to increasingly explicit content, including CSAM, through Instagram’s recommendation system. Meta said it had removed the accounts and content identified by the BBC, but that did not end the issue. Things get even bleaker when the content being distributed is AI-generated.
One AI model can generate a completely synthetic image of a child, and another model can modify it and turn it into a video. When the final file is uploaded to a social media platform, only the final output is visible and the systems used to create it may not be traceable.
Back to the squaree
Praneeth Panyam, a Hyderabad-based software developer, explains how images and videos are generated by AI systems. He tells FII that a user’s prompt is converted into a mathematical representation that combines language with visual concepts. The image itself does not appear all at once. “The image begins at a purely static point,” he says.
The same basic process can be fed into another system: an image can first be generated, modified using another model, and then converted into a video by another AI system. At each stage, the output of one system can become the input for the next. The main security concern with AI-generated images and videos is the intent behind their creation, which is not always clear from the final output. As Panyam puts it, “The user’s intent is not calculated in the image pixels.”
Once the material is available, remedies will be available. However, increasing platform accountability will ensure that AI-generated CSAM is addressed at the source, where such content is generated at scale.
An image can therefore move from one AI platform to another and end up on social media as a video, including as an AI-generated CSAM. The technical difficulty lies in reconstructing what happened before the final edition appeared online. However, this does not necessarily mean that it is outside the law.
Afnan Husain, a lawyer at the Bombay High Court, told FII: “Indian law makes no distinction between an image captured by a camera and an image created using AI or computers when it comes to sexually explicit images depicting a child or a child-like figure.” In such cases, creating, storing or sharing that image or video may result in serious criminal prosecution under our laws including Section 67B of the IT Act and Section 15 of the POCSO Act.”
Once the material is available, remedies will be available. However, increasing platform accountability will ensure that AI-generated CSAM is addressed at the source, where such content is generated at scale.
While generative AI tools often have some built-in security measures, simply adjusting the language in the prompt can bypass the platform’s keyword-based security measures around sexually explicit content. Speaking to FII, Siddharth Rao, a Hyderabad-based political researcher and journalist, says manipulating AI systems is becoming easier day by day. Rao offers FII a possible solution: “AI models should include an overarching instruction to either not produce AI-generated videos and images that depict a child or a tiny human that even remotely resembles a child.” Or, if the output resembles a child, it should include information about where and when such images were created, as well as a clear watermark to identify the exact AI model used in the creation of the video.”
While the final platform should ultimately be held responsible for preventing its further distribution, responsibility should begin at the place of creation: the AI model in which the content was first generated.
India’s IT Rules 2026 are already moving in this direction by mandating safeguards against prohibited synthetically generated information and, where technically feasible, requiring labeling and persistent metadata or other provenance mechanisms to identify the computer resource used to create or modify such content.
CSAM images and videos go through multiple systems before reaching social media platforms like Instagram. While the final platform should ultimately be held responsible for preventing its further distribution, responsibility should begin at the place of creation: the AI model in which the content was first generated.
Editor’s Note: Some quotes in this article have been edited for clarity and length.
Madhuri Kankipati is an independent writer and researcher based in Khammam, Telangana. She translates between Telugu and English, with a focus on women writers. Her work focuses on politics, gender, literature, digital culture, AI and the impact of AI in Indian publishing. Her writing has been published by The Chakkar, Muse India and Borderless Journal. She also writes about books and AI ethics.
