AI Tools Are Quietly Using Real Images of Children for Training

The recent Human Rights Watch report has brought to light a serious issue involving the unauthorized scraping and use of images and personal details of over 170 Brazilian children in an open-source dataset for AI training. This dataset, known as LAION-5B, included images posted online as far back as the mid-1990s and as recent as 2023. The collection of these images occurred without the knowledge or consent of the individuals involved, leading to significant privacy concerns.

The use of these images to train AI technologies means that realistic imagery of children can be generated and potentially misused. Hye Jung Han, a researcher at Human Rights Watch, emphasized the initial privacy violation when the photos were scraped and included in these datasets. Han warned that any child with photos or videos online could be at risk of manipulation by malicious actors using these AI tools. The troubling aspect is that many of these images were not easily accessible through standard reverse image searches, suggesting a deliberate and invasive effort to collect them.

LAION, the organization behind the dataset, has responded to these concerns by removing the problematic LAION-5B dataset following a report by Stanford University that found links to illegal content within it. LAION is working with various organizations, including the Internet Watch Foundation, the Canadian Centre for Child Protection, Stanford, and Human Rights Watch, to eliminate all known references to illegal content. However, Nate Tyler, a spokesperson for LAION, acknowledged that removing links from the dataset does not eradicate the content from the internet, highlighting a broader and more challenging issue.

The unauthorized scraping of data also violates the terms of service of platforms like YouTube, which strictly prohibit such activities. YouTube spokesperson Jack Maon reiterated that unauthorized scraping is a breach of their policies, and they are actively working to combat this abuse. This situation underscores the complexity and difficulty of enforcing terms of service in the digital age, particularly when it involves large volumes of data scraped for AI training purposes.

Stanford University's December report on AI training data revealed that the LAION-5B dataset contained child sexual abuse material (CSAM). This discovery adds another layer of urgency to addressing the issue, as explicit deepfakes have been increasingly used for bullying in schools, especially targeting girls. The potential for AI to generate CSAM and expose sensitive personal information, such as locations and medical data, is a significant concern. Han's findings also included a case where a US-based artist found her image in the LAION dataset, originating from her private medical records.

The misuse of children’s images in datasets like LAION-5B highlights the inadequacy of current data protection measures and the need for stronger regulations. Although LAION confirmed the existence of the identified images and agreed to remove them, Han fears that her team's findings represent only a small fraction of the problematic content. She believes that similar images from around the world may also be included in the dataset, exacerbating the privacy violations and potential risks.

Efforts to mitigate these risks must go beyond removing specific datasets. Last year, a German ad campaign utilized AI-generated deepfakes to caution parents against posting children’s photos online, warning of potential misuse. However, this campaign does not address the issue of existing images that are already available online. The broader problem requires systemic changes to protect individuals’ privacy and prevent the exploitation of personal data.

Hye Jung Han argues that the responsibility to protect children and their parents from such abuses should fall on governments and regulators. The Brazilian legislature is considering laws to regulate the creation of deepfakes, and in the US, Representative Alexandria Ocasio-Cortez has proposed the DEFIANCE Act. This legislation would allow individuals to sue if they can prove that a deepfake of their likeness was made without consent. Han stresses that children and their parents should not bear the burden of protecting themselves against sophisticated technologies, highlighting the need for legal and regulatory frameworks to address these challenges.

The revelations from the Human Rights Watch report emphasize the ethical and privacy concerns surrounding the use of personal data in AI training. The misuse of children’s images in the LAION-5B dataset demonstrates the potential harms of unchecked data scraping and the urgent need for comprehensive regulations to protect vulnerable populations. As AI technologies continue to evolve, it is crucial for policymakers, technology companies, and civil society organizations to collaborate on creating robust frameworks that prioritize the safety and privacy of individuals.

The ethical use of AI training data is paramount to ensuring that technological advancements do not come at the cost of personal privacy and safety. The Human Rights Watch report serves as a stark reminder of the potential risks and the necessity for vigilance and regulation in the digital age. Governments and regulators must take decisive action to safeguard the rights and privacy of individuals, particularly children, in the face of rapidly advancing AI technologies.