Exploring 12 Million of the 2.3 Billion Images Used to Train Stable Diffusion's Image Generator - Waxy.org

The article explores the LAION-5B dataset, particularly its filtered subset used for training AI models like Stable Diffusion, contrasting it with restricted models such as DALL-E 2. Key findings include:

1. Content Diversity: The dataset includes unrestricted topics like fictional characters (e.g., MCU heroes, Batman) and adult content, enabling Stable Diffusion to generate imagery blocked by other models. Popular characters like Captain Marvel (4,993 images) and Black Panther (4,395) are well-represented, while Mickey Mouse barely enters the top 100 (520 images).

2. Celebrity Representation: Captions show prominence for celebrities like Elon Musk and Kim Kardashian but omit newer stars like social media influencers, despite the dataset including recent data (e.g., from 2021). This discrepancy suggests gaps in CommonCrawl’s web scraping or timing limitations.

3. NSFW Content: Only ~0.002% of images were flagged as “unsafe” with high confidence (punsafe score = 1), but lower thresholds (e.g., 0.99) include ambiguous content like nudity. Aesthetic filtering may have reduced explicit material compared to the full LAION-5B dataset, raising questions about safety definitions.

4. Technical Notes: The analysis used Datasette to host and query data, highlighting aesthetic scores and NSFW predictions. Scripts are available for public exploration, emphasizing transparency in AI training datasets.

The article underscores how dataset composition shapes model capabilities: Stable Diffusion’s flexibility with pop culture and adult themes contrasts sharply with DALL-E 2’s strict restrictions. However, mysteries remain around missing modern celebrities and the criteria defining “unsafe” content. The LAION-5B subset’s limitations—despite its scale—highlight challenges in curating balanced training data for AI systems.