Exploring Bias in over 100 Text-to-Image
Generative Models

Text-to-image (T2I) generative models, while capable of high-fidelity image synthesis, inherently reflect the biases present in their training data (Garcia et al., 2023; Mehrabi et al., 2021; Zhang et al., 2023). The wide accessibility of training, fine-tuning and deployment resources has resulted in a plethora of T2I models being published by AI practitioners and hobbyists alike. Whereas there are many debates on the biased nature of these models, there is no concrete evidence on how the community is responding in terms of accounting for bias in T2I generative models, particularly in light of the volume of models continuing to be released. Hence, we conduct this crucial research.

The abundance of publicly available data and models democratizes AI development, but also underscores the need for responsible usage (Arrieta et al., 2020; Bakr et al., 2023; Teo et al., 2024) and comprehensive evaluation tools that characterize bias characteristics of these black box models (Bakr et al., 2023; Chinchure et al., 2024; D’Incà et al., 2024; Hu et al., 2023; Luo et al., 2024; Vice et al., 2023). The ability to develop unsafe, inappropriate or biased models presents a significant challenge and evaluating fundamental bias characteristics is a crucial step in the right direction.

Biased representations in generated images stem from factors such as class imbalances in training data, human labeling biases, and hyperparameter choices during model training and fine-tuning (Garcia et al., 2023; Mehrabi et al., 2021; Zhang et al., 2023). Theoretically, generative model biases are not confined to a single concept or direction. Analyzing a model’s overall bias provides a more comprehensive understanding of its learned representations and underlying manifold structure. For instance, when generating generic images of “animals,” a model may disproportionately favor certain species or environments. While social biases (e.g., those related to age, race, or gender) are particularly consequential in public-facing applications (Abid et al., 2021; Luccioni et al., 2023; Naik & Nushi, 2023; Seshadri et al., 2023), they are manifestations of broader model biases, observed from a specific viewpoint. Since biases extend beyond social domains, it is essential to first characterize the general bias properties of learned concepts to better understand their implications.

In this work, we perform an extensive analysis of publicly available T2I models to examine how bias characteristics have evolved over time and across different generative tasks. We construct a comprehensive evaluation framework that considers: (i) distribution bias, (ii) Jaccard hallucination, (iii) generative miss-rate, (iv) log-based bias scores, (v) model popularity, and (vi) metadata features such as the intended generative task and timestamp.

Repositories such as the HuggingFace Hub offer a vast array of fine-tuned models, including approximately 56,240 text-to-image (T2I) models¹¹1as of the time of writing this manuscript. This extensive collection enables our comprehensive evaluations. The field of conditional image generation has evolved significantly, from the widely-used Stable Diffusion architecture (Rombach et al., 2022) (spanning versions v1 to v3/XL) to the latest rectified-flow transformer (FLUX)-based models (BlackForestLabs, 2024). To capture this progression, we conduct extensive evaluations across more than 100 unique models, varying in artistic style, generative task, and release date.

To quantify bias along distribution bias ‘$B_{D}$’, Jaccard hallucination ‘$H_{J}$’ and, generative miss-rate ‘$M_{G}$’ dimensions, we utilize the open-source “Try Before You Bias” (TBYB) evaluation code (Vice et al., 2023), which aligns well with models hosted on HuggingFace. We introduce a log-based bias score that integrates these metrics into a single, interpretable value, computable in black-box settings. This approach provides a unified framework for evaluating and comparing model biases.

Our evaluations offer valuable insights into the bias characteristics of various categories of generative models, revealing a trade-off between artistic style transfer and perceived bias. We also observe that modern foundation models and photo-realism models have benefited from larger datasets, improved architectures, and careful curation efforts, leading to a positive trend in bias mitigation over time. By analyzing model popularity, we further explore whether user engagement is influenced by bias. This study represents a significant step forward in understanding how the community responds to biases T2I models, particularly in light of the rapid proliferation of diverse models.

Through this work we contribute:

1. 1.

   an extensive evaluation of bias trends in generative text-to-image models over time, uncovering key observations across three dimensions: distribution bias, hallucination and generative miss-rate.

2. 2.

   a singular, log-based bias evaluation score that advances existing methodologies. This score enables end-to-end bias assessments in black-box settings, eliminating the need for normalization relative to a corpus of evaluated models.

3. 3.

   a categorization and analysis of bias characteristics across several classes of trained and fine-tuned text-to-image models, namely: foundation, photo realism, animation, art. Additionally, we provide a quantifiable measure of model popularity, offering insights into how bias may influence user engagement and adoption.

Generative Text-to-Image Models have gained significant attention among AI practitioners and the wider, general public. These models, composed of tokenizers, text encoders, denoising networks, and schedulers, enable users to generate unique images from conditional prompts. The foundational de-noising process proposed by Sohl-Dickstein et al. (2015) inspired many of the underlying generative capabilities of modern T2I models. Subsequent advancements include denoising diffusion probabilistic models (DDPMs) (Sohl-Dickstein et al., 2015; Ho et al., 2020), denoising diffusion implicit models (DDIMs) (Song et al., 2020a), and stochastic differential equation (SDE)-based approaches (Song et al., 2020b). Rectified Flow-based de-noising paradigms have recently gained prominence, as seen in Stable Diffusion 3 (Esser et al., 2024), FLUX (BlackForestLabs, 2024) and PixArt(Chen et al., 2023; 2025).

These models often use a modified, conditional U-Net (Ronneberger & Fischer, 2015) for latent denoising. Conditional generative models integrate a network to convert user inputs into guidance vectors, steering the denoising process to match input prompts. In T2I models, Contrastive Language-Image Pre-training (CLIP) (Radford et al., 2021) and T5 encoders (Ni et al., 2021) are commonly used to map textual inputs into semantically rich embedding spaces. Larger models often combine multiple text encoders to enhance performance (Esser et al., 2024; BlackForestLabs, 2024).

By combining embedded denoising networks and text encoders, various T2I foundation models have been developed and released to the public. Notable examples include Stable Diffusion (v1 to v3.5/XL variants) (Rombach et al., 2022; Esser et al., 2024; Podell et al., 2023), DALL-E 2/3 (Betker et al., 2023; Saharia et al., 2022), and Imagen (Ramesh et al., 2022). Through cost-effective fine-tuning techniques like DreamBooth (Ruiz et al., 2023), Low-Rank Adaptation (LoRA) (Hu et al., 2021), and Textual Inversion (Gal et al., 2022), AI practitioners and hobbyists can create custom T2I models with tailored representations of learned concepts. However, these models are often shared on platforms like the HuggingFace Hub without sufficient acknowledgment of their potential biases, raising concerns about their responsible dissemination.

Bias and Ethical AI Evaluation Frameworks. Modern foundation models are trained on large, uncurated internet datasets, which often contain harmful, inaccurate, or biased representations that can manifest in generated outputs (Ferrara, 2023; Mehrabi et al., 2021). Unlike biased classification systems, bias in generative models is subtler and harder to detect due to their expansive input/output spaces and complex semantic relationships arising from massive training datasets. Without proper mitigation or quantification, these biases can lead to the proliferation of harmful stereotypes and misinformation. Compounded training and fine-tuning processes can thereby exacerbate or shift a model’s bias characteristics, raising ethical concerns, especially in front-facing applications. This underscores the critical need for bias quantification to address ethical AI considerations.

Several ethical AI evaluation frameworks have manifested as a result of these open research questions (Cho et al., 2023; Luccioni et al., 2023; Luo et al., 2024; Chinchure et al., 2024; Vice et al., 2023; Bakr et al., 2023; Hu et al., 2023; Teo et al., 2024; Gandikota et al., 2024; Huang et al., 2024; Schramowski et al., 2023; Seshadri et al., 2023; Naik & Nushi, 2023; D’Incà et al., 2024), addressing issues of fairness, bias, reliability and safety. While this work focuses primarily on biases, it is important to consider the synergy that exists across these four ethical AI dimensions. To conduct these evaluations, many works deploy auxiliary captioning or VLM/VQA models to facilitate the extraction of descriptive metrics.

The TIFA method introduced by (Hu et al., 2023) defines a comprehensive list of quantifiable T2I statistics, leveraging a VQA model to provide an extensive evaluation results on generated image and model characteristics. In a similar vein, the HRS benchmark proposed by (Bakr et al., 2023) also considers a wide range of T2I model characteristics - beyond the bias dimension, as it considers image quality and semantic alignment (scene composition). The StableBias (Luccioni et al., 2023) and DALL-Eval (Cho et al., 2023) methods have been proposed to assess reasoning skills and social biases (including gender/ethnicity) of text-to-image models, deploying captioning and VQA models for their analyses. Similarly, frameworks like FAIntbench (Luo et al., 2024), TIBET (Chinchure et al., 2024) and OpenBias (D’Incà et al., 2024) each consider he recognition of biases along several dimensions, proposing a wider definition of biases, all incorporating LLM and/or VQA models in their evaluation frameworks. FAIntbench considers four dimensions of bias i.e.: manifestation, visibility and acquired/protected attributes (Luo et al., 2024). In comparison, the TIBET framework identifies relevant, potential biases w.r.t. the input prompt (Chinchure et al., 2024). The ‘Try Before you Bias’ (TBYB) evaluation tool encompasses the evaluation methodology proposed by Vice et al. (2023), characterizing bias through: hallucination, distribution bias and generative miss-rate.

While evaluation frameworks are extensive, large-scale bias analysis of open-source, community-driven models remains limited. Existing efforts often focus on narrow subsets of models, leaving a critical need for a systematic, scalable approach. We bridge this gap with a comprehensive evaluation of over 100 models, utilizing the TBYB tool for its compatibility with the HuggingFace Hub.

In this work, we conduct comprehensive bias evaluations of 103 unique T2I models that have been released from August 2022 to December 2024. To identify general bias characteristics, we employ the general bias evaluation methodology defined in Vice et al. (2023) to generate images of 100 random objects, (3 images/prompt = 300 images per evaluated model). This allows us to infer diverse, fundamental bias characteristics of each model.

Our appraisal of the general bias characteristics of text-to-image models allows us to conduct a suite of evaluation studies to explore and formalize relationships between observed biases and model characteristics. Temporal-, categorical- and popularity-based analyses allow us to identify how bias characteristics: (i) have evolved over time, (ii) change with respect to different generative tasks or embedded de-noising schedulers and, (iii) impact how users engage with these models.

High-level Observations of General Bias Characteristics. We report a truncated list of evaluation results in Table 1, highlighting models that exhibit high, low and median bias behavior. Along with these, we also report results for highly-popular foundation models like the various stable diffusion versions. Analyzing Table 1 and Figs. 2, 3, we can observe that photo-realism and foundation models tend to generate relatively unbiased representations, which is expected given that these models are designed for general user inference tasks and improvements in generative fidelity - as is the case with photo-realism models. In comparison, at the bottom of Table 1, we can observe that many animation and art-tuned models report relatively more biased behavior, resulting from the task-oriented generative tasks. Observing the outputs of these models, we found that the tendency to focus on generating specific characters or art-styles irrespective of the prompt, resulted in high levels of hallucination and misalignment (see Figs. 2, 3).

Figure 2 presents a qualitative overview of bias manifestations in model outputs, using examples from Table 1 to contrast biased and unbiased behaviors. These results align with quantitative metrics: a higher average $M_{G}$ will generally demonstrate a greater semantic misalignment (e.g. lambdalabs/sd-pokemon-diffusers). Low $B_{D}$ models show constrained diversity or representational bias. Changes in $H_{J}$ are straightforward, reflecting disparities between input and generated objects.

The varying scales of the three metrics necessitate a logarithmic scale for comparing overall model bias. Each metric uniquely characterizes bias. Table 1 shows that low-bias models ($\downarrow\mathcal{B}{\log}$) typically report $B_{D}\geq 14.0$, indicating a fairer distribution of generated objects. In contrast, highly biased models ($\mathcal{B}{\log}\geq-1.0$) report $B_{D}\leq 7.0$, which suggests outliers or peaks in the output distribution. For $H_{J}$, T2I models inherently hallucinate due to their semantically rich latent spaces. The average $H_{J}\approx 0.55$ implies a 55% IoU between prompted and generated objects. Foundation and photo-realism models cluster near the mean, whereas highly biased models exhibit extreme values, with a maximum $H_{J}=0.9721$ meaning just 2.79% correlation between the input and output for the biased model. $M_{G}$ remains low across most models, with a mean ($M_{G}=0.0333$) near the minimum ($M_{G}=0.0000$), indicating valid outputs $\approx$97% of the time despite hallucinations. Models with high $M_{G}$ ($\geq 0.60$) exhibit misaligned behavior, which based on model design may be intentional.

Evolution of Biases over Time. The release of the seminal latent diffusion work (Rombach et al., 2022), culminating in the public availability of the popular stable diffusion architecture on August 22, 2022, marked a pivotal moment in text-to-image generative models. Its launch on the HuggingFace Hub and subsequent community engagement spurred significant advancements in foundation models and task-specific variants. Accordingly, we use August 2022 as the starting point for our time-based analyses, with the latest evaluated model released in December 2024.

Our evaluation spans 103 models over 28 months, presenting time-based bias analyses by individual metrics (Fig. 3) and model categories (Fig. 4). The timeline ($08/22\rightarrow 12/24$) is consistent across sub-figures, with models grouped by task categories to examine trends. Bias trends, such as the steep increase in art and animation models’ bias over time (Fig. 4(e)), highlight the impact of hobbyists and practitioners embedding stylistic preferences or specific characters into these models. These intentional biases are reflected in their outputs, as supported by observations in Fig. 3.

In comparison, we see that models associated with general tasks i.e., those that belong to the foundation and photo-realism model categories, have maintained consistent if not lower bias characteristics over time, as is the case with foundation models (See Fig. 3 and 4(e)). The increase in training data sizes and conscious improvements made to human-labeling and captioning in general have resulted in wider and denser manifolds with a greater diversity of concept representations. Significantly, looking at Table 1, if we compare stable diffusion v1.4/2.1/3.5 rows, we can see that hallucination and distribution bias scores improve with each significant version upgrade through time.

On the Influence of Model Type and Popularity. We conducted an evaluation of biases w.r.t. model categories and their popularity, exploiting Eq. (6) to quantify the latter. We report the results of these findings in Table 2 and observe that foundation and photo-realism models are on-average, the most popular for users. Interestingly, these models tend to have more unbiased output representations when we consider the quantitative findings. Additionally, by analyzing the $\mathcal{B}_{\log}$ standard deviation results in Table 2, we see that foundation and photo-realism model performances are typically more consistent than art/animation counterparts.

De-noising Scheduler-Dependent Bias Evaluation. Much of the conditional latent diffusion process is predicated on the deployed de-noising scheduler. While similarities exist across different families of schedulers and the task remains the same i.e. use a conditional vector to guide latent de-noising steps to generate an aligned image representation of the input prompt, the mathematical foundations of each scheduler is unique. We report the descriptive statistics of different schedulers in Table 3, highlighting eight scheduler categories. The FlowMatchEulerDiscrete scheduler is deployed in Stable diffusion 3 variants which points to its high popularity and low $\mathcal{B}_{\log}$ score. Recently, flow-based de-noising schedulers have gained increased attention in state-of-the-art T2I models like Stable Diffusion 3 and FLUX (Esser et al., 2024; BlackForestLabs, 2024).

In comparison, the EulerDiscrete scheduler (Karras et al., 2022) reports the largest bias and the highest average miss-rate. Incremental improvements in scheduler architectures and technology since the release of EulerDiscrete, along with the modern T2I models opting for modern schedulers are logical reasons as to why this scheduler reports significantly high bias scores. Similarly, the EulerAncestralDiscrete scheduler, which contributes “ancestral sampling” also boasts consistent performance with its predecessor. These seminal works have inspired improvements which as shown through the FlowMatchEulerDiscrete scheduler, have resulted in significant performance gains.

We note that while using quantifiable metrics like those reported here present a step in the right direction, any definitive correlations will require a deeper analysis into the schedulers themselves.

We have conducted an extensive evaluation of text-to-image models, utilizing the open HuggingFace Hub to facilitate our analyses of the bias characteristics of 103 unique models. To improve on existing evaluation methodologies, we combine three independent metrics i.e., (i) distribution bias, (ii) Jaccard hallucination and, (iii) generative miss-rate into a single log-scaled metric. By accounting for various generative model categories and quantifying public engagement, we have presented a comprehensive set of model evaluations. Identifying the fundamental bias characteristics of large, publicly-available text-to-image models is a critical task that must be considered in a democratized AI environment, considering the exposure of these models to wider audiences that continues to grow over time. So, to answer the question of “are models more biased now than they were 3 years ago?” really depends on the task. We see that iterative releases of Stable diffusion models for example, have resulted in marginal improvements in bias characteristics over time (from SD 1.1 to 3.5). Foundation and photo-realism models have demonstrated significant reductions in hallucination and increases in alignment, which is beneficial for improving the reliability for a wider range of audiences. As it pertains to style-transferred, art and animation models, these have demonstrated increased bias characteristics - a byproduct of intentionally-designing models to achieve specific tasks. We hope this work inspires further research in the field and a greater exposure to bias evaluation efforts.