Can Prompt Modifiers Control Bias?
A Comparative Analysis of Text-to-Image Generative Models

Within the dynamic realm of artificial intelligence, the advent of text-to-image generation models [16, 32, 2] marks a significant leap forward. Leveraging deep learning, these models convert text descriptions into detailed images, captivating users and pioneering new avenues in artistic creation, design, and communication [5, 11]. These models, powered by vast datasets [24] and advanced algorithms [15, 26, 27], promise a new era of creativity and efficiency. However, with great power comes great responsibility, particularly in ensuring that these innovations do not perpetuate or amplify societal biases [19].

Unfortunately, initial observations highlight a significant variance in the depiction of culturally and geographically nuanced concepts within existing text-to-image models. Consider, for instance the archetype of the “monk,” traditionally associated with Asian cultures and male roles: A preliminary analysis of image outputs for a generic “monk” prompt across various models unveils a marked inclination towards representing monks as Asian males, as detailed in Tab. 1. This tendency, while possibly reflective of historical accuracies, prompts scrutiny over the data and algorithms that inform these models, particularly in how they navigate cultural and gender biases. Interestingly, the Firefly (FF) model showcases a notably more balanced gender and racial representation, indicating a distinct internal approach to bias attenuation.

The complexity of this issue deepens when examining the models’ responses to compound prompts aimed at eliciting non-traditional representations, such as a “Monk who is black,” shown in Tab. 2. Notably, despite explicit instructions, Stable Diffusion (SD) and Dall$\cdot$E 3 (DallE/DE) continued to predominantly produce imagery tied to Asian cultural markers, highlighting a proclivity to default to historical and cultural stereotypes over direct prompt cues.

The divergent responses to these prompts, particularly Firefly’s shift towards equitable representation, spotlight the nuanced challenge of bias within AI systems. Such variance raises pivotal questions about the objective of these models in reflecting the diversity of human experience. Should they aim to accurately mirror historical and sociodemographic realities, or aspire towards an idealized inclusivity that may diverge from factual representation? While Firefly’s inclusive approach is laudable, it ignites debate on the validity of achieving balance at the potential expense of demographic authenticity.

Motivated by these observations, this study aims to dissect and understand the bias embedded within these AI technologies. It undertakes a thorough analysis of bias across three forefront text-to-image models: Stable Diffusion [21], OpenAI’s DALL$\cdot$E 3 [4], and Adobe Firefly [1]. Our structured examination employs singular prompts to compare and contrast biases and statistical variations within these models. We navigate this research through three critical phases. Initially, we perform an analysis of each model using standardized prompts to identify biases related to gender, race, geography, and religion/culture, providing a baseline for bias assessment. Subsequently, we investigate the use of “modifiers” in prompts, integrating various bias aspects into a singular prompt to see if biases can be mitigated. This exploration into “Base Prompt + Modifier” configurations reveals the potential of prompt engineering to create more equitable AI applications. Lastly, we assess the impact of prompt sequencing—whether placing the modifier before or after the base prompt affects image generation—suggesting that even minor adjustments in prompt structure can significantly alter outcomes, thereby illustrating the complex dynamics of bias within text-to-image models.

By examining gender, race, geography, and religion/culture biases with the aid of base prompts and modifiers, this study aims to deepen the understanding of bias in AI. Through comparative analysis, we illuminate each model’s specific biases and underscore the role of prompt engineering in bias reduction. Specifically, the paper highlights:

• •

  Prompt Modifiers as a Tool for Bias Adjustment: We introduce the use of prompt modifiers as a means of adjusting bias within image generation models. Importantly, our experiments with this form of prompt engineering do not yield uniform results, highlighting the fundamental nature of this challenge and the need for more complex strategies.

• •

  Demonstration of Control-resistant Biases: While prompt engineering may seem to be a direct and nearly trivial fix for overcoming model biases, we demonstrate both several examples of inherent biases that are not overcome by adding prompt modifiers and several more where the behavior with respect to modifer addition is fragile (i.e. sensitive to ordering).

• •

  Impact of Prompt Sequencing on Bias Control: By analyzing how the sequence of base prompts and modifiers influences image generation, we highlight the importance of prompt structure in bias control within AI-driven processes.

• •

  Introduction of a Taxonomy and Validation Method: We introduce a taxonomy to gauge models’ sensitivity to prompt engineering and validate this approach through a quantitative metric of distributional shift, based on modifier application. Providing this structure enhances our understanding of bias control mechanisms in AI models and yields a framework for future characterizations and cross-comparisons in measuring both bias and attempts at its adjustment in AI models.

• •

  Broad Comparative Analysis Across Multiple Models and Bias Categories: Our investigation expands on the scope of prior work by providing a comparative analysis of four bias categories over three leading text-to-image generation models: Stable Diffusion, DALL$\cdot$E 3, and Firefly, and their entanglement with LLMs via prompt processing.

Text-to-image generation models mark a significant leap in creative capabilities at the confluence of artificial intelligence and the arts, with Stable Diffusion, DALL$\cdot$E 3, and Firefly at the helm. These advancements have not only revolutionized the way textual inputs are visualized but also prompted a critical examination of the biases inherent within these technologies. A growing body of scholarly work has begun to explore the various dimensions of bias present in these models, providing a foundation for the comparative analysis we undertake in this study. The summary of the bias categories and the corresponding models examined in the related literature is presented Tab. 3

Tab. 4 provides an illuminating snapshot of the complexities involved in mitigating biases across various categories within text-to-image generation models. Turning to the ’Season in January’ category, a notable distinction arises in the geographical representation of seasons. Stable Diffusion and Firefly revealed a Northern Hemisphere winter bias, which inadvertently reflects the demographic and climatic realities of more than 85% of the global population residing in the Northern Hemisphere. Conversely, DallE showcased a more balanced depiction of both summer and winter scenes, thus acknowledging the seasonal contrasts between hemispheres.

This balance raises an intriguing question regarding the role of AI in mirroring versus moderating real-world disparities. While DallE’s balanced output may seem fair and inclusive at face value, it may also inadvertently gloss over the demographic predominance of the Northern Hemisphere, suggesting that a truly balanced AI model must navigate the fine line between representational fairness and demographic fidelity. These contrasting approaches underscore the complexity of bias in AI, where the pursuit of balance must be carefully weighed against the representation of statistical realities, such as the population distribution across hemispheres, which directly impacts the prevalence of seasonal experiences worldwide. These findings compel a deeper consideration of how text-to-image models encapsulate and convey societal norms and raise fundamental questions about the benchmarks for unbiased AI representations.

In examining the presence of biases across the specified categories, it becomes evident that not all bias types manifest uniformly or are even applicable to each category. This is reflective of the nuanced reality that certain societal constructs and roles carry specific historical and cultural biases [6], while others may be more universally recognized and less prone to subjective bias [20]. To anchor our investigation in empirical rigor, we have leveraged prior scholarly work and widely acknowledged consensuses to establish our base prompts and categories that have historically exhibited strong biases [3]. These informed baselines serve as a critical reference point for assessing whether the models merely replicate known biases [17] or whether they have the capacity to transcend these limitations [18], potentially yielding a more diverse range of outputs as required by the user.

For instance, the nurse category across Stable Diffusion, DallE, and Firefly did not display any overt racial biases, as the models generated diverse racial representations in the absence of a clear skew towards any particular group, but did exhibit gender skew. The lack of overt racial biases could be seen as a positive step toward unbiased AI, reflecting an equitable cross-section of racial identities in the nursing profession. Cultural and geographical factors were similarly nondescript, indicating that these models may not strongly encode or perpetuate biases along these dimensions within the scope of the tested prompts. However, the gender bias observed, with a skew towards female representations, resonates with societal associations of the nursing profession. Firefly’s more balanced gender output, intimates the potential for mitigating such biases, although it also prompts further scrutiny into the methods and training data employed for such counter-bias modeling efforts: As demonstrated in Section 5, the opacity of counter-bias modeling can impact the ability to understand and manipulate distributional outcomes via prompt engineering.

In our experimental setup, we engaged three distinct models—Stable Diffusion, DallE, and Firefly—to create images from a set of base prompts, aiming to uncover any inherent biases. With Stable Diffusion, we generated a suite of 50 unique images for each prompt to ensure a robust sample size. In the case of Firefly, we leveraged its functionality to differentiate between real and stylized characters, opting for the generation of real-person images. For each prompt, Firefly produced images of four distinct individuals, culminating in a total of 52 images per prompt. Meanwhile, our use of DallE was facilitated through the ChatGPT4 interface, which serves as a gateway to the DallE image generation backend. Due to operational constraints for ChatGPT, we were limited to crafting 40 prompts every three hours. To circumvent this and maximize output, we utilized compound prompts requesting the creation of images in a grid format, specifically instructing the model to ”generate A with 3 rows and 3 columns” where A is a prompt of interest. While there was no strict limit on the number of images generated, we aimed for upwards of 30 images per prompt to ensure a statistically significant sample that could provide a meaningful analysis of distribution trends across the models.

In our study, we employed 16 distinct base prompts, intentionally chosen to span the breadth of biases commonly associated with gender, geography, religion/culture, and race. These categories, as detailed in Tab. 5 and discussed in Sec. 3 , do not encompass the entire scope of possible biases, yet they offer a representative cross-section of biases that are visually identifiable within the images produced by the models. A comprehensive list of the base prompts utilized for this study is available in the supplemental materials.

When these prompts were deployed across three distinct models—Stable Diffusion, DallE, and Firefly—we were able to detect certain biases that these base prompts seemed to induce in the model outputs. Delving deeper, our analysis involved the introduction of modifiers to these base prompts, which effectively altered the bias distribution observed initially. This modification approach not only provides a straightforward means of disrupting the detected biases but also opens up new avenues for understanding the dynamics of bias within AI-generated imagery. Moreover, we explored how the sequencing of these prompts and modifiers (either ‘Base + Modifier’ or ‘Modifier + Base’) might impact the models’ image generation, probing the influence of prompt structure on the visual representation of societal categories.

In this section, we delve into the nuanced aspects of our analysis, segregating the discussion into qualitative and quantitative evaluations for the three models under consideration. Section 5.1 is dedicated to a qualitative review, offering in-depth insights into the interpretative outcomes generated by each model. Following this, Section 5.2 presents a quantitative analysis, systematically comparing the effects of the prompt configurations Modifier + Base’ and Base + Modifier’ on the model outputs. This structured approach enables a comprehensive exploration of the models’ performance across different dimensions of analysis.

Bias is an inherent characteristic of models trained on real-world data, which inevitably contain biases. Our approach—utilizing modifiers as a form of prompt engineering to influence bias distribution—represents an unexplored method of bias adjustment within the field. This preliminary strategy did not yield consistently effective results, indicating that simplistic applications of modifiers are insufficient. This finding points to the necessity for a more nuanced approach, potentially involving a larger-scale, subjective analysis to tailor bias distribution when the intent is to generate data points from the extremes of a distribution.

Reflecting on the challenges faced by the Gemini case [8, 10], we recognize that any attempts to correct biases in models are fraught with complexity. Gemini’s failures—oversights in presenting a diverse range of individuals and an overly cautious response to benign prompts—exemplify the difficulties in achieving balance. [14] The question of whether to align model outputs with geographical or demographic realities remains open. More concerning, however, is the presence of unacknowledged biases within models, as unrecognized biases that are not addressed pose a significant issue. This underscores the imperative for thorough, in-depth studies into bias correction, an area ripe for future research to advance the field.

This study explores biases in text-to-image models, revealing how societal biases are embedded and can be mitigated within these AI systems. Our characterization experiments showed that while Stable Diffusion and DallE often reproduce biases from their training data, Firefly shows the potential for less biased outputs, pointing to differences in data handling and model design. Meanwhile, our study of prompt modification highlights the uneven success of using modifiers for bias adjustment and the importance of prompt structure in shaping outputs, demonstrating that direct approaches to prompt engineering are not sufficient to reliably overcome intrinsic model biases in all cases.

The observed complexity in model responses to even these relatively straightforward adjustments in stimuli underscores the ethical imperative for AI developers to balance innovation with sensitivity, advocating for transparency and inclusivity in AI development to prevent the reinforcement of societal inequalities. This work introduces a taxonomy for categorizing model robustness to prompt modification and a quantitative, expectation-based metric for conformity with supplied prompt modifies that can be utilized by future work for similar cross-comparative studies. Both the limitations and opportunities highlighted by this research point to the necessity for ongoing efforts to understand and correct biases in AI, suggesting future exploration into more effective bias-controlling strategies and diverse AI development approaches.