Looking for a Handsome Carpenter!
Debiasing GPT-3 Job Advertisements

Collecting and generating job ads for all possible jobs is prohibitively timely and costly. Hence, we restrict our experiments to a sample of 15 jobs selected via three criteria: (1) prevalence, jobs with a sufficiently large labour force in the UK ($N\geq 40{,}000$), (2) relevance, jobs which have a sufficiently large number of real-world job ads on a popular online forum ($N\geq 1{,}000)$ and (3) bias, jobs which represent the most male-biased, female-biased and neutral parts of GPT-3’s prior distribution in how frequently certain jobs are associated with a given gender. To apply these criteria, we first filter jobs in the UK economy by prevalence and relevance ONS (2018). Then to estimate GPT-3’s priors of occupational bias, we generate $1{,}000$ completions for the prompt “What gender is the {job}? The {job} is a [token]”, where a completion could be: “What gender is the plumber? The plumber is a [woman]”. Using the ratio of male-to-female tokens in these $1{,}000$ completions, we select the top five male-biased, female-biased and neutral jobs (see Appendix˜C for further detail on job selection and pre-processing).²²2Male-biased jobs: plumber, engineer, carpenter, electrician, software developer; Female-biased jobs: nurse, housekeeper, occupational therapist, secretary, social worker; Neutral jobs: artist, tester, administrator, project manager, writer.

To generate a dataset of human-written ads, we collect live job ads matching the 15 selected job titles from a popular UK job site in January 2022. After de-duplication, our sample includes 85 ads per job.

We use the OpenAI Davinci GPT-3 model which has been adapted for natural language requests. We use default parameters values and 500 maximum tokens per completion (see Appendix˜B for hyperparameter details). Keeping default parameters better reflects when non-technical users apply large-scale generative models “out-of-the-box” Kirk et al. (2021). In our experiments, we assess zero-shot, prompt-engineered and fine-tuned generation. We use templated prompts e.g., “Write a job ad for a {job}” which we populate with 1 of the 15 selected job titles. We then allow the model to generate the ad without specifying what details should be included in the output to examine its native behaviour.

We define two baseline datasets of job ads. The first dataset contains all the real-world job ads that we collected from a popular job advertising website ($N=1{,}275$).⁴⁴4We assume that ads in this dataset are human-authored. The second dataset contains ads generated by GPT-3 using zero-shot generation with nine syntactically-varied but neutral prompts such as “Write a job ad for a {job}” (see Appendix˜D for prompts). For each prompt, we generate 40 completions per job title ($N=5{,}400$).

In contrast to the neutral prompts in the zero-shot generative baseline condition, we define a set of prompts that explicitly attempt to force the generation of an unbiased ad. These include references to diversity in hiring such as “We are focused on hiring minority groups, write a job ad for a {job}” or explicit references to bias, e.g., “Write a job ad for a {job} which appeals equally to men and women” or “Compose an unbiased job ad for a {job}” (see Appendix˜D). For each prompt, we generate 40 completions per job title via zero-shot generation ($N=5{,}400$).

We construct three fine-tuning datasets from the real-world job ads: (1) the all jobs dataset has all real-world job ads for the 15 selected jobs ($N=1{,}163$). (2) the low bias dataset includes the 10% least biased real ads for each job title ($N=127$), as measured by our bias metric. (3) the high bias dataset conversely uses the 10% most biased real ads ($N=125$). We then fine-tune a model on each dataset and generate 40 completions per job title ($N=600$ per model).

We develop our bias measure using dimensionality-reduction over six existing lists of gender-laden words: (1, 2) Gender-Coded Word Prevalence: Gaucher et al. (2011) define masculine-and-feminine-themed words from an experiment on job ads that discouraged female applicants. (3) Superlative Prevalence: Schmader et al. (2007) assess the relative frequency of positive and negative superlatives used to describe male versus female job candidates in recommendation letters. We use an established set of superlative words Veale (2016). (4) Gender-Laden Scoring: Sap et al. (2017) analyse 32 properties related to a set of norms to score $2{,}311$ words based on their “gender-ladenness”. (5) Connotation Frames: Sap et al. (2017) define linguistic markers of power and agency associated with female versus male characters in modern films. (6) NRC VAD Lexicon: Mohammad (2018) presents a lexicon of words coded by valence, arousal, and dominance whose interpretation may interact with gender.⁵⁵5For a fair comparison, we implement unweighted word count measures for each list. See Appendix E for further detail and mathematical definitions.

We employ principal component analysis (PCA) on the six bias measures on real-world job ads to collapse them into interpretable components. We then replicate the PCA on synthetic job ads (zero-shot) and project all data points onto the first two principal components of real job ads and vice versa.

Prompt-engineering does not effectively lower bias nor increase realism in generated ads (see Tab.˜1 and Fig.˜1), apart from a small but significant reduction in bias for male-dominated jobs (Fig.˜3). In 97% of sampled cases, our annotators correctly identify the ads as synthetic. The bias averaged across all generated ads in this condition is marginally worse than the baseline zero-shot condition (GPT-3) but there is considerable variation between prompts, with the least biased generations coming from “We are focused on hiring minority groups, write a job ad for {job}”.⁸⁸8See Appendix D for full results per prompt.

We find that fine-tuning on real ads increases the length of generated ads, with an average of 260 words compared to 82 words in the zero-shot baseline. The outputs are also more realistic, containing better detail, such as salary information, specific responsibilities and required experience. Additionally, formatting is improved, with outputs containing separate paragraphs, lists and bullet points. The annotator majority vote mistakenly labels the synthetic ads from a fine-tuned GPT-3 model as real in 90% of sampled cases for the high bias condition and all cases for the low bias condition, suggesting these ads were practically indistinguishable from real-world ads. Specifically, fine-tuning on low bias ads results in a significant bias reduction across all job types (Fig.˜3). This reduction in bias even outperforms the average bias of real job ads, yet retains realism (Fig.˜1).

Our measures of bias and realism are relatively simplistic. On bias, using lists of gender words is a blunt tool and may in fact reinforce linguistic gender stereotypes. Furthermore, we use our composite measure of bias for evaluation and also for filtering ads for fine-tuning. Thus, future work is needed to derive more complex and diverse measurements of bias in job ads and to cross-validate how debiasing approaches affect independent bias measures. We restrict our bias measures to the axis of binary gender, because when estimating GPT-3’s priors using the prompt “What gender is the {job}?”, the model never returned a non-binary gender, a problematic bias in itself. Future audit of language models is urgently needed beyond the axes of binary gender bias.

On realism, while we proxied realism with a classifier and validated these results in a small-scale experiment with human annotators, more work is needed to assess reactions to machine-written ads “in the wild”. Furthermore, while fine-tuning and prompt-engineering increased realism in the aggregate, some job ads were still nonsensical or simply parroted the prompt text, e.g., “The job ad should not have any biases in it.”. We briefly assess some outputs qualitatively in Appendix˜F but make our bias measure generation process publicly available to encourage more human-directed assessments of bias and realism.⁹⁹9https://github.com/oxai/gpt3-jobadvert-bias It remains to be seen whether realism (as measured by similarity to human-authored ads) is a necessary characteristic for success (as measured by the number of applications). Prior research identifies fluency and a clear presentation of relevant skills and experience as relevant to the creation of a “good” job ad Liu et al. (2020), but it is not clear whether an ad must appear human-written to achieve this. Our assumption for this project is that human-written job ads follow styles, conventions and a level of detail that effectively encourage prospective employees to apply, but further research is required to understand whether ads clearly identified as machine-written can be equally or more effective in this regard.

Our chosen domain of generative job ads is unlikely to be a widely used application of GPT-3 in the near future. While the computational cost of generating a single job ad is significantly lower than a human writing an ad, the human cost of reviewing generated ads and adapting them to company-specific requirements likely diminishes the cost savings. A near-term application of the technology could be to use GPT-3 to re-write a human-written job ad, demonstrated by Dover’s “GPT-3 Job Description Rewriter”, with an additional focus on debiasing human-authored text.¹⁰¹⁰10https://producthunt.com/posts/gpt-3-job-description-rewriter-by-dover Our findings demonstrate that generative models must be carefully applied when creating texts for a downstream, real-world setting in hiring and recruitment, especially when used zero-shot with no debiasing techniques. This is relevant to other applications but the specifics of other domains can be explored further in future work.

While our goal was to generate gender-neutral job ads, it remains possible that neutrality may still dissuade a particular group from applying Gaucher et al. (2011). Our work cannot comment experimentally on whether less-biased ads at the text-level result in a greater diversity of applicants. Further social science and experimental research is thus necessary to understand the effects that language in job ads has on applications from various protected groups.

While we have established methods for measuring and mitigating binary gender bias, we have not achieved the same for non-binary genders nor for any other protected characteristics defined in the Equality Act 2010 Fell and Dyban (2017). Practitioners tackling more varied presentations of identity-directed bias may be less able to find pre-existing lists of biased words to define bias measurements.