Study: Bias in AI-generated reference letters

Research summary showing why we must be proactive in bias mitigation

Hi friends,

Today I’m highlighting findings from a great study on bias in AI-generated reference letters. It shows that the AI-written reference letters have a massive gender bias.

For me, the takeaway is that we cannot rely on generative AI tools to be aligned with our values. We must be proactive if we want to prevent bias when using generative AI.

I recommend reading it for yourself!

“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters by Wan et. al (2023)

What counts as bias?

The study looked into three types of biases, based on social science literature:

1. Biases in Lexical Content - this is bias in word choice, particularly in nouns and adjectives, such as “brilliant” or “friendly”

2. Biases in Language Style - this incudes three types:

   1. Language Formality - percentage of formal sentences

   2. Language positivity - percentage of sentences with positive sentiment),

   3. Agency language - percentage of sentence attributing agency to the candidate

3. Hallucination…