Case study: Yes, you can mitigate AI bias even with low resources
I'm sharing of the story of a startup to explain how they did it
Hi friends,
There's a pervasive myth in the AI industry that harmful bias is just the cost of doing business with generative AI. The story goes that unless you're Google or OpenAI with billions to spend, meaningful bias mitigation is out of reach. This simply isn’t true.
I've been studying how different organizations approach this challenge, and stumbled on a startup called Change Agent AI. They are doing it, and it was so exciting to see how. So I just had to share with the world how they do it!
They are not paying me and I’m not getting anything from this . I just want everyone to know that bias in AI is not inevitable - even if you’re a small startup!
You can download the full case study here, or read the summary below.
Change Agent's Approach to Bias
Change Agent AI offers a generative AI platform specifically designed for social impact and advocacy organizations. They provide both a standard chatbot that most clients use and customized versions tailored to individual organizations. As a small startup founded in 2023, they've managed to develop effective bias mitigation approaches while working with limited resources and without building models from scratch.
Change Agent approaches bias from a fundamentally different perspective: they see bias as endemic to languages and LLMs. They see language itself as never being socially or politically neutral. CEO Aram Fischer puts it bluntly: "Mainstream AI pretends that training on available data is 'neutral' when that is, in our estimation, a cop out—they just don't want to be responsible for the bias they encode through their choices."
Instead of pursuing the myth of neutrality, Change Agent makes intentional choices about which values to embed in their AI systems. They focus on encoding principles they believe in, such as anti-racism, anti-misogyny, and anti-homophobia.
Values-based decisions extend to client selection as well. Aram explains their approach: "We don't serve clients who question women's bodily autonomy, for example ."
Bias Detection Methods
Change Agent uses "red team testing" to identify bias before it becomes a problem. This involves deliberately crafting prompts designed to elicit biased responses, exposing undesirable behavior.
For example, when asked about redlining (discriminatory lending practices), ChatGPT described it as a "mid-20th century practice" that ended. This subtle framing suggested that redlining ended decades ago, which is false—redlining extended well beyond the 1968 Fair Housing Act and its effects continue today. Catching this type of subtle bias shows how red team testing can reveal problematic content that might otherwise go unnoticed.
With limited resources, they've developed a practical approach to prioritization:
They organize by issue areas (gender, race, etc.) while acknowledging intersectionality
They monitor public incidents with other AI systems: "If I see something on Twitter... 'Oh, ChatGPT answered this way,' I'm going to make sure ours doesn't do that too," explains CPO Tyler McFadden
They adjust priorities based on client needs: "If we have a sudden influx of reproductive justice clients, that's something that I'm going to make sure that we're on top of"
They maintain an "open door policy" for feedback from clients and communities they serve
Bias Mitigation Approaches
Once bias is identified, Change Agent uses two primary approaches to address it.
In simple cases they use system prompts, which are direct instructions given to the chatbot about how to behave. "Sometimes it's simple—don't use slurs. I think of those as like a ‘thou shalt’ - just don't do that," explains CTO Craig Johnson. For example, they might add prompts such as "don't be racist" or "don't be misogynistic" behind the scenes to guide the AI's responses on clear-cut issues.
For more complex concerns, they use fine-tuning techniques. Fine-tuning is essentially teaching the chatbot to do better by giving it examples of desired behavior and training it to mimic them. Their approach includes:
Enriching their model with content that reflects their values. "If there is something that is anti-trans that comes up, we're gonna front load [our chatbot] with trans theory... movies, even movie scripts, or song lyrics," Craig explains.
Creating synthetic data by developing custom question-answer pairs. As Tyler McFadden notes, they carefully craft examples for complex issues such as "representation in leadership, economic inequality, or systemic discrimination."
Last, rather than pursuing perfect solutions, Change Agent embraces what they call a "forever beta" mindset. "When they run into a new problem, they tell me and then I fix it," explains CTO Craig about their responsive relationship with clients.
Takeaway: Bias Mitigation is Within Reach
The case of Change Agent AI demonstrates that harmful bias in AI isn't inevitable. While completely eliminating bias may be as difficult as eliminating it from human society, organizations can significantly reduce harmful bias through intentional efforts.
What makes their approach particularly noteworthy is that it doesn't require massive resources. It's about making clear value choices, incorporating diverse perspectives, and implementing practical methods that match your capabilities.
What steps is your organization taking to address bias in AI systems? The journey toward more responsible AI starts with recognizing that the responsibility for addressing bias falls on all organizations using these technologies. And as Change Agent shows, it is a responsibility that's within reach for companies of all sizes.
Dessert
An AI-generated take on this post!
Ready for more?
Read more of my posts about AI bias, such as bias in AI generated reference letters, and bias in image generation. Search for “bias” in the AI Treasure Chest Archive to see them all.
Interesting approach, but what about the fact that, because of the inherent nature of probabilistic sampling and the tendency to confabulate, an LLM can quite often just ignore system prompts or finetuning instructions? Seems like you're always going to be playing bias whack-a-mole.