There’s now an open-source tool to help detect biased language in writing and AI datasets, developed by a Canadian research hub.
**The news: **Toronto’s Vector Institute released UnBias-Plus on Tuesday, a free tool that Vector research scientists say can detect and rewrite biased language, both in written content and within AI training data. Its developers say the tool scans for biased language regarding race, gender, age, and political framing, then generates an explanation as to why it’s been flagged and suggests “neutral” alternatives.
From the source: “ What drove us to build this was simple,” Shaina Raza, a Vector Institute applied machine learning scientist, said in a statement. “The people most harmed by biased language are often the last to know it’s there. A patient doesn’t see the assumptions buried in their clinical notes. A job candidate doesn’t know why a door keeps closing.”
Following the thread: Since large-language models (LLMs) have largely been trained on a collection of human-generated data, they can replicate humanity’s social biases, including those that are racist, sexist, or otherwise discriminatory. For example, algorithmic hiring tools that screen job candidates have been shown to recreate systemic bias against Black and Asian applicants in the US. In healthcare, a London School of Economics and Political Science study found that AI tools used by UK councils downplayed the severity of women’s health issues compared to men’s.
**Final thought: **Vector’s new tool aims to catch those structural problems and, in doing so, help Canadian organizations align with the country’s national AI strategy, which identified bias as a challenge. The government’s new legislation regarding online harms doesn’t propose explicit fixes to get rid of bias in AI models—it simply imposes a “Duty to Act Responsibly” upon social media and AI chatbot services, which includes mitigating the risk of exposing users to harmful content.
Feature image courtesy Firosnv. Photography via Unsplash.