Mitigating Bias in AI applications:

Applications such as generative Artificial Intelligence aka GenAI, computer vision and Natural Language Processing aka NLP can find insights, make predictions, and streamline operations. But as these applications become more sophisticated in learning and reasoning, they are highly dependent on data used to train them. These data inevitably contain biases. Biased algorithms limit the potential of AI and lead to flawed or distorted model outputs that can negatively impact marginalized groups.

For example, the bias demonstrated by Chatbots has surfaced as follows:

Racial bias in salary recommendations: A Stanford Law School study found that chatbots like ChatGPT 4 and Google's PaLM-2 suggested lower salaries for job candidates with Black-sounding names compared to white-sounding names. For instance, a candidate named Tamika was recommended a $79,375 salary as a lawyer, while Todd was suggested $82,485 for the same position.

Hidden racial bias in language perception: While AI models tend to use positive words when directly asked about Black people, they display negative biases towards African American English. Users of this dialect were often associated with "low status" jobs like cook or soldier, while Standard American English users were linked to "higher status" jobs like professor or economist.

Gender bias in translations: ChatGPT was found to perpetuate gender stereotypes when translating between English and languages with gender-neutral pronouns. For example, it tended to assign "doctor" to men and "nurse" to women.

Cultural biases: A comparison between ChatGPT (US-based) and Ernie (China-based) showed that ChatGPT displayed implicit gender biases, while Ernie showed more explicit biases, such as emphasizing women's pursuit of marriage over career.

Socioeconomic biases: Researches use medical images as training data to recognize diseases and even while it is infused with human expertise to annotate the images, the amount of data available might not only be skewed by those who could afford to have their images taken but also by those who could afford but had no reachability to the diagnostic imaging or representation in its collection.

From these examples, bias can be tracked to “blind spots” in training data. And as it takes many forms such as selection, exclusion, prejudice, cognitive, confirmation, historical and availability biases, outcomes often disenfranchise groups of people. That is not the only risk though. Algorithms trained on biased data can falsely predict risk, negatively impact hiring, lending and more, expose existing societal prejudices, create mistrust across boundaries and even lead to fines in regulated environments. How data is collected, who collects it, whether it is wholesome in representation, and such others determine how biased the data is. More often than not, collection of data is from existing literature and these sources already demonstrate influences.

Diverse and Inclusive data sets are the biggest antidote to biases. AI models trained on a broad swath of sources do better to respond to queries from a vast group of people. As part of this strategy, methods to detect and mitigate biases in data and continuous refinement and updates to datasets are mandatory. Some of this can be realized by having a large network of data contributors, facilitation of peer reviews, multimodal data collection, medallion architecture curation, inclusive data sources and robust coverage, timestamped or versioned data, securing data at rest and in transit, least-privilege access control, ubiquitous reach to and from data, and facilitation of asset intake and metadata collection