Nigel Cannings, founder and CTO at Intelligent Voice, a company leading the international development of proactive compliance and technology solutions for various forms of media, considers the dangers of biased AI and the impact it could have on diversity, equity and inclusion 

Artificial intelligence (AI) was deployed by around 15% of UK businesses in January 2022, with a further 12% either trialling it or intending to adopt it shortly. It is being used across sectors, from IT to legal and health services and it is being used for a range of processes, including data analysis, machine learning (ML), HR, recruitment, and language processing. AI is becoming such a versatile tool, that UK spending on the technology is predicted to reach £83.5 billion in 2040 – up from just £16.7 billion in 2020. As a labour and time-saving tool, AI is incomparable, enhancing operational efficiency at the same time as saving money. But.

For all its positives, there is no getting away from the concern of AI bias and what this means for diversity, equity, and inclusion (DE&I).

AI bias and its impact on DE&I

In the early stages of development, AI was positioned as an opinion-free decision-maker. Although unable to ‘think’, AI would enable machines to analyse data in such a way that facilitated rapid, impartial, dispassionate decision-making. And that vision has been realised – up to a point. Because while AI is theoretically capable of impartiality, it can only reach decisions based on the data which is used to train it. So, if the data fed into the system during the training process holds any form of bias, all decisions subsequently reached by the AI software may carry that same prejudice. And the major problem that we face is that without this ‘human labelled data’ that is so subject to individual personal bias, there is – currently – no way to program AI. So, we can’t just remove the bias at source.

What this means for DE&I is that unless AI bias is addressed, as the technology is taken up by more and more businesses, the potential for unintended discrimination will become unprecedentedly huge.

There has already been a number of embarrassingly high-profile cases of AI bias documented. PredPol, America’s ‘predictive policing’ algorithm was found to be racist. Amazon’s AI recruitment system was sexist. IDEMIA’S facial recognition algorithm was biased against black women. While the AI used by Apple to make credit decisions was found to be misogynistic. In each of these cases, the software was withdrawn from use – at least until the problem could be addressed. But with AI being so widely adopted, what is to say that these issues won’t recur on a smaller scale in places where they won’t be detected? And how can we stop this from having a damaging impact on DE&I?

Can AI be debiased?

We’ve already established that without ‘human-labelled data’ there can be no AI but with this data, there will always be scope for bias. Until we create fully autonomous self-training technology that can build a database without human intervention, there is no way to remove the potential for bias to find its way into AI. However, the issue can be addressed with explainability.

Explainable AI (XAI) allows us to interrogate AI to ascertain what information it used to make any given decision, on a case-by-case basis. And how that decision was made. This means that although bias cannot be removed from the decision-making process from the get-go, if the reasoning behind any decision is proven faulty or biased, not only can that decision be overruled, but the data that led to it can be corrected.

In the first instance, for DE&I, this means the opportunity for individuals to challenge decisions they may view as unfair, whether in recruitment, remuneration, training, or credit. More broadly, it opens the door to discrimination-free business, where data alone fuels decisions, but that scenario relies upon the widespread adoption of XAI.

AI is an incredible tool and so far, its full potential is nowhere near being tapped. However, before that can happen, we need to get AI bias well and truly under control.

About the author

Nigel Cannings is the CTO at Intelligent Voice. He has  30 years of experience in both Law and Technology, is the founder of Intelligent Voice Ltd and is a pioneer in all things voice. Nigel is also a regular speaker at industry events, such as NVIDIA GTC.