Getting Smart About Artificial Intelligence

Artificial intelligence and machine learning have become so prevalent that many businesses now heavily rely on algorithms to make decisions that can impact our daily lives, from security scans and medical diagnostics to mortgage approvals and customer service interactions.

Indeed, the technology has become so ingrained into routine operations that there are serious concerns that AI does not just facilitate processes, but often dictates them—without effective oversight, transparency or accountability.

There have been numerous high-profile incidents involving AI failures. In 2018, Amazon scrapped the AI-powered recruiting software it was developing after finding the system vastly preferred male candidates.

he machine learning models at the heart of the system were trained on 10 years’ worth of resumes submitted to the company, most of which were from men, a reflection of the gender gap in the tech industry.

As a result of that training data, the system learned that male candidates were preferable and started penalizing resumes that included the word ‘women’s’ and even downgraded candidates from all-female colleges. In effect, the technology learned the existing gender bias in hiring and attempted to formally incorporate it into the system.