We need to open the AI black box before it's too late.

For a number of years in the 1980s, applicants to St George’s Hospital Medical School in London were selected with a high-tech method. A computer program, one of the first of its kind, took the first look at their résumés, carrying out the initial selection of about 2,000 candidates every year. The program analyzed the admissions records to learn the characteristics of successful applications, and was adjusted until its decisions matched those of the admissions team.

But the program had learned to look for more than good grades and signs of academic prowess. Four years after the program was implemented, two doctors at the hospital discovered the program tended to reject female applicants and those with non-European-sounding names, regardless of their academic merit