AI Can Discriminate in the Workplace, EEOC Warns Employers

Robots are not just coming to take our jobs, they’re increasingly deciding who gets hired and who gets fired. They’re also evaluating our job performance, and they may be discriminating against us, too.

On Thursday, the Equal Employment Opportunity Commission released a technical assistance document designed to help employers prevent artificial intelligence from discriminating against job seekers and workers.

“As employers increasingly turn to AI and other automated systems, they must ensure that the use of these technologies aligns with the civil rights laws and our national values of fairness, justice and equality,” said EEOC Chair Charlotte Burrows. 

The EEOC is charged with enforcing Title VII of the 1964 Civil Rights Act, which prohibits discrimination based on race, color, national origin, religion, or sex – including pregnancy, sexual orientation, and gender identity. 

Computers, software, algorithms and emerging artificial intelligence technologies are increasingly being used by employers to help screen applicants, but they aren’t always programmed to analyze every human trait. 

In fact, demographic data is often removed from AI systems to help it make blind decisions, but this practice can make AI decision-making more discriminatory, particularly when sorting through large and skewed data sets, according to a study published in the Harvard Business Review in March. 

“This became evident in 2019 when Apple Card faced accusations of gender-based discrimination despite not having used gender data in the development of their ML [machine learning] algorithms,” the Harvard researchers wrote. “Paradoxically, that turned out to be the reason for the unequal treatment of customers.”

Ironically, the Harvard researchers noted their own data problems. “In our study, gender was reported as either woman or man; we acknowledge gender is not binary, but were restricted by our dataset,” they wrote.

The Harvard Researchers also noted a case study from Amazon which designed a machine learning algorithm to predict job performance. The algorithm used resumes of current employees with gender and names removed to prevent discrimination.

“The result was the exact opposite — the algorithm discriminated against women, by predicting them to have significantly lower job performance than similarly skilled men,” Harvard researcher wrote. “Amazon, thankfully, caught this discrimination before the model was used on real applicants.”

A recent survey by NewVantage Partners shows that 8 out of 10 firms plan to invest in some form of machine learning technology this year. “Investments in data and analytics capabilities and initiatives are greater than ever,” study authors wrote, “but organizations are coming to the growing realization that embedding these capabilities within their firms and transforming the cultures of their organizations will take time.”

About 83% of employers now use automation as part of their hiring process, according to the EEOC, and several issues have already emerged.

Resume scanners, for instance, may reject applicants with employment gaps, possibly turning down qualified applicants with disabilities. Software that analyzes speech patterns might give low scores to people with speech impediments. Programs may also be excluding people with small social media presences or older workers unfamiliar with some of the latest technology.

A study published in 2019 revealed that a healthcare prediction algorithm, similar to tools used on 200 million patients, heavily favored white patients over black patients in determining which patients might need extra medical care.

In what is certain to be a harrowing transition for humanity as AI develops, the EEOC wants employers to proceed with caution.

“I encourage employers to conduct an ongoing self-analysis to determine whether they are using technology in a way that could result in discrimination,” said Burrows. “This technical assistance resource is another step in helping employers and vendors understand how civil rights laws apply to automated systems used in employment.”