AI-powered hiring systems are increasing the rate at which Black and Asian candidates are screened out, with research showing that some of these tools still carry inherent human biases.
A new study led by researchers at Stanford University shows that employers using the Pymetrics hiring platform, a high-volume AI screening tool that uses online games in the application process, had systemic rejection patterns and human-like biases. It examined a dataset from December 2018 to December 2022, covering 4 million applications across 156 employers.
According to the research paper, about one in 10 positions had “adverse impact” for Black applicants and about one in 20 for Asian applicants. Although this was only a minority of positions, those postings accounted for a much larger share of affected applications, affecting 26 percent of Black candidates. This suggests the bias was concentrated in the highest-volume roles, or in roles that attracted more Black and Asian candidates.
Adverse impact is a term used by the federal government to denote when a race, sex, or ethnic group is selected at less than four-fifths of the rate of the most-selected group.
Alongside bias in the model, employers who use the Pymetrics platform often rely on the same algorithmic setup to screen candidates. If a candidate was rejected from one organization, they were more likely to be rejected from another using the same algorithm. The study said that 42 models were shared across the 156 employers.
Explosion in AI usage for hiring
While the study used data from before the explosion in AI usage after the launch of ChatGPT, it is another high-profile study showing that inherent hiring biases are intensifying rather than being reduced through the use of AI. This comes alongside candidates having to go through an elongated testing process, with very few making it to a stage where they interact with a human.
AI usage in the hiring process has increased dramatically over the past few years. According to the Society for Human Resource Management, AI adoption in HR increased from 26% of organizations in 2024 to 43% in 2025.
The Stanford study is just one of a growing number that have shown AI is not infallible when it comes to the hiring process. A study by researchers at the University of Illinois and Ahmedabad University found that AI hiring recommendations favored men over women, with women often recommended for lower-wage roles.
Add to that the fact that Workday, one of the biggest human resources software companies, has recently been sued over claims that its AI hiring software unfairly screened out candidates, and it looks like we are still far away from the utopian dream of AI removing bias from hiring.
EU AI Act to improve the quality of AI hiring systems
This was part of the reason hiring was identified as a key high-risk area by the EU AI Act, which requires AI systems to have stronger safeguards before and during use. Any hiring company or employer deploying AI in the region needs to maintain proper technical documentation, transparency, and human oversight to ensure compliance with requirements.
The Act targets both the businesses that deploy these AI systems and the model makers, ensuring that both take more care when designing, training, and deploying these systems.
There are inklings that the United States may be moving toward broader AI hiring regulation. New York, Colorado, and Illinois have all enacted laws that increase the risk of using AI systems for hiring, with potential penalties if these systems are not audited or fail to meet key requirements. California and other states have blended AI hiring practices into existing laws, ensuring that businesses and AI model developers meet certain requirements.
The larger question is no longer whether AI can speed up hiring. It is whether employers can prove these systems are fair, explainable, and compliant before they filter thousands of candidates out of the process.
Also read: For more on how AI is reshaping the workplace, check out our coverage of Cloudflare’s workforce cuts despite record revenue.