MArtin Burch had been working for the Wall Street Journal and his parent company Dow Jones for a few years and was looking for new opportunities. One Sunday in May 2021, I applied for a data analyst position at Bloomberg in London that looked like the perfect fit. He received an immediate response, asking him to take a digital assessment.
It was strange. The assessment showed him different shapes and asked him to figure out the pattern. I have started feeling incredulous. “Shouldn’t we be testing my abilities on the job?” he asked himself.
The next day, a Monday, which happened to be a public holiday in the UK, he got a rejection email. I have decided to email a recruiter at Bloomberg. Maybe the company made a mistake?
What Burch discovered offers insight into a larger phenomenon that is baffling experts: while there are record level job openings in both the UK and in the US, why do many people still have to apply to sometimes hundreds of jobs, even in sought-after fields like software development, while many companies complain they can’t find the right talent?
Some experts argue that algorithms and artificial intelligence now used extensively in hiring are playing a role. This is a huge shift, because until relatively recently, most hiring managers would handle applications and resumes themselves. Yet recent findings have shown that some of these new tools discriminate against women and use criteria unrelated to work to “predict” job success.
While companies and vendors are not required to disclose if they use artificial intelligence or algorithms to select and hire job applicants, in my reporting I have learned that this is widespread. All the leading job platforms – including LinkedIn, ZipRecruiter, Indeed, CareerBuilder, and Monster – have told me they deploy some of these technologies.
Ian Siegel, the CEO of ZipRecruiter, said that artificial intelligence and algorithms have already conquered the field. He estimates that at least three-quarters of all resumes submitted for jobs in the US are read by algorithms. “The dawn of robot recruiting has come and went and people just haven’t caught up to the realization yet,” he said.
A 2021 survey of recruiting executives by the research and consulting firm Gartner found that almost all reported using AI for at least one part of the recruiting and hiring process.
Yet it is not foolproof. One of the most consequential findings comes from Harvard Business School professor Joe Fuller, whose team surveyed more than 2,250 business leaders in the US, UK and Germany. Their motives for using algorithmic tools were efficiency and saving costs. Yet 88% of executives said that they know their tools reject qualified candidates.
Despite the prevalence of the technology, there have just been a few famous cases of misfires. A few years back, Amazon discovered that its resume screener tool was biased against women. The algorithm was trained on resumes of current employees, who skewed male, reflecting a gender disparity in many tech fields. Over time, the tool picked up on male preferences and systematically downgraded people with the word “women” on their resumes, as in “women’s chess club” or “women’s soccer team.” Amazon’s engineers tried to fix the problem, but they couldn’t and the company discontinued the tool in 2018.
“This project was only ever explored on a trial basis, and was always used with human supervision,” said Amazon spokesperson Brad Glasser.
AI vendors that build these kinds of technologies say that algorithm-based tools democratize the hiring process by giving everyone a fair chance. If a company is drowning in applications, many human recruiters read only a fraction of the applications. An AI analyzes all of them and any assessments and judges every candidate the same way.
Another benefit, these vendors say, is if employers choose to focus on skills and not on educational achievements like college degrees, applicants from diverse backgrounds who are often overlooked can get to the next stage of the process.
“At the end of the day, we don’t want people to be hired into roles that are going to drain them and not utilize their strengths. And so it’s really not about rejecting people, it’s about ‘screening in’ the right people,” said Caitlin MacGregor, CEO of Plum, which built the assessment Burch found so puzzling. MacGregor said the company’s clients have increased their diversity and retention rates since they started to use Plum. She said the assessments helped hone in on applicants’ “potential”.
But job candidates who have the necessary experience worry they’re being unfairly weeded out when companies focus on elusive factors like potential or personality traits.
“This was the first time in my life, in my career, where I was sending out resumes and there was nothing,” said Javier Alvarez, 57, a distribution and sales manager from Monrovia, California, who sent out his resume more than 300 times on sites like LinkedIn and Indeed for jobs he said he was qualified for. No job offer materialized, and he started to wonder if he was being automatically excluded in some way – perhaps because of his age or salary requirements from him. “I felt hopeless. I started to doubt my abilities.”
Ronnie Riley, a 29-year-old event planner from Canada, had a gap of several years in their resume because of an illness. Riley applied to more than 100 event planning and some administrative assistant jobs in December 2021, and over 70 jobs in January, but ended up with a total of five interviews and no job offers. They worry about the gap is the reason. “It just seems it’s discounting a whole bunch of people that could be perfect for the job,” they said.
Fuller’s research has helped provide answers to how exactly automatic rejections occur. One reason, I have found, is that too often, job descriptions include too many criteria and skills. Many employers add new skills and criteria to existing job descriptions, building a long list of requirements. Algorithms end up rejecting many qualified applicants who may be missing just a couple of skills from the list.
One executive Fuller spoke with said their company’s tool has been rejecting qualified candidates because they scored low in one important category, even when they got a near perfect score in all the other important categories. The company found that it was left with job applicants who received mediocre scores across the board. (Longer job descriptions may also determine more female applicants, Fuller believes, since many women apply to jobs only when they fulfill most of the requirements.)
Another reason qualified candidates are rejected by automated systems are so-called knockout criteria. In Fuller’s research, I have found that almost 50% of the executives surveyed acknowledged that their automatic systems reject job applicants outright who have a work gap longer than six months on their resumes. These applicants never get in front of a hiring manager, even if they are the most qualified candidates for the job.
“The six month gap is a really insidious filter,” said Fuller, since it’s probably built on the assumption that a gap signifies something ominous, but may simply represent military deployments, pregnancy complications, caregiving obligations or illness.
Experts contacted by the Guardian also described automatic resume screeners making mistakes similar to the infamous Amazon example, rooted in learning biases from an existing dataset. This hints at how these programs could end up enforcing the kinds of racial and gender biases observed with other AI tools, such as facial recognition tech and algorithms used in health care.
John Scott is the chief operating officer of APMetrics, an organization that helps companies identify talent, and is often brought in by larger companies to check if new technologies the company wants to buy from a vendor are fair and legal. Scott has examined multiple resume screeners and recruiting tools and discovered problems in all of them. He found biased criteria unrelated to work, such as the name Thomas and the keyword church, to “predict” success in a job.
Mark Girouard, an employment lawyer in Minneapolis, found that the name Jared and having played lacrosse in high school were used as predictors of success in one system.
Martin Burch, the London jobseeker, discovered he had been weeded out in a different way.
I have contacted a human recruiter at Bloomberg and asked her to look at his CV. His experience of him lined up with the job description and this was a direct competitor, making his background of him all the more valuable, he thought. But the problem turned out to be the pattern-finding and personality test he had taken, which was created by Plum.
A recruiter at Bloomberg replied: “I can see that your application was rejected due to not meeting our benchmark in the Plum assessment that you completed. Unfortunately on that basis we are not able to take your application any further.” Burch felt stunned that he had indeed been rejected by a piece of code.
I have retained a lawyer, and in communications with Bloomberg asked for a human review of his application.
Bloomberg informed Burch that the role he applied for was no longer available and he wouldn’t be able to be considered for it.
Bloomberg did not return emails and calls asking for comment.
As adoption of AI tools in hiring expands, lawmakers are starting to take a closer look. In the UK, the government is planning new regulation of algorithmic decision making. In the US, a recent local law requires employers to inform job seekers how their application materials are screened by AI upon request. And congressional lawmakers have introduced bills that would regulate AI in hiring at a national level, including the Algorithmic Accountability Act of 2022, but have faced hurdles getting them passed.
Burch decided to file an official claim with the Information Commissioner’s Office, an independent organization that upholds privacy laws in the UK. In February the office reprimanded Bloomberg, writing: “From reviewing the information provided, it is our decision that there is more work for you to do. As such, we now expect you to take steps to address any outstanding issues with the individual.”
Burch has since accepted £8,000 ($9,864) in compensation from the company. He says he also fought to demonstrate a point: “I am trying to prove to them that it’s probably weeding out good candidates so they should probably stop using it.”
Plum’s CEO Caitlin MacGregor declined to comment on Burch’s case directly, citing privacy concerns, but she stands behind her product: “I should not be interviewing somebody that is a 35, regardless of how much experience they have. There is somewhere else that they are going to be their own 95 [percent] match.”
How to write a resume in the age of AI
-
Instead of trying to stand out, make your resume machine-readable: no images, no special characters such as ampersands or tildes. Use the most common template. Use short, crisp sentences – declarative and quantitative, said Ian Siegel, CEO of the job platform ZipRecruiter
-
List licenses and certifications on your resume
-
Make sure your resume matches the keywords in the job description and compare your resume to the job description using online resume scanners to see if you are a match for the role
-
For entry-level and administrative jobs, consider stating that you are competent in Microsoft office suite applications even if it’s not in the job description, said Harvard business professor Joe Fuller.