Do AI Interviewers Actually Work? What the Research Says

Ask a vendor "do AI interviewers work" and you'll get a yes and a case study. Ask a researcher and you'll get "for which roles, measured how, under what deployment?" We wrote this article for the second crowd. Below is what the research actually says, how strong that research is, and where AI interviewing falls apart in practice.

A quick scope note: we're talking about conversational AI interviewers, the voice and chat agents that talk to a candidate and collect structured responses. Resume parsers, matching scores, and AI screening broadly are a different conversation, one we cover in the ultimate guide to AI recruiting.

The study that reset the conversation

In 2025, economists Luca Henkel (Erasmus University Rotterdam) and Brian Jabarian (University of Chicago Booth) ran the largest controlled test of AI interviewers to date, randomly assigning roughly 70,000 applicants to a human recruiter, an AI voice agent, or a free choice between the two.

The recruiters themselves had predicted the opposite of what happened. Candidates interviewed by the AI were 12% more likely to receive an offer and more likely to still be employed a month later. When given the choice, 78% picked the AI.

The fine print matters. These were high-volume, entry-level customer service roles, a near-ideal case for structured screening. Technical difficulties hit about 7% of AI interviews, and a human made every final hiring decision after reviewing transcripts and test scores. Strong evidence for frontline screening. Not evidence that AI should pick your next VP of Engineering.

Executive takeaway: The best evidence for AI interviewing is real and rigorous, and it is specific to structured, high-volume, first-round screening. Treat it as a boundary, not a blank check.

Throughput: strongly supported

This is the least contested claim in the debate. An AI agent runs interviews in parallel, at 2am, in seven languages, without a calendar. Where a human phone screen might reach a candidate in three to five days, an AI screen can happen within hours.

The mechanism is straightforward: remove the recruiter's minutes per candidate as the bottleneck and time-to-first-response collapses. That matters because response time drives completion. Hourly candidates in particular finish conversational screens at rates well above what typing-heavy or callback-dependent processes produce. (More on what drives those gaps in our breakdown of how to navigate high-volume recruitment.)

The caveat is the whole game. Throughput only counts if the quality of who advances holds up. Screening five times as many people faster is worthless if you're advancing the wrong five times as many. The real question is whether the candidates AI advances are as good or better.

Strength of evidence: high. Multiple datasets, consistent direction, and it matches basic operational logic.

Executive takeaway: Faster screening is a means, not a result. If throughput rises but your interview-to-offer ratio doesn't hold, you sped up a broken funnel.

Candidate experience: mixed by design quality

Averages are misleading here. The Henkel and Jabarian study found satisfaction held steady across AI and human interviews, with 78% preferring the AI when given the choice. In follow-ups, many candidates, especially women, reported feeling less judged and less anxious with the AI. That finding is worth taking seriously.

But that study used a carefully designed voice agent. "AI interviewing" also covers a clumsy chatbot that loops on the same question. Experience tracks design quality, not the technology label. A 2023 survey found 66% of U.S. adults say they would not want to apply for a job where AI helps make hiring decisions, with women 50 and older most opposed. Phone-comfortable candidates tend to rate AI screens well. Candidates who need accommodations or want to ask a human a question tend to rate them lower.

The aggregate NPS bump you'll see in vendor decks is real on average and misleading in the particulars. The average hides the candidates your deployment is failing.

Strength of evidence: moderate. Aggregate improvement is consistent, but modality and design quality swing the result enough that any single number is suspect.

Executive takeaway: Don't buy the average. Segment candidate experience by population, and watch the tails, because that's where adverse impact hides.

Quality of hire: structured AI beats unstructured human

The evidence is strong here, but only for one comparison: structured AI screening versus unstructured human interviews. Henkel and Jabarian found AI-interviewed candidates were more likely to still be on the job at 30 days, a quality signal, not just a speed one.

The mechanism is older than any AI product. Structured interviews, where every candidate gets the same questions scored against the same rubric, predict job performance at r = .51 against r = .38 for unstructured ones, roughly a third more predictive (Schmidt & Hunter, 1998). AI doesn't add magic. It enforces structure that human interviewers drift away from by the third screen of the day. For a fuller treatment, see what recruiters get wrong about AI interview accuracy.

The caveat: the strongest configuration is AI screen plus human final round, not AI-only selection. Henkel and Jabarian kept humans on the hiring decision. No comparable evidence base exists for AI making final calls on senior or complex roles.

Strength of evidence: high for structured AI versus unstructured human. Weak for AI-only selection at senior levels.

Executive takeaway: The quality gain comes from consistency, not from the model's opinion. Use AI to standardize the screen and keep humans on the decision.

Equity: it depends entirely on deployment

AI interviewing can reduce demographic disparities at the screen or encode them at scale. Which one you get depends on training data and audit practice, not the technology.

The optimistic case is real. Unilever's AI-assisted screening process increased new-hire diversity by about 16% while cutting time-to-hire by 90%. A consistent, structured screen removes some of the interviewer-to-interviewer variability where bias enters.

The risk is equally real. If your scoring rubric is trained on who your company hired before, the AI can learn and amplify that pattern faster than any single biased manager. New York City's Local Law 144 exists because legislators recognized this, requiring an annual third-party bias audit for automated employment decision tools used in city hiring. Deployment without an audit isn't equity-neutral. It's equity-risk. Our view on where bias sneaks back in is covered in fairness in AI interviewing.

Strength of evidence: moderate. Positive results exist, but they're deployment-dependent and the counter-risk is well documented in law and practice.

Executive takeaway: AI interviewing is equity-positive only with an outcome audit attached. No audit, no claim.

Three failure modes no vendor will show you

If the sections above are the honest case for AI interviewing, these are the honest conditions under which it fails. If you're evaluating AI interviewing, you need to see both sides.

1. Deployment without a bias audit. AI screening at high volume with no outcome monitoring has produced documented adverse-impact cases. The technology does whatever its rubric rewards, consistently, which is exactly why unmonitored deployment is dangerous. The fix is straightforward and non-negotiable: an annual bias audit and ongoing outcome monitoring by segment. At Humanly, we treat that audit cadence as part of the product, not an afterthought.

2. Single-modality deployment across a mixed pool. A voice-only agent aimed at hearing-impaired candidates, ESL speakers, and people without reliable phone access will produce uneven completion rates. Those gaps become adverse impact at scale. Deploy one modality without a documented human fallback path and you're building disparity into the funnel by design. Multimodal coverage across chat, voice, and video is one way to close that gap, which is part of why we built Humanly's AI Interviewer to support multiple channels.

3. AI replacing human judgment in senior roles. The evidence base, Henkel and Jabarian included, is frontline and entry-level. No equivalent research shows AI interviewers work for manager, director, or executive hiring. What limited signal exists points the other way: lower completion, lower satisfaction, no quality advantage. Deploying AI screening for senior roles borrows credibility from studies that never tested it.

Executive takeaway: If you can't produce a bias audit, a fallback path, and role-appropriate scope, you don't have a defensible AI interviewing program. You have a risk you haven't measured.

The verdict on whether AI interviewers work

Yes, within defined lines. The strongest evidence supports structured conversational AI for frontline and first-round screening, with a human on the final decision and a bias audit on the calendar. The weakest evidence covers senior roles and single-modality deployments with no accessibility alternative. Be cautious of certainty claims that extend beyond those lines.

Match the tool to the role, keep humans on judgment, audit outcomes, and the research is on your side. That's the principle behind how we built Humanly's end-to-end hiring platform: AI handles the structured screening, humans stay on the decision, and audit trails are built in from the start.

If you want specific outcome data for your role types rather than a generic average, talk to our team about what the numbers look like for your funnel.

FAQs

Do AI interviewers actually work?

Yes, for specific use cases. A 70,000-candidate field experiment by Henkel and Jabarian shows structured AI interviewers can match or beat unstructured human phone screens for high-volume, entry-level, and first-round roles when a human makes the final decision. Evidence is weak to nonexistent for AI making final calls on senior or complex roles.

How accurate are AI interviews compared to human ones?

Accuracy comes from structure, not from the AI itself. Structured interviews predict job performance at r = .51 versus r = .38 for unstructured ones (Schmidt & Hunter, 1998). AI's advantage is enforcing that structure for every candidate, eliminating the drift that erodes human accuracy over a long day of screening.

Are AI interviewers biased?

They can be, and they can also reduce it. An AI scored on historical hiring patterns can encode and amplify past bias at scale. A well-audited AI screen can reduce the interviewer-to-interviewer variability where bias enters. The deciding factor is whether you run an annual bias audit and monitor outcomes by segment, which is why NYC's Local Law 144 requires exactly that for automated employment decision tools.

Should we use AI interviewers for senior or executive roles?

The evidence doesn't support it. Every rigorous study to date covers frontline and entry-level hiring. For senior roles, AI screening tends to produce lower completion, lower candidate satisfaction, and no demonstrated quality gain. Keep AI for first-round and high-volume screening; keep human judgment on complex, senior decisions.

What's the difference between AI interviewing and AI screening?

AI interviewing means a conversational agent, by voice or chat, that talks with a candidate and collects structured responses. AI screening is the broader category that also includes resume parsing, matching scores, and ranking. This article is about conversational AI interviewers specifically; the wider category is covered in our guide to AI recruiting.