How to tell real AI capability from resume inflation
There has never been an easier time to sound like an AI expert, or a harder time to be sure someone is one. The vocabulary is everywhere. Titles have inflated. Every team now has a story about how it uses machine learning. For anyone hiring senior AI talent, the central risk is no longer finding candidates. It is distinguishing genuine capability from a well-rehearsed surface. The pressure makes this harder. Around 87 percent of organisations report struggling to hire AI developers, with time-to-fill averaging close to 142 days, so the temptation to say yes to a fluent, confident candidate is intense, and that is exactly when expensive mistakes are made.
This matters because the cost of getting it wrong is asymmetric. A strong AI leader compounds the work of everyone around them, raises the technical bar, and makes good downstream hires. A leader who can talk the language but cannot do the work sets a false direction, makes hires in their own image, and the damage is not visible until months later when the results do not arrive and the team has been shaped around a hollow centre.
Surface signals are easy to fake
The signals most hiring processes lean on are exactly the ones that inflate most easily. A prestigious employer on a CV tells you someone was hired there, not what they actually did once inside, and large companies contain a wide range of contributors behind the same logo. A senior title tells you about internal politics and tenure as much as about capability. Familiarity with the latest models and techniques tells you someone reads and follows the field, not that they can build. None of these are worthless, but all of them are now cheap to acquire and easy to perform in an interview.
The deeper problem is that AI fluency is unusually performative. The field moves fast and rewards people who can speak about it confidently, which means the gap between sounding capable and being capable is wider here than in most domains. A candidate can hold a fluent, current, impressive-sounding conversation about frontier methods and still never have shipped anything that worked.
What real capability looks like under questioning
Genuine capability shows up when you move from the general to the specific. Ask a candidate to walk through a decision they personally made and why, not a project they were merely near. Ask what did not work and what they changed in response. Ask them to explain a hard tradeoff to a non-expert, because real understanding can be made simple while borrowed understanding collapses into jargon under that pressure. Ask where they think a popular technique fails, because people who have actually shipped have scar tissue and specific opinions, while people who have only read have enthusiasm and slogans.
The tell is specificity. Someone who has done the work owns the details: the dead ends, the judgment calls, the things they got wrong and fixed, and the things they would do differently next time. Someone who has been adjacent to the work speaks in the passive voice, credits "the team" for everything concrete, and stays at the level of the headline. The questions that separate them are not about cleverness or trick puzzles. They are about lived experience, and lived experience is hard to fake when the questioning is patient and specific.
Beware the two failure modes
Hiring processes tend to fail in one of two directions. The first is being dazzled by fluency and hiring the best talker. The second, in over-correction, is fetishising a narrow technical test and screening out genuinely capable leaders who do not perform well in an artificial exercise. The goal is neither. It is to assess whether this person has actually built and led real work at the level the role requires, using evidence that is hard to manufacture: specific decisions, honest accounts of failure, and the respect of credible peers.
Why references and networks matter more here
Because the surface is so easy to fake, the signal moves to sources that are harder to game. The most reliable read on an AI leader usually comes from people who have worked alongside them on real problems, and from a network deep enough in the field to know who is genuinely respected by their peers. Reputation among practitioners is slow to build and hard to fake, which is exactly why it is valuable. A handful of candid conversations with people who have actually shipped alongside the candidate will tell you more than any number of polished interviews.
Designing a process that resists inflation
Because the surface is so easy to perform, the process has to be designed to get underneath it without tipping into the opposite error of fetishising a narrow test. A few principles help. Favour work-grounded conversation over trivia, because the goal is to understand how someone thinks about real problems, not whether they can recall a definition. Bring a credible technical assessor into the loop, ideally someone who has done the work, because a peer can tell depth from fluency in minutes where a generalist cannot. Use references as a primary instrument, not a final formality, and ask referees specific questions about decisions, failures and judgment rather than inviting a character reference. And be explicit, internally, about which signals you are weighting and why, so the team is not quietly seduced by the most confident performer in the room.
The aim is calibration, not suspicion. Treating every candidate as a probable fraud is as damaging as treating every fluent talker as a genuine expert, because it screens out thoughtful people who interview modestly and rewards those who perform. The skill is to hold a high bar while staying open to the many forms real capability takes, some of which are quiet.
This is why hiring senior AI talent rewards depth in the specific community, not breadth across many. The firms and teams that consistently make good AI hires are the ones close enough to the work to know the difference between someone the market believes is excellent and someone who actually is. In a field this easy to perform, that distinction is the entire job, and it cannot be outsourced to a process that only sees the surface.