Over the years, I have had the opportunity to evaluate several of the “big name” resume and job matching applications that claim to use artificial intelligence, and I can say that the claim that they can find the same resumes that an “experienced recruiter” would choose is both accurate and inaccurate.
From my experience, most AI matching applications can return some well-matched resumes based on an example resume or job description. However, some of the results that are returned are definitely NOT good matches, although I can see why they were returned in the results. This is especially prevalent when searching for job descriptions/resumes/hiring profiles in which many different types of candidates can mention the same words in their resumes.
What I’ve found from my own extensive hands-on experience testing these applications is that in addition to some well matched resumes, every AI matching application I have tested also ranks some results relatively highly who match the keywords, but not the “essence” of the job or resume. By “essence” I mean what the person will be responsible for/what the candidate has been primarily responsible for. If the search results don’t match the “essence” of the example resume or the job description – they are of little to no value. Although in all fairness, junior sourcers or recruiters who are not very proficient at running Boolean searches may similarly struggle in creating Boolean search strings that return a high percentage of results of candidates who match the “essence” of an example resume or job description. Hopefully your staffing organization does not consist of only junior sourcers and/or recruiters who are not proficient at creating effective and precise Boolean queries.
I believe the root of the limitation of AI matching apps is that essentially, all an AI matching application is capable of doing is finding resumes that it “thinks” are matched, based on its algorithms and pre-programmed search logic. These applications are simply taking words from an example resume or a job description and matching them to words that someone (the person or team who programmed the application) decided was related and relevant. In other words – regardless of marketing hype – the applications are simply buzzword matching based on someone else’s programming. I’d argue that this is also essentially true for applications that make the claim that they “learn” as they are used. Let’s not forget the operative word in “Artificial Intelligence” is “Artificial”.
What should be especially disturbing to most people is that there is no objective way to assess whether the matching technology finds the BEST, or ALL, of the potentially qualified candidates in the database or system. In my opinion, finding SOME matches is not a significant accomplishment and certainly does not qualify an AI matching application to claim to be able to replace or be as effective as a human in the role of a sourcer or recruiter.
I think a good way to think about what all of these solutions do is to take a look at Amazon’s “you might also like” feature. While searching Amazon for a particular product, Amazon’s AI/matching engine tries to suggest other products you might be interested in based on the data collected from other online shoppers who have looked for or purchased similar products. Sometimes the suggestions are accurate – meaning you might actually be interested in the suggested product, but sometimes the suggestions are WAY off. What can we expect? After all – it’s just a computer program trying to “think” like a human (let alone YOU specifically).
Pretty much everyone in the staffing industry is aware of how imperfect resumes are in terms of their ability to represent a person’s experience and capabilities. The same is true of job descriptions. So you can pick your poison – allow a human sourcer or recruiter to interpret the the job description or the resume of the “ideal” candidate and create a search in an attempt to find qualified candidates, or allow an AI matching application to perform some exact and “fuzzy” buzzword matching.
Let’s get real here – a software application does not have the analytical and interpretive power of human cognition. With no example resume and a bare-bones job description, a good sourcer or recruiter can still find well qualified candidates. A software application cannot do the same thing – it can only “work” when given text to work with. Poor job descriptions will throw AI matching apps for a major loop and effectively render them useless. And while some job descriptions come with plenty of words – pages worth – in many cases all of the words actually don’t in fact accurately describe the position, its requirements, or define the ideal candidate.
Matching resumes is equally fraught with issues. Two candidates of similar skills, experience, and capability often have very different resumes. After all – the people we are looking to hire are not professional resume writers, nor should we expect them to be. Experienced sourcers and recruiters can “read between the lines” using cognitive processes and interpret resumes for skills, experience, and capabilities that are not explicitly mentioned in the resumes. This is something that an AI matching application may claim to do, but is hard pressed to prove. Let’s remember – AI matching apps can only work with text they are given – both the example resume or job description and the resumes it searches for. Although the people who program these applications can draw relation between multiple words/terms that can represent the same thing, they cannot possibly account for the unpredictable variety of ways in which a person can express – explicitly or implicitly – skills, experience, and capability.
AI matching applications do have value – in my opinion they provide “suggested reading”. They can certainly be used to assist junior sourcers/recruiters or sourcers and recruiters who are not particularly adept at creating effective Boolean search strings. However, I would never implicitly trust that the matching engine is finding all of and the best matches in your database. The funny thing is – you’ll never know if this is the case or not…unless you’re really good at running searches, because then you can run your own searches and see if you can find more and better candidates that the matching engine did not find. Which is what I do.
This should go without saying, but one thing you should always keep in mind is that that all of these vendors with AI matching engines are SELLING you a product. They know that creating effective Boolean search strings can be hard to learn so they want to sell their solution as something that relieves sourcers and recruiters of having to run searches to make matches.
In my opinion, an ideal solution for most staffing organizations would include a resume/candidate database/ATS with BOTH a fully configurable search interface that supports full Boolean logic and ideally extended Boolean (with at the very least configurable proximity and variable term weighting), AND an AI matching engine – not one at the expense of the other. AI resume and job matching is great, but it is not a magical solution that replaces the human cognition of talented sourcers and recruiters. While AI matching apps may help junior recruiters, or those people who simply aren’t “wired” to run Boolean searches, it is not something anyone or any staffing organization should trust with making all of their matches. That would be scary!
This article is part of the Boolean Black Belt archives. You can view the original article here.