What’s the impact of cheating on skills-based hiring?

Picture this. 

You’re hiring a web developer. You’re looking for someone who can produce clean code. Using skills tests to assess candidates’ proficiency with basic Python algorithms seems like a good start, right?

But as your finger hovers over the button to send a test to candidates, you hesitate. You’re putting a lot of trust in this test. What if someone cheats?

You’re not alone in having this concern. In fact, the idea that skills tests are easy pickings for cheaters is one of the biggest myths about skills-based hiring. But make no mistake – it is just a myth.

In this blog, we explain the impact of cheating in traditional and skills-based hiring, and how using anti-cheating protocols during skills-based hiring can drastically reduce your chances of making a mis-hire. We’ll also show you how to start building a virtually cheat-proof skills-based hiring process.

The impact of cheating in traditional hiring 

Before we discuss the impact that cheating has on skills-based hiring, let’s take a look at what cheating has traditionally looked like.

It’s no secret that people lie on their resumes. A 2020 survey of 400 jobseekers found that 78% had lied, or at least thought about lying, on their resume.[1]

In fact, it’s even uplifted as a smart tactic by many recruitment advice gurus. Candidates are frequently encouraged online to exaggerate many parts of their skills and experience, for instance:

• Their proficiency in key skill areas

• The level of responsibility they’ve been entrusted with in the past

• Their leadership role in key projects

Many employers now rely on resume screening software to not only save time but, in theory, bypass many of these challenges. However, in practice, the knowledge that their resumes aren’t even being read by a human may actually incentivize candidates to cheat.

Our 2022 State of Skills Based Hiring report found that 55.3% of candidates agreed that they have trouble standing out from the crowd with their resumes and 56.5% said they felt they didn’t pass screening due to a lack of relevant experience on their resumes.

To help themselves get noticed, some candidates choose to bend their past experiences and responsibilities to match the vocabulary of the job ad, or even hide keywords that aren’t relevant to their skills by matching the font to the background color of their resume.

This is known as “keyword stuffing”, and is designed to ensure that a resume is sent to the top of the pile by screening algorithms.[2]

Traditional hiring also leaves room for candidates to cheat at the interview stage. Websites like Glassdoor let candidates see questions that have been asked of interviewees in the past, so they can prepare in advance.

This is made even easier when hiring managers borrow from a list of common interview questions or brain-teasers online, so think twice before asking how many ping pong balls fit in a 747. (It’s 22,870,000, by the way.)

All of this cheating can lead to candidates underperforming once they’re hired and businesses having to deal with the costly business of replacing a bad hire.

But how does this differ in skills-based hiring?

What does cheating look like in skills-based hiring?

In skills-based hiring, the emphasis is removed from resumes as screening tools and put instead on pre-interview assessments: online tests that candidates can take to demonstrate their skills in action, instead of simply listing them.

The benefit of this is that it’s more efficient for both the candidate and the hiring manager and provides a basis of hard data when it comes to shortlisting and selecting applicants for interview.

However, candidates do often try to cheat the system, with methods such as:

• Sharing answers with other candidates

• Asking a friend to take the test for them

• Retaking the test multiple times

• Attempting to tamper with the results 

• Googling the correct answer, either before or during the test

• In situational judgment or personality questions, choosing the answer that they think most appeals to the employer, rather than the one that is most true to how they think or behave

Source

The knowledge that candidates attempt to use these methods has understandably left a dent in skills-based hiring’s reputation with employers. 

In our State of Skills Based Hiring report, we also found that 32.7% of employers still doubted the accuracy, validity, and fairness of skills-based assessments. After all, when you can find the answer to almost anything online, it’s no surprise that you might mistrust online skills tests.

However, there is a key distinction to be made between the two types of online skills tests (unproctored or proctored) and how each impacts cheating.

Let’s get into it.

Unproctored internet tests

An unproctored internet test is probably the first type of assessment that comes to mind when you think of an online skills test. 

In an unproctored test, you log on and take the test. There’s no supervision software in place to ensure that you don’t have other tabs open or that you’re the person who’s signed up for the evaluation; the whole thing is based on a system of trust.

Unsurprisingly, these are the tests most open to cheating. A 2017 study found that in an unproctored assessment, around 13.84% of test-takers were suspected of cheating.[3] 

However, while this figure is high enough that you can’t dismiss it out of hand, it’s still not the epidemic of cheating that you might expect, and 2018 research helps explain why. 

When individuals were instructed to cheat in an unproctored test, researchers found that although they were able to influence their scores, their level of success depended largely on their cognitive ability.

Those with less cognitive ability for the tests they were cheating on were less likely to be able to cheat well, so even though the study did find evidence that cheating was possible, researchers doubted that the results accurately depicted the real-world impact of cheating on test scores.

They also found that timing candidates had little effect on cheating outcomes due to their ability to search answers on the internet, so although many publications recommend this method as a way to minimize cheating, take that advice with a grain of salt.[4]

More up-to-date estimates put the rough percentage of cheaters in most tests even lower than these older studies. 

Research by coding challenge provider Codility suggests that only around 10-11% of test takers cheat across all industries, and that it’s lower in some sectors than others. For instance, media and entertainment only saw 3% of candidates cheat. Researchers also observed that rates of cheating were higher in societies with more wealth inequality.

That’s the story with unproctored tests. But in 2023, we now have complex anti-cheating measures in place, one of which is the use of digital proctoring.

Proctored internet tests

In proctored online tests, candidates are monitored while taking the exam. This monitor might be a real person on the other end of a webcam connection who can verify their identity and watch their screen while taking the test, or a software that performs a similar function.

Studies suggest that proctoring has a significant effect on cheating, with one concluding that while cheating must have been prevalent prior to proctoring, it is an effective tool to mitigate dishonesty in online academic courses. 

Proctoring has also been shown to have no impact on the performance of your honest test-takers.

So, what is the overall impact of cheating in skills-based hiring? 

Looking at the research, it’s pretty negligible. Even in unproctored tests, only around 10% of candidates cheat, and just because they’re cheating doesn’t mean they’re necessarily cheating well enough to still be shortlisted.

That leaves about 90% honest responses, even without sophisticated anti-cheating measures in place. 

But there are ways to reduce even this small margin for cheating.

5 best practices to minimize the impact of cheating on your skills-based hiring strategy

As we’ve mentioned, online proctoring enables you to reduce the already low chances that cheaters bypass the challenge presented by your pre-interview skills assessments.

Here are our five best practices for implementing anti-cheating protocols in your skills-based hiring process.

5 best practices to minimize cheating in skills-based hiring: Summary table

Want to get started with cheat-proofing your skills-based hiring process? Here’s a quick summary of our recommended best practices.

1. Communicate positively with candidates to reduce motivations for cheating 

Most hiring managers assume that candidates cheat simply because they don’t have the ability to stand out otherwise, but this isn’t always true. Candidates might also cheat because they want to save time on a process they see as being of greater benefit to the employer than to themselves. 

Think about what we’ve brought up regarding resumes. 

Candidates know that hiring managers are using software to streamline hiring. They might not necessarily believe that recruiters even look that closely at their scores; instead, they think they use them simply to create an automatic shortlist. They believe they deserve a place on that list, so why not game the system to ensure that outcome?

We know what you’re thinking here. Who cares why candidates cheat? It only matters that you know how to stop them.

Firstly, understanding cheaters’ motivations might help you consider how to deal with them should you identify one. If you’re feeling lenient, you might give them a temporary ban from applying to your organization – say, a year. If you uncover their dishonesty during an interview, you can discuss the incident with them and explain how it sets them back.

But even if you’re not interested in reforming a cheating candidate, understanding cheaters’ motivations can help you lower the chances of it occurring.

Don’t believe us? One study proved it: Researchers found that understanding what was motivating people to cheat on assessments helped reduce the occurrences of it. Although no system could mitigate all cheating in online exams, the most efficient strategy was to lower these motivations.

The best way to do this is with positive and proactive communication about the purpose of skills tests.

Explain to all candidates that the goal with the skills test is to find the right fit for the role and the company, and that you’ll be consulting the results throughout the hiring process. This helps them see that the test is a chance for them to prove not just that they can do the job, but that they align with the organization in a more holistic way.

You can also explain that pre-interview assessments reduce unconscious bias on the part of the hiring manager and give them a fair shot at the role. If they believe they are not being assessed fairly in traditional hiring, this lowers the likelihood of them cheating on your skills test. 

All of this creates a more positive candidate experience by showing that you respect applicants’ time and efforts in applying for the role. This not only reduces the motivations for cheating but also:

• Improves your employer brand 

• Reduces attrition during hiring 

• Sets a positive tone for the successful applicants’ tenure at your company

Source

2. Use multiple anti-cheating protocols in your skills tests

There’s no silver bullet that removes all possibility of cheating on skills tests, but by combining a range of anti-cheating protocols you can drastically reduce the risk.

The first protocol you should prioritize is using online proctoring software. This probably seems pretty obvious, and yet, it’s one of the pre-employment testing hacks that many recruiters still aren’t utilizing.

At TestGorilla, we use a specially-designed anti-cheating monitor to confirm the identities of our test-takers via webcam and monitor their activity during the test, just like a human proctor would do at an in-person examination.

We also prevent cheating on our tests by:

1. Using a subset of questions from a much larger question set for each test. These are then rotated from candidate to candidate to ensure no two tests are exactly alike.

2. Retiring each question after a certain number of uses. This ensures that the test stays fresh, even if one or more questions are leaked online.

3. Requiring a business address to sign up on our site. This prevents candidates from registering as employers to see our sample questions.

4. Keeping our full set of test questions hidden, even from our customers. This means that even if a candidate did slip through the net, they wouldn’t get away with the answers they were looking for.

Fortifying your screening test against manipulation with these protocols then enables you to make skills tests do more heavy lifting during your hiring process. 

3. When screening candidates, replace resumes and cover letters with proctored skills tests

It’s no secret that we think resumes are not a good screening tool. As we mentioned above, they’re extremely vulnerable to manipulation by candidates, and even when they contain honest information, they’re vulnerable to bias.

Don’t believe us? One study found that applicants with White-sounding names were almost 10% more likely than those with Black or African-American-sounding names to get a response from employers – even from the exact same applications.

Even if bias wasn’t a factor, resumes simply aren’t very good at what they aim to do. Work experience has been shown to be a poor indicator of how well candidates perform on the job, and it also doesn’t predict retention. And yet, resumes have historically been the tool that recruiters most rely on. 

Not anymore. 

Skills tests with strong anti-cheating protocols in place provide a more efficient alternative to resume evaluation during the screening process, as pre-employment screening tests ensure that you invite only the candidates with proven aptitude to be interviewed. 

You’re likely to be rewarded with better longevity among your hires (further evidence that cheating doesn’t make a significant impact on skills-based hiring), as in a study of 300,000 hires, employees who were hired with job testing stayed 15% longer in their positions than those hired without.[5]

4. Make it hard to share questions and answers – both in tests and structured interviews

As we’ve discussed briefly above, just as the internet can give you access to top-tier candidates, it also gives less capable candidates the ability to cheat. You can mitigate this by making it harder to share the questions you ask your candidates, both in your skills tests and in your interviews.

In skills tests, use a testing service with anti-cheating protocols like the ones we’ve outlined above, such as cycling a subset of questions from a much larger question pool, and retiring questions after they’ve been used a certain number of times.

This ensures that even if candidates do get their hands on previous test questions, the best they can do is use them to study, not to replicate an answer.

It’s also important that candidates cannot share questions and answers easily for the interview process, though the challenge here is greater. 

Admittedly, this is particularly true when using the structured interview techniques recommended by skills-based hiring, as unlike the traditional unstructured approach, the former requires that hiring managers ask all candidates:

1. The same questions

2. In the same order 

3. Using the same criteria, agreed beforehand by stakeholders, to judge the quality of the answers

One of the ways you can limit question and answer sharing for structured interviews is by always tailoring interview questions to the role at hand.

The best way to do this is to conduct a skills gap analysis prior to hiring, even when you’re hiring for a role you’ve hired for before. You can then fit your questions to the particular needs of your team without recycling questions, even if you’re picking questions from the same list.

Depending on the availability of your candidates, you can also conduct structured interviews as close together as possible, so that even if a proactive candidate shares their questions online directly afterwards, it has minimal impact on their fellow interviewees’ performance.

5. Use multiple methods to assess candidates 

Finally, the most effective way to reduce the impact of cheating on your skills-based hiring process is to carry insights from one stage in the hiring process through into the subsequent stages.

Rather than dropping the skills test results once the screening process is over, take a holistic approach to hiring by comparing notes at each stage. That means comparing insights you’ve gained from:

• Skills tests

• Work samples

• Structured interviews

• Technical interviews

• Unstructured, more qualitative interviews

• References and background checks

This helps you spot cheating faster and prevents you from making a mis-hire. 

For example, you might notice that a candidate did exceptionally well in their skills tests but performed below average in their technical interview. You could then further probe this disparity in the interview and determine whether nerves impacted their performance or if their skills test results were manipulated.

An over-reliance on any one tool leaves you open to a small chance of cheating; by combining tools, on the other hand, you cover more of your blind spots and give yourself the best chance of minimizing cheating.

Use TestGorilla to minimize cheating and ensure your next skills-based hire is the real deal

So, what’s the impact of cheating on skills-based hiring? Thanks to current anti-cheating protocols like online proctoring and monitoring software, very little – and certainly less than traditional hiring.

It’s one of the many reasons why skills-based hiring delivers better results than traditional hiring.

To get started, read our guide to best practices when applying skills based hiring in your organization.

Or, if you’re already on board, use our Culture Add test to find a candidate who truly aligns with your company values.

Sources

1. “Hiring Charlatans?”. (2020). Checkster. Retrieved March 03, 2023. https://www.checkster.com/are_you_hiring_charlatans 

2. Hu, James. “Are You Guilty of Resume Keyword Stuffing?”. (October 6, 2021). Jobscan. Retrieved March 03, 2023. https://www.jobscan.co/blog/resume-keyword-stuffing/ 

3. Aguado, David; Vidal, Alejandro; Diaz, Julio Oleo; Ponsoda, Vicente. “Cheating on Unproctored Internet Test Applications: An Analysis of a Verification Test in a Real Personnel Selection Context”. (December 2018). The Spanish Journal of Psychology. Retrieved March 03, 2023. https://www.researchgate.net/publication/329364614_Cheating_on_Unproctored_Internet_Test_Applications_An_Analysis_of_a_Verification_Test_in_a_Real_Personnel_Selection_Context 

4. Cavanaugh, Katelyn J. “Predicting Score Change: An Empirical Investigation of Cheating on Unproctored Employment Tests”. (2018). Old Dominion University Digital Commons. Retrieved March 03, 2023. https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1066&context=psychology_etds 

5. Hoffman, Mitchell; Kahn, Lisa B.; Li, Danielle. “Discretion in Hiring”. (November 2015). NBER. Retrieved March 03, 2023. https://www.nber.org/papers/w21709