How can valid, reliable employee selection tests minimize selection errors and increase selection hits?

Science series materials are brought to you by TestGorilla’s team of assessment experts: A group of IO psychology, data science, psychometricians, and IP development specialists with a deep understanding of the science behind skills-based hiring.

If you read our previous blog post about selection error, you’ll remember that we defined exactly what it is and explained that you can make better hiring decisions by reducing it. 

Here, we’re taking it one step further. In this post, we’ll expand on those points by showing you how using valid, reliable selection tests during your hiring process can reduce selection errors, ultimately leading to more successful hires and fewer mis-hires.

A quick reminder about selection hits and misses

Before we get started on the role that valid, reliable selection tests can play in minimizing selection error, here’s a quick reminder of what selection error, selection hits, and selection misses actually mean.

• Selection error, in the context of employment screening and hiring, refers to mistakes made during the selection process that lead to the hiring of unsuitable candidates or the rejection of suitable candidates (i.e. selection misses). 

• A selection miss occurs either when an employer hires an unsuitable candidate (this is known as a ‘false positive’) or rejects a suitable candidate (this is known as a ‘false negative’).

• A selection hit occurs either when an employer hires a suitable candidate (this is known as a ‘true positive’) or rejects an unsuitable candidate (this is known as a ‘true negative’). 

You can see a breakdown of the different types of selection hits and misses in the table below. In summary, selection error must be reduced if employers want to maximize hits and minimize misses during hiring: minimized selection error = better hiring decisions.

What makes a selection test valid and reliable? 

As the definitions above indicate, the goal of any hiring process is, ultimately, to land selection hits – to hire suitable candidates and reject unsuitable ones. Pre-employment tests come in handy here: By using valid and reliable selection tests, employers can actually increase their selection hits. 

Before we explain exactly how and why they can do this, we’ll explore exactly what we mean by the words “valid” and “reliable” in the context of selection tests. That way, you’ll understand why tests need to be valid and reliable in order to successfully minimize selection error during the hiring process.

Test reliability

Test reliability refers to the degree to which a selection test works consistently. In employee selection, this consistency takes various forms. For example, a test must measure a skill similarly, regardless of who is taking the test. 

Suppose you bought a new phone to make important business calls. Your new phone has valuable features and high sound quality. However, it randomly powers off while making calls about 15% of the time. While you may like the phone’s features, you cannot rely on it for business calls due to inconsistent performance. You need a reliable phone to make business calls. Similarly, you need reliable selection tests to make hiring decisions. 

An employee selection test, such as a skills-based hiring test, must work each time you use it. The degree to which a test works consistently refers to its reliability. You can find more information about reliability in our guide to psychometric properties of pre-employment tests.

Test validity

Test validity refers to the degree to which a selection test measures what it’s intended to measure and is associated with test accuracy. For example, a skills-based test should include test items that fully assess the target skill area and avoid assessing irrelevant characteristics of the candidate. 

In short, valid selection tests consistently assess what they were designed to measure. For more information about test validity and its different forms, read our brief introduction to test validation.

How do valid, reliable selection tests reduce selection error?

Valid, reliable selection tests can increase selection hits by improving the accuracy and effectiveness of the selection process. When organizations use them, they are more likely to hire candidates who are a good fit for the job and the organization. Outlined below are three ways that valid, reliable selection tests reduce selection error:

1. By ensuring candidates are evaluated on job-related criteria

Valid, reliable selection tests assess candidates objectively based on job-related criteria. Alignment between the selection test and the job role ensures that candidates who perform well on the selection test possess the necessary qualifications to do well on the job, increasing selection hits. 

2. By reducing bias and subjectivity

Valid, reliable selection tests focus on assessing job-related candidate characteristics and avoid assessing personal characteristics irrelevant to the job. Removing bias and subjectivity, valid, reliable selection tests are more objective, leading to better (and fairer) hiring decisions.

3. By predicting future job performance accurately

Valid, reliable selection tests predict future job performance accurately. When candidates perform well on these tests, there is a higher likelihood that they will excel in the actual job role because they’ve been evaluated on job-related criteria. 

Valid, reliable selection tests in action: 2 examples of how they can reduce selection error

Example 1: Employer uses an invalid, unreliable selection test

A manufacturing company called Space Electrics is hiring machine operators. As part of the hiring process, they administer a Machine Operator test that assesses candidates for skills using a certain machine – let’s call it Machine A1. 

The selection test is outdated, because employees at the company now use a different machine – Machine A2. The Machine Operator test includes test questions that focus on the now irrelevant processes of Machine A1, and fails to assess important operating processes of Machine A2. This means it’s not valid. Because the test is outdated, it also uses language that isn’t job-relevant, and that a subset of the candidate pool would be unfamiliar with. Since it has a systemic bias towards a subset of candidates, it is also unreliable.

Space Electrics used the invalid, unreliable selection test to assess 140 candidates. They saw the following results:

• 30 false positives: Thirty candidates who know how to operate Machine A1 but not Machine A2 were hired and performed poorly 

• 45 false negatives: Forty-five candidates who know how to operate Machine A2 were not hired where they could have performed well 

• 20 true positives and 45 true negatives, with 40% of those hired being successful hires (or true positive hits)

False positives and false negatives increased Space Electrics’ selection misses (75 candidates) and decreased its selection hits (65 candidates). 

Example 2: Employer uses a valid, reliable selection test

Space Electrics is hiring a second cohort of machine operators and has since become aware of the Machine Operator test’s issues. As part of the hiring process, they will now administer the Revised Machine Operator test that measures skills in using the more up-to-date machine – Machine A2 – and uses job-relevant language.  

Space Electrics used the valid, reliable selection test to assess 140 candidates. They saw the following results:

• 10 false positives: Ten candidates who know how to operate Machine AI but not Machine A2 were hired and performed poorly

• 15 false negatives: Fifteen candidates who know how to operate Machine A2 were not hired when they could have performed well

• 40 true positives and 75 true negatives, with 80% of those hired being successful hires (or true positive hits)

False positives and false negatives decreased from the first hiring phase, so Space Electrics’ selection misses decreased (25 candidates), and selection hits increased (115 candidates). 

In the examples above, using valid, reliable selection tests ultimately led to better hiring decisions, increasing the percentage of those hired who were successful hires (or true positive hits) from 40% to 80%. 

The business impact of such a change is hard to overestimate: Doubling the rate of successful hires means reducing turnover, minimizing training costs, and it ensures fewer future hiring rounds are necessary.

5 key organizational benefits of using valid, reliable selection tests

For employers, ensuring selection tests are valid and reliable leads to the following key benefits:

1. Reduced bias and subjectivity: Valid, reliable selection tests focus on assessing job-related candidate characteristics and avoid assessing personal characteristics irrelevant to the job. Removing bias and subjectivity, valid, reliable selection tests are more objective, leading to better hiring decisions.

2. Increased diversity: Valid, reliable selection tests reduce bias and discrimination in the hiring process, helping your organization to identify and hire diverse talent who will perform well and add to your organizational culture.

3. Improved culture fit: Valid, reliable culture add tests can lead to greater culture fit as you maximize your likelihood of selecting candidates who will add to and align with your organization’s values.

4. Reduced turnover rates: Turnover rates decrease because employees typically stay in roles that suit them. Hiring the right candidate from the start reduces the costs of recruiting, training, and onboarding new employees due to hiring misses.

5. Improved legal defensibility:  Valid, reliable tests help organizations ensure their hiring practices are legally defensible by reducing the risk of discrimination and legal challenges in the hiring process. 

Without monitoring validity and reliability, employers could miss out on these benefits. That’s why it’s important to a) get your tests from a provider who continuously monitors and improves them, and b), monitor their reliability and validity for the roles you’re using them to select candidates for.

Turn to TestGorilla for valid, reliable selection tests

In summary, valid, reliable selection tests increase selection hits by ensuring you evaluate candidates based on job-relevant criteria, reducing bias and subjectivity, and predicting future job performance accurately. They ultimately lead to better hiring decisions, increased selection hits, and improved organizational outcomes.

The benefits of using valid and reliable selections are manifold, and TestGorilla’s Assessment team works hard to ensure we create valid and reliable tests. We also conduct continuous monitoring and improvement of existing tests in our test library – check them out! Some of our newest tests include the Microsoft Project test, the Video Content Creator test, and the Pharmacy Technician test.

If you want to learn more about the science behind TestGorilla, we recommend reading: 

• Legal risk and skills-based hiring: The legal defensibility of TestGorilla’s tests

• A brief introduction to test validation

• How to prepare for a criterion-related validity study