What is predictor-criterion congruence and how can it help me hire successfully?

Science series materials are brought to you by TestGorilla’s team of assessment experts: A group of IO psychology, data science, psychometricians, and IP development specialists with a deep understanding of the science behind skills-based hiring.

In the dynamic world of hiring, finding the right candidate isn't just a stroke of luck – it's a science. 

While landing a top performer can feel magical, there is nothing elusive or mystical about the selection process that can help you consistently hire the best candidates. Having a  hiring process that’s relevant to the role you’re hiring for is crucial to identifying top performers. But job relevancy doesn't just happen by accident. So how, exactly, can you ensure it? 

In this article, we’ll explore one specific factor that is crucial to building a high-quality, job-relevant, and therefore successful hiring process: predictor-criterion congruence. If you want your hiring process to be job-relevant and predict on-the-job success, predictor-criterion congruence is the name of the game; and we’re here to help you understand and implement it.

What makes a good hiring process?

Before we dive into the good stuff, let’s get something straight. What makes a good hiring process?

Simply put, a good hiring process is one that enables you to hire future top performers. The process must be fair and unbiased in order to accomplish this fully: top performers come from all walks of life, and you don’t want to overlook great candidates because of a biased hiring process.

To give a more scientific answer, a good hiring process leads to selection hits – that is, it leads you to select and hire good performers, and to reject poor performers. 

What the science says

There are many scientific studies that look at the best approaches to designing hiring processes and implementing different hiring tools: for example, in 2022 Sackett and colleagues conducted a large meta-analysis that considers the criterion-related validity of different hiring tools. (Read more about the meta-analysis here if you’re interested.) 

Big studies like the study mentioned above consider selection in many different organizations, roles, and contexts. Although every hiring process has its own unique elements, researchers mostly focus on important general patterns, communicating additional nuance where they are able to. 

This means they can focus on the how of hiring – for example, is it better to give candidates a role-specific skills test or a general cognitive ability test? However, the what of hiring is equally, if not more, important.

If you opt to give your candidates pre-employment tests, you have to ask yourself a very obvious question: which knowledge, skills, abilities, and other characteristics should I test for? In other words, what do I want to measure?

By even asking yourself this question you are working towards a crucial feature of any good hiring process: job-relatedness.

Digging deeper into job-relatedness

Any criteria used during the hiring process must follow the rule of job-relatedness. This means that the criteria measured (i.e., knowledge, skills, abilities, and other characteristics) are related to performance and consistent with business necessity.

We all intuitively understand what job-related means. It is obvious that hiring insurance sales representatives based on their baking skills is not going to land you great salespeople, although you could end up enjoying some great fresh cookies in the office. Furthermore, this hiring process could be seen as unfairly discriminatory towards great salespeople with hobbies other than baking. 

In this example, the selection procedure is not likely to stand up in court: baking skills are completely unrelated to being a successful sales representative. Ensuring good congruence between the selected hiring tools (or criteria) and job requirements ensures that recommendations from influential hiring standards and guidelines are met. 

However, if we want to build a selection process that is not only legally defensible but also highly predictive, there is another type of congruence that we should consider: predictor-criterion congruence. 

Unpacking predictor-criterion congruence

In the hiring context, test scores are called “predictors” (e.g., interview scores, scores on a problem-solving test, scores on a work sample test), while “criteria” refer to scores on work-relevant behavior and/or work outcome that we are trying to predict (e.g., job performance ratings). 

Predictor-criterion congruence refers to the match between what is assessed (i.e., the knowledge, skill, or ability assessed by some hiring tool) and what is predicted (which aspects of performance are assessed). [1] This congruence is an important determinant of the level of validity obtained for your hiring process. 

Recent research found that structured interviews, job knowledge tests, and work samples tests have the highest mean validity among hiring tools. [2] These tools’ higher validity estimates are, at least in part, due to the fact that they tend to tap into the same knowledge, skills, or abilities that are measured in the performance criterion (i.e., they have greater predictor-criterion congruence). [1]

As an example, if you are hiring a manager who needs to manage competing demands, prioritize effectively, and make quick decisions, you can assess this during the hiring process with a structured interview by asking candidates to recall situations in which they demonstrated these behaviors. Similarly, you can tap into the performance criterion by giving candidates take-home assignments that simulate a work task and evaluating them using the same dimensions that you would use to evaluate actual work tasks.

How can I increase predictor-criterion congruence?

Job performance is multi-dimensional: it comprises both what you do (i.e. behaviors) and what you achieve (output or outcomes). For example, a sales manager’s performance could be evaluated based on their time management skills, communication skills, planning and follow-up, pipeline development, forecasting accuracy, application of Sales and CRM processes, number of deals closed, revenue generated, etc. 

Greater predictor-criterion congruence can be achieved by aligning predictors with specific performance facets rather than an overall performance rating. For example, we would expect scores on a time management test to be better at predicting sales managers’ time management effectiveness on the job than their overall job performance. Similarly, sales manager candidates’ scores on a SalesforceCRM test are likely to be a better predictor of their effectiveness in applying Sales and CRM processes than their overall job performance rating.  

This more nuanced approach enhances the validity of selection processes, as it allows for a more precise evaluation of how well a candidate's attributes and skills match the diverse and specific demands of the job.

In short, aligning your hiring tools to the important performance criteria of the job, increases predictor-criterion congruence and helps to ensure that success in the hiring process is the same as success in the role. 

4 ways to ensure job-relatedness and predictor-criterion congruence in your hiring process

Now that we’ve covered what criterion-related congruence is, here are 4 ways you can ensure it in your hiring process.

1. Job analysis, job analysis, job analysis

What’s the first rule of hiring? Ask any organizational psychologist and they will tell you: “job analysis, job analysis, job analysis”. Conducting a job analysis of the role you’re hiring for to identify your requirements and drivers of success is key to ensuring job-relatedness and predictor-criterion congruence. 

A great starting point for a job analysis can be one of the large frameworks of different roles and their tasks and requirements, like ESCO (the multilingual classification of European skills, competencies, and occupations) or O*NET (the occupational information network). Even if you’ve never hired for an accountant or auditor before, O*NET can give you a good overview of the role and what to look for in candidates. TestGorilla’s resource centre also provides tips on the key skills and competencies for multiple roles. 

If this is beyond the scope of the time and resources you can devote, consider talking with the future manager and teammates of the new hire. They are often experts in knowing the requirements of the role you’re hiring for and can help you understand it deeper than what is mentioned in the job description. Moreover, you can engage with top performers and see firsthand what knowledge, skills, and abilities are needed for success in that role.

Taking the time to understand the role and what skills are needed for the new hire to succeed can greatly help you craft a hiring process that will be able to select for top performers.

2. Select hiring tools in line with the requirements of the job

Recognizing different important elements of the role, its requirements, and its prerequisites for success, is only one part of the hiring process. Once you have built a good understanding of the role requirements, you should select hiring tools that maximize your likelihood of hiring top candidates. 

Consider tools that have high validity coefficients, such as structured interviews or job knowledge and skills tests. Make sure to adapt them to suit your hiring setting, and avoid overlap between different hiring tools.

With a library of 350+ tests (and counting!), TestGorilla can help you decide on the right hiring tool for your process – we have assessment templates that are pre-filled with potentially relevant tests for many roles. On the odd chance that you’re looking to evaluate skill, knowledge, or even previous experience that is not covered in our library, our platform allows you to set up custom questions or build your own custom tests.

3. Consider any assumed job requirements and state them explicitly

Often roles come with requirements that are not explicitly evaluated in a performance review but are essential to accomplishing everyday tasks. A common assumed prerequisite in international organizations, for example, might be good command of English or other regional languages. 

Software skills and knowledge of how to use tools that are commonplace in your organization are another example: these might be essential for success without ever being explicitly mentioned or evaluated. There are also organization-wide skills or competencies that might be a business necessity – a fast-growing tech company could argue that innovation skills are an implicit requirement for all roles, but if they fail to mention this in their job descriptions, then candidates won’t know this. 

Taking the time to recognize and examine the assumptions about what future hires should be good at is certain to help build a more legally defensible and effective hiring process.

4. Consider how performance is evaluated in the role

Performance evaluations come in many different forms and flavors. Take the time to understand how this process is conducted in your organization and the role that you’re hiring for. 

Ensuring congruence between your hiring tools and the performance evaluations could mean the difference between missing out on or hiring a top performer. This is because greater alignment between hiring tools and performance criteria results in greater validity and effectiveness [1]. 

The effectiveness of a hiring tool is established through validation studies. These studies look at the strength of the relationship between test scores and job performance ratings. Read our blog about how to prepare for a criterion-related validity study for more information about validation studies at TestGorilla.

Ensure predictor-criterion congruence using TestGorilla’s pre-employment tests

A high-quality hiring process is one that enables you to select and hire top performers. One of the hallmarks of a high-quality hiring process is that it is effective, fair and unbiased, allowing you to identify future top performers. 

To build such a process, it is necessary to ensure the tools and practices you use are related to the job and to job performance: in other words, to ensure job-relatedness and predictor-criterion congruence.  

If you understand the requirements and work-relevant outcomes or “criteria” that you want to predict in your hiring process, and you’re keen to try skills tests as your job-relevant “predictors”, try TestGorilla for free today or book a free demo with us to find out more about our platform.

Sources

1. Hough, L., & Oswald, F. (2023). Revisiting predictor–criterion construct congruence: Implications for designing personnel selection systems. Industrial and Organizational Psychology, 16(3), 307-312. https://doi.org/10.1017/iop.2023.35 

2. Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040. https://doi.org/10.1037/apl0000994