TestGorilla LogoTestGorilla Logo
Pricing
homeblogsHiring & recruiting
What is internal consistency and how can you use it to improve your skills assessments?

What is internal consistency and how can you use it to improve your skills assessments?

November 8, 2024
internal consistency featured image
Share

What is internal consistency?

Internal consistency is a term that describes how well the different parts of something align or fit together. The term is often used in the context of organizations and testing. 

Internal consistency within an organization could refer to how well its code of conduct, corporate policies, and leaders’ behaviors line up and reinforce one another. 

In the context of testing and assessments, internal consistency refers to how well test items or sets of test items measure what they’re supposed to. For instance, questions in a communication skills test intended to measure active listening skills should all reliably measure active listening skills. 

Tests with weak internal consistency have questions that don’t correlate well and may measure different aspects. The result? Test-taker's scores may vary significantly across these questions, even though they’re meant to measure the same skill.  

Meanwhile, tests with strong internal consistency have strong correlations between items – and these usually give more consistent results. In classical test theory, this consistency signals reliability; the theory states that in reliable tests, a candidate’s “true score” for a skill (like reading comprehension) shows up consistently across similar questions.

Examples of internal consistency in assessments

To see internal consistency in action, check out the examples table below.

Example test 

Questions 

Level of internal consistency 

Reason

A customer service skills test

How comfortable are you approaching new customers?

How well do you work in a team? 

❌ Internal consistency is weak

One question measures a customer service skill, and the other measures teamwork.

A reading comprehension test 

What is the main idea of the passage?

How does the author support their argument in the passage?

✅ Internal consistency is strong

Both questions reliably measure reading comprehension.

An admin skills test

How often do you double-check documents for errors before submitting them?

How frequently do you review emails for typos and formatting mistakes before sending them?

⚠️ Internal consistency is too strong

Both questions essentially ask the same thing, providing no new insights.

The importance of internal consistency

Internal consistency is important because it helps ensure all parts of a process or system work harmoniously toward the same goal. In assessments, when all test items measure the same skill, the results are more reliable and useful.

How internal consistency leads to better hiring and decision-making

Internal consistency does three critical things for your company’s hiring process:

How does intenal consistency lead to better hiring and decision making graphic

1. Reduces hiring errors

Inconsistent test results lead to confusion during hiring and hiring errors. Weak internal consistency could show a candidate is a strong fit based on one set of questions while flagging concerns in another. 

For example, if one section of a leadership skills test indicates leadership potential, but another section contradicts it, the inconsistency can confuse you. You could dismiss a strong candidate who didn’t perform well because the results obscure their fit for the role. Alternatively, you could hire someone who doesn’t have what it takes to succeed in the role. 

Reliable testing creates confidence in recruiting and minimizes poor hiring decisions.

2. Supports talent development

Internal consistency is also vital when using assessments for talent management and development – whether you're evaluating new hires or tracking current employees’ growth and progress. 

This is because test discrepancies and unreliable tests can cloud results. Say you’re testing your employees’ skills with a specific software before adopting the software across the company. If your tests aren’t reliable, you won’t know the best way to train employees to use the software. 

In contrast, high internal consistency reliability in assessments allows talent teams to gain meaningful insights, identify areas for improvement, create effective learning programs, and ensure people are developing the right skills.

3. Gives you trust in test results

Whether you’re hiring for technical skills, assessing cultural fit, or evaluating employee growth, you need test results you can count on. Internal consistency is one psychometric property that helps guarantee reliable test results. 

The bonus? Candidates can also rest assured that you’re testing them fairly and accurately. 

Ways to measure internal consistency

There are several ways to evaluate internal consistency reliability, and using them together is a good idea for creating and maintaining internal consistency.

ways to measure internal consistency graphic

1. Cronbach’s alpha

Described by Lee Cronbach in 1951 as the ”coefficient alpha,” Cronbach’s alpha is a formula used in statistical tests that helps pin down how closely related different questions are as a group. This formula can look at all the items in the test or specific “subscales” or sets of test items. Scores run from 0 to 1. 

As a rule of thumb, you want your tests to have a high Cronbach’s alpha of around 0.70. This indicates strong internal consistency. A weak Cronbach’s alpha would be 0.50 or less. 

However, you don’t want too high internal consistency, such as a Cronbach’s alpha of 0.90. That means some questions might be redundant and will offer little additional insight.

2. Split-half reliability

Split-half reliability divides a test or set of similar test items into two halves and compares the scores from each. If the scores are similar, the test might have strong internal consistency. 

This method helps ensure that different test parts contribute equally to the overall result and there’s no data skew from each half.

3. Item response theory (IRT)

Item response theory (IRT) is a way to evaluate the performance of individual questions on a test. Instead of assuming each question is equally valuable, IRT looks at questions’ difficulty, discrimination (how well the question separates high and low performers), and pseudo-guessing parameter (how likely it is that a test-taker will answer correctly by guessing).

IRT is particularly useful for tests with varying difficulty levels, like skills testing. If you use IRT in a structured way along with the two methods above, you can improve internal consistency in your assessments.

To learn more about how TestGorilla uses internal consistency, visit our science page. You can also visit our blog to learn about various assessment-related topics. 

FAQs

How do I know if a test has strong internal consistency?

You can evaluate a test’s internal consistency by looking at its Cronbach's alpha score, which should be above 0.70 for reliable assessments.

What does a low internal consistency mean?

If an assessment has low internal consistency, it contains some questions that don’t measure what the overall test – or set of related test items – is designed to measure. For instance, if a test is supposed to measure business ethics but some of its questions focus on unrelated topics (like marketing strategy), the test will have a lower internal consistency.

Share

You've scrolled this far

Why not try TestGorilla for free, and see what happens when you put skills first.

The best insights on HR and recruitment, delivered to your inbox.

Biweekly updates. No spam. Unsubscribe any time.

TestGorilla Logo

Skills tests to hire the best

Our screening tests identify the best candidates and make your hiring decisions faster, easier, and bias-free.

Free resources

Skills-based hiring handbook cover image
Ebook
The skills-based hiring handbook

This handbook provides actionable insights, use cases, data, and tools to help you implement skills-based hiring for optimal success

Ebook
How to elevate employee onboarding

A comprehensive guide packed with detailed strategies, timelines, and best practices — to help you build a seamless onboarding plan.

Top talent assessment platforms comparison guide - carousel image
Ebook
Top talent assessment platforms: A detailed guide

A comprehensive guide with in-depth comparisons, key features, and pricing details to help you choose the best talent assessment platform.

The blueprint for boosting your recruitment ROI cover image
Ebook
The blueprint for boosting your recruitment ROI

This in-depth guide includes tools, metrics, and a step-by-step plan for tracking and boosting your recruitment ROI.

Skills-based hiring checklist cover image
Checklist
The skills-based hiring checklist

A step-by-step blueprint that will help you maximize the benefits of skills-based hiring from faster time-to-hire to improved employee retention.

Onboarding email templates cover image
Checklist
Essential onboarding email templates

With our onboarding email templates, you'll reduce first-day jitters, boost confidence, and create a seamless experience for your new hires.

HR cheat sheet cover image
Checklist
The HR cheat sheet

Get all the essentials of HR in one place! This cheat sheet covers KPIs, roles, talent acquisition, compliance, performance management, and more to boost your HR expertise.

Employee onboarding checklist cover
Checklist
Employee onboarding checklist

Onboarding employees can be a challenge. This checklist provides detailed best practices broken down by days, weeks, and months after joining.

Key hiring metrics cheat sheet cover image
Checklist
Key hiring metrics cheat sheet

Track all the critical calculations that contribute to your recruitment process and find out how to optimize them with this cheat sheet.