How TestGorilla continuously improves the quality of its tests

Our primary goal is to help you make better hires. That’s why we’re continuously improving our skills tests and adding new features (e.g., the ability to integrate with Greenhouse ATS) to create a more efficient hiring process. 

To test your candidates’ skills, you can create your own tests or you can use tests from TestGorilla’s test library. There are plenty of tests to choose from, and we’re constantly adding more.

But how do we make sure our tests are the absolute best?

What are the criteria for a good test?

First of all, what does it mean for a test to be “good”? If the goal is to make better hiring decisions, then a good test will be predictive of job performance. So to start with, our skills tests are developed by industry experts (we call them testpreneurs). By relying on specialists in each field, we are able to start with high-quality tests that will determine a candidate’s skill in a particular area.

There are a few other criteria we use to define what a good test is, and our test creators implement those right from the beginning when creating new tests. Here are a few things we look for in our test questions:

• They should be straightforward and easy to understand.

• They should cover each skill area appropriately and completely.

• They should be up-to-date.

• They should provide all relevant information.

• The question and each answer choice should be clear.

In addition to that, we want to make sure the tests are not too easy or too difficult as a whole, and that they don’t contain any misleading information. This is why, once a test is added to our library, we track candidates’ performance on each test.

We check the success rates of different questions and compare them, we track the amount of time needed to answer each question and to do the whole test, and in this way, we detect potential issues. After that, in order to be able to improve their tests, our testpreneurs get advice and ideas based on our data on applicants’ performance. 

How do we improve the tests in our test library?

Once a certain number of candidates have taken a given test (usually a few hundred), we give our test creators specific recommendations on how to improve the test further. 

The recommendations we give can be either for:

• Specific questions, or

• The test as a whole.

There are a few goals we’re aiming for at that stage:

Improve the questions’ clarity and formulation

Based on how applicants are performing, and whether there are questions that take them much longer or much shorter to answer, we make estimations on the clarity and difficulty of each question.

To be able to attract the right talent, your candidates need to have a smooth experience. For this reason, if some questions don’t make sense to highly-skilled professionals (i.e. those who answer all or most other questions correctly), we ask our testpreneurs to check them and reformulate, if necessary.

When they do this, they need to make sure the questions aren’t confusing or unclear, on one hand, and also that the questions aren’t too easy or too difficult, compared to the rest. 

Keep the questions fresh to protect test validity

To protect the integrity of our tests and avoid having the answers leaked online, we limit the number of times a given question is displayed to candidates. In practice, this means that we’re asking test creators to create sets of questions that we can then rotate: for example, to display a test of 20 questions, we actually have a set of 100 questions, and alternate questions each time a new candidate sees it. 

In short, testpreneurs regularly update the content of their tests, add new questions, and remove the ones that candidates have seen too many times. 

Make sure each question adequately tests the relevant skill

Each test is designed for a specific skill and needs to test it adequately. If candidates score well on all other questions except for one, for example, there’s probably a problem with the question. The formulation might be confusing, there might be a problem with the scoring key, the question might be too difficult, and so on. If this is the case, testpreneurs check it and update it—or write a new question. 

In addition to that, each test needs to feature questions that are sufficiently varied, in order to test all aspects of the given skill. If some of the tests are too narrow or too broad, we aim to adjust their scope.

Questions need to cover the relevant topics well: if a given question is not related to the topic of the test, this means that it’s not aligned to the test’s purpose. In this case, the testpreneur responsible for this specific test changes or removes it. 

Ensure a positive candidate experience

Organizations need to guarantee a positive candidate experience in order to attract the best talent. This is based, first of all, on clear and efficient communication with your applicants, but it also means that all elements of the hiring process need to be well-designed and well-aligned to the process as a whole—including skills tests.

Because of this, we’re aiming to always finetune the balance between testing for specific skills and guaranteeing a positive candidate experience. We aim to make sure that qualified candidates can finish the test in time, that there aren’t any discrepancies between the questions asked and the skills that you’re testing for, and that the test, as a whole, fits well into your hiring process. 

Make sure you have the best tests available

Finding and hiring the best candidate for a given position is a long and complicated multi-step process. You need to advertise on the right platforms to attract the best applicants, screen them, interview them, and figure out who has the right skill set for this exact role.

Skills tests take the guesswork out of this process, and we work to improve and update our tests continuously, in order to help you make great hiring decisions easily. We update the content of tests to prevent leaking, regularly improve the questions’ and tests’ quality, and frequently add new tests to our library.