Courtesy of
Certification Galore

Adaptive Testing

Certification and Skills Assessment Group

Microsoft, Inc.

Comparison with Fixed-Length Exams

CATs are NOT more accurate at determining pass/fail certification decisions than fixed-length computerized exams—they are equally accurate. But they are more efficient. That is, the CAT determines the pass/fail decision with fewer test questions and less testing time. Over the past 20 years there have been numerous research results supporting this fact. Below are two of them, one of which is a comparison of certification tests in the IT industry.

The sixth grade mathematics test of the California Assessment Program (CAP) was administered in both CAT and computerized fixed-length (CT) versions. The results, shown in Figure 2, indicated that the average scores from the two tests were the same, while the time savings was 66% in favor of the CAT.

wpe2.jpg (6565 bytes)

Figure 2. A comparison of the scores and times between a CAT version and computer administered fixed-length version of a sixth grade mathematics test for the CAP. Notice that the scores are virtually identical, while the testing times are very different. The CAT took 66% less time than the CT.

Figure 3 shows a similar pattern for a comparison of a pair of computerized tests of network administration skills. In this exam, the time savings was even greater: 79%.

wpe3.jpg (6649 bytes)

Figure 3. A comparison of the scores and times between a CAT version and computer administered fixed-length version of a network administration exam. The scores are the same, while the testing times are very different. The CAT took 79% less time than the CT.