Courtesy of
Certification Galore

Adaptive Testing

Certification and Skills Assessment Group

Microsoft, Inc.

Adaptive Testing Comparison with Fixed-Length Exams

Advantages and Disadvantages

Often individuals confuse the advantages of CATs with those of computerized testing in general. Computerized testing, whether adaptive or not, provides significant benefits to the test taker, the test publisher, and anyone else who uses the test results. These benefits are listed below:

Computerized Testing Benefits

bulletImmediate Scoring and Feedback. The most important benefit of computerized testing is to know immediately the result of the testing. Getting a score and a pass/fail decision right away for a certification candidate is very important.
bulletUnbiased Scoring. Computers score everyone the same way and do not consider factors irrelevant to the score, such as examinee gender or culture.
bulletAccurate Scoring. Computerized tests are much more accurate in scoring test results than are paper-and-pencil tests or oral exams.
bulletIncreased Efficiency. Research has shown that 50-question computerized test takes much less time to administer than a 50-question paper-and-pencil test.
bulletConvenient Individualized Administration. Computerized tests can be administered at times and locations more convenient to the test taker.
bulletImproved Test Security. Test results are more meaningful because test security is improved. Random ordering of test questions, adaptive testing, and other innovations expose test questions less often and make it impossible to copy someone else’s answers. New performance-based test questions are difficult (and sometimes unnecessary) to memorize for future test coaching.
bulletNew Question Types (point-and-click, drag and drop, and simulations). New types of questions improve the ability to measure the important skills.
bulletImproved Performance-Based Testing. With software simulations, for example, it is possible to require a certification candidate to demonstrate the job skill directly on the test.
bulletReduction in Answering Errors. Test takers make fewer extraneous errors answering computerized test questions than by filling out the small circles on answer forms for paper-and-pencil tests.
bulletUnobtrusively Pretesting Items. Computerized testing makes it easy for a test publisher to insert experimental and unscored questions into a test.
bulletEnhanced Motivation. While not well understood, it seem that taking a test on the computer is more interesting and less intimidating than taking the test on paper.
bulletLess Expensive. Although it is probably not true today, the increased use of computers in test development and delivery promises to reduce the testing costs in the future for the test developer, test publisher, test user and examinee.
bulletBetter Item Selection Routines (for example, Adaptive Testing). The computer allows for better test designs with more flexible item selection routines. CATs are examples the use of better item selection algorithms.

CATs provide additional advantages.

bulletIncreased Efficiency (that is, quicker, shorter tests). While computerized tests have proven to be more efficient, CATs offer even more efficiency. The data on many comparisons between CTs and CATs show that the CAT may save a test taker as much as 80% in test taking time.
bulletImproved Test Security. CATs improve security in several ways. First, CATs expose items at a much reduced rate, allowing the items to be effective for a longer period of time. Second, test coaching efforts that focus on individual items are less effective because it is not clear which items a person will be presented. Finally, coaching efforts that send in people to memorize the test questions are less effective because those individuals, usually unskilled in the test content, are not able to see the questions that are most effective for determining a person’s ability to pass the test.
bulletImproved Accuracy for Scores for High and Low Ability Test Takers. An adaptive test is as accurate as any other test at determining a pass/fail decision; however, for providing a score for high and low ability individuals, it is actually more accurate. Because it can present many items at any ability level, it can compute an accurate score. Traditional tests are less accurate at these extremes.
bulletMore Time Per Question Relative to Fixed-Length Exams. Compared to traditional exams and typical time limits set for them, experience has shown that CATs, even with their shorter time limits, provide more time per test question. For example, a typical 60-item test may have a 60-minute time limit, or 1 minute per question. A CAT version of the same test may have a maximum number of 25 items, but be given a 30 minutes to complete the test. Even at the maximum number of items, the CAT gives over 1 minute per item. And if the test ends at the minimum number of items, then 2 minutes per item is given. This is a significant advantage for every test taker, but is better for individuals whose native language is not that of the test. A few extra seconds per test can mean the difference between passing and failing.
bulletReduced Testing Costs. Being able to keep the testing times down reduces the test delivery costs. For some test publishers this means being able to reduce testing costs to test takers. Or at least it means that test costs may not rise for a longer period of time.
bulletEasier Test Revision. CATs allow easier revisions of the test. If a test question is not functioning well, it can be removed without a complete republishing of the test. Development and publishing costs are saved without affecting the integrity of the test or resulting scores.
bulletEquivalently Challenging Exams. CATs produce a similar psychological test taking experience for everyone. Each test taker answers questions that are personally challenging without being too hard or too easy. Boredom from answering many easy questions and frustration from answering too many hard questions can be avoided.
bulletPreferred Exam Design. Test takers prefer adaptive tests for many of the advantages described above. See Test Taker Reactions below.

Up Next