AMC Overview

• See also: The CAT MCQ Examination Matrix

Computer Adaptive Format Examination
A Computer Adaptive Test (CAT) format of MCQ is a form of computer-based test administration in which each candidate takes a unique, customised examination.  Research over the past decades has confirmed the comparability between CATs and fixed length tests, but with CATs requiring less than half the number of questions to obtain more precise results. The development and refinement of Item Response Theory and Rasch models provided the theoretical structure and calibration data necessary for building item banks on a common scale.  Each test is created from a common pool of previously calibrated questions and is automatically equated to any other test drawn from that pool.  A pass/fail point (cut score or standard) is defined for the examination, and each candidate is measured against this point regardless of the group of candidates with whom they are examined, or the particular set of questions they are administered, or when they take the test.

Delivery of the AMC CAT format MCQ examination
In the CAT MCQ format examination, for each candidate the first question is randomly selected from the item pool.  If the answer to the question is incorrect, the next question chosen is a less difficult question; if correct, the next question is a more difficult question. This process is continued until the last question is reached. After each response, the computer calculates the ability estimate (“score”) based on all the information available and obtained to that point. The precision of the ability estimate improves as the process is continued because the questions are targeted to the latest ability estimate of each candidate.

This mode of measurement is much more efficient than a traditional examination where all candidates have to answer the same set of questions. Some of the questions in a traditional examination will be too easy or too difficult for some or all candidates sitting that examination. Such questions are not administered to candidates in CATs since items are selected from the pool of calibrated questions that match the ability (knowledge and understanding) of each individual candidate, which is the reason why a shorter examination can yield more precise results for all candidates.  Computer adaptive testing ensures that each candidate is given questions appropriate to his/her knowledge and understanding.

Review of items
In the AMC CAT MCQ computer-administered examination a candidate is provided with a ‘review’ facility on completion of the examination, if time permits.  A candidate cannot go back to a question and change a response during the examination (before all of the 150 questions have been attempted) as the computer uses all the current information to select the next question to administer.

However, once the examination has been completed and there is still examination time left, a candidate can go back to any question and change any response.  If a response is changed from correct to incorrect, or from incorrect to correct, the computer will recalculate the ability estimate.  In other words the final candidate ability (“score”) is calculated on the final response to each question.

Scoring of the examination
It is important to note that the number of correct answers is not used to calculate a candidate’s ability. Some candidates would have been administered difficult questions whilst others would have been administered easier questions – depending on the correct or incorrect answers to previous questions. Therefore, if candidates have the same number of correct answers, the ability of the candidate that answered the difficult questions will be higher than the ability of the candidate that answered the easier questions.  The AMC score is based on the candidate ability level and represented on the AMC scale.

Pilot items
Pilot items are non-scored items used to test and calibrate new questions, which may be used in future examinations. In addition to the 120 scored questions, each candidate will be administered 30 pilot items, which are integrated through the examination content. However, it is important to note that the final 20 items in the examination are scored items only, therefore candidates are encouraged to pace themselves during the examination [a timer-clock is included in the examination platform for each individual candidate]