This Content Is Only For Subscribers
In our previous article, we discussed test analysis. We will now turn our attention to item analysis to gauge how each and every item (question) performed in the test. Note that items can be analysed only after they have been scored. The purpose of item analysis varies depending on your situation.
For the classroom teacher, item analysis is valuable in identifying specific areas of course content which need greater emphasis or clarity and what interventions to put in place in order that learners master the subject content. In the process, the teacher’s skills in test construction improves.
For the test developer, it helps in improving items which will be used again in later tests. Because of that, it is essential that when developing the test, piloting becomes an integral part of the process. High stakes tests are based on proven evidence because of their purpose which differs significantly from that of classroom testing.
Item analysis is a process of analysing individual test items (questions) in order to assess the quality of those items and of the test as a whole. The analysis can be easily done using excel without requiring any specialised software.
This is an important step in the test development cycle in addition to improving the quality of the test, it provides evidence of validity, which is the extent to which a test measures what it is supposed to measure It is one of the most common applications of psychometrics (the science of assessment).
The two most common analysis conducted during item analysis are the difficulty level and discrimination level.
Item difficulty is simply the percentage of students who answer an item correctly. In this case, it is also equal to the item mean. The item difficulty index ranges from 0 to 100; the higher the value, the easier the question. If 95 learners out of 100 get the item correct, then the difficulty level of the item is 95% or discrimination index is 0.95. That means the item is very easy, compared to one with the difficulty level of 35%. Item difficulty is relevant for determining whether students have learned the concept being tested.
Item discrimination as the name implies, is the item that discriminates between those who know from those who do not know. It may be referred to the ability of an item to differentiate among students on the basis of how well they know the material being tested. Because of that, item discrimination finds relevant application in final examinations which are purely conducted for selection or placement or examinations that employ norm-referenced grading, which compares learners by rank-ordering them. The higher the value of the item, the more discrimination between the learners. Its computation is a bit complicated and we will not go into that as it has little application in the classroom assessment.
Furthermore, in multiple test items, analysis can also be done on how each option functions. Remember multiple choice item is made of a stem and options. The stem refers to the portion of the item that presents a problem for the respondents (learners) to solve, while the options are the available alternatives to choose from. Options are made of a key and distractors. Distractors are incorrect options, whereas a key is a correct option.
In any given item, more examinees should not select any distractor than the key. If that happens, it means that smart students are selecting the wrong answer, and tells you that there is a problem with the item. In some cases, the item is just bad. In others, the answer is just incorrectly recorded, perhaps by a typo. This is called a miskey of the item. In both cases, the item is flagged for further investigation of the distractors’ statistics to figure out what is wrong.
Yes, It’s Possible!
