Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing

Authors : Melek Gülşah Şahin

Pages : 191-206

Doi:10.21449/ijate.676947

View : 23 | Download : 11

Publication Date : 2020-06-13

Article Type : Research

Abstract :Computer Adaptive Multistage Testing (ca-MST), which take the advantage of computer technology and adaptive test form, are widely used, and are now a popular issue of assessment and evaluation. This study aims at analyzing the effect of different panel designs, module lengths, and different sequence of a parameter value across stages and change in b parameter range on measurement precision in ca-MST implementations. The study has been carried out as a simulation. MSTGen simulation software tool was used for that purpose. 5000 simulees derived from normal distribution (N (0,1)) were simulated. 60 different conditions (two panel designs (1-3-3; 1-2-2), three module lengths (10-15-20), 5 different a parameter sequences ("0.8; 0.8; 0.8” - "1.4; 0.8; 0.8”-"0.8;1.4; 0.8” - "0.8; 0.8;1,4” - "1.4; 1,4; 1.4”) and two b parameter difference (small; large) conditions) were taken into consideration during analysis. Correlation, RMSE and AAD values of conditions were calculated. Conditional RMSE values corresponding to each ability level are given in a graph. Dissimilar to other studies in the literature, this study examines b parameter difference condition in three-stage tests and its interaction with a parameter sequence. Study results show that measurement precision increases as the number and length of the modules increase. Errors in measurement decrease as item discrimination values increase in all stages. Including items with a high value of item discrimination in the second or last stage contributes to measurement precision. In extreme ability levels, large difficulty difference condition produces lower error values when compared to small difficulty difference condition.
Keywords : ca-MST, Panel design, Module length, Item discrimination sequence, Difficulty difference condition