Automated Essay Scoring Effect on Test Equating Errors in Mixed-format Test

Authors : Ibrahim Uysal, Nuri Doğan

Pages : 222-238

Doi:10.21449/ijate.815961

View : 18 | Download : 10

Publication Date : 2021-06-10

Article Type : Research

Abstract :Scoring constructed-response items can be highly difficult, time-consuming, and costly in practice. Improvements in computer technology have enabled automated scoring of constructed-response items. However, the application of automated scoring without an investigation of test equating can lead to serious problems. The goal of this study was to score the constructed-response items in mixed-format tests automatically with different test/training data rates and to investigate the indirect effect of these scores on test equating compared with human raters. Bidirectional long-short term memory (BLSTM) was selected as the automated scoring method for the best performance. During the test equating process, methods based on classical test theory and item response theory were utilized. In most of the equating methods, errors of the equating resulting from automated scoring were close to the errors occurring in equating processes conducted by human raters. It was concluded that automated scoring can be applied because it is convenient in terms of equating.
Keywords : Test equating, Automated scoring, Classical test theory, Item response theory, Mixed-format tests