La evaluación ha cambiado”: a case study on construct and face validity in the transition from face-to-face to computer-assisted L2 assessment

Abstract

As a response to the global pandemic caused by COVID-19, Chile was forced into a transition from face-to-face format to online computer-based tests to measure L2 proficiency. In our programme, Licenciatura en Lingüística y Literatura Inglesa this was echoed in a new format for the L2 proficiency assessment called Oral Project. By using a matrix of dimensions describing Computer Assisted Language Testing (CALT) proposed by Suvorov & Hegelheimer (2013), we observed the changes in construct validity of an English L2 oral proficiency test produced by the transition from a face-to-face to an online test. We elaborated a survey based on the same dimensions to see if face validity had also been affected. The survey was applied to 24 undergraduate students of an English Language and Literature programme in Chile who had taken the two versions of the test. Results indicated that the change of format affected construct validity in several dimensions. However, face validity was not affected as students, even when identifying the changes of the test, still believed that it was measuring the same construct. These results suggest that the relationship between construct and face validity is complex and that the main construct and test purpose are strong factors when establishing face validity.