Validating the retelling task of Test for English Majors Band 4 Oral in China: Evidence from a corpus-based exploration

Abstract

The retelling task is the first task in the Test for English Majors Band 4 Oral (TEM4-Oral), a nationwide English speaking test for undergraduate English major students in China. Despite its wide use, little work has been done on ensuring its validity. This is reflected in the absence of a systematic validation program or review since its launch in 1999 (Duan, 2011). In the case of the retelling task, the provision of a source story which candidates ‘retell’ raises concerns about the use of source material and how it is linked to language proficiency (e.g. Plakans & Gebril, 2012). This study examines the validity of the TEM4-Oral retelling task through analysing features of test-taker performance across four test administrations. Combining corpus analysis and qualitative explorations, it addresses how candidates’ proficiency levels are reflected in the discourse characteristics of their retellings. It also explores the consistency of scoring across task versions. Findings suggest significant differences in the discourse features of higher- and lower-ranked retellings, which partly supports the score interpretation from test-taker performance in the retelling task. On the other hand, some inconsistencies in candidates’ discourse features have been observed across administrations. This indicates that the generalisability of the task is threatened. This study constitutes an initial step in the validation process of TEM4-Oral. As well as shedding light on the design of the retelling task, it highlights the importance of test validation in the Chinese context, which would benefit thousands of people. Show more