The interdependency between institutional resources, teachers'formation, and the validity and reliability of english L2 rubrics in Chile: a case study

Abstract

Within the field of L2, rubrics are evaluating tools that are largely used in testing procedures but have never been studied thoroughly. The influence of contexts in rubrics is another unknown territory in the area of language assessment. This investigation reports on a study of the factors that affect the quality of rubrics at the level of 10th grade (segundo medio) in Chile. The aim of our investigation is to observe the relationship between the formation of teachers regarding rubrics and the resources given by an educational institution, and how they affect the quality of the rubric in terms of construct validity and reliability. Data was collected by interviewing the teachers in charge of 10th grade from two schools of different socioeconomic context, including questions about the rubrics and about the factors we dealt with. We also analyzed two rubrics per school that were used by the interviewed teachers and evaluated them with a meta-rubric designed and used by the investigation team. It was found that both resources and teachers‟ formation had an interdependent relationship. Additionally, teachers‟ formation and their expert judgement seemed to affect directly the quality of the rubric, while resources in most cases affected indirectly the reliability and validity of the rubric. Time appears to be the most relevant resource as it limits the opportunities to improve the rubrics. It was also observed that teachers are critical in determining the quality of rubric design. The investigation reported could be expanded beyond the local context studied and include other stakeholders as to cover a national scope and enhance the quality of rubrics.