Anatomy of composition : how are Japanese compositions evaluated?
Date
2014
Authors
Imaki, Jun
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This study aims to clarify the mechanisms for evaluating Japanese compositions written by non-native learners. In this paper, I discuss the effect of the background of raters, and the marking order and the carryover effect for L2 Japanese compositions. In the first part of this study, 29 raters are grouped into three categories: native Japanese speakers with Japanese teaching experience, non-native speakers with Japanese teaching experience, and native speakers without a teaching background. I conducted evaluation and comparisons of several different pairs in this study including native and non-native Japanese teachers, by gender, and by people with and without a Japanese teaching background. These raters were asked to evaluate ten compositions written by non-native speakers in terms of five categories; overall; accuracy; structure and form; content; and richness. I discovered a difference amongst the groups in terms of some raters being more lenient was discovered from this experiment. The results revealed that non-native Japanese teachers are more generous with learners' errors and gave higher marks for all five evaluation categories. When comparing Japanese language teachers and people without a teaching background, their overall evaluation did not show any statistically significant differences even though teachers tended to give higher scores across the other four categories. In the second part of the study, I discovered the effect of the order of evaluation to be influential upon rater decisions. The compositions which were evaluated first and sixth out of the ten compositions tended to receive higher marks. The carryover effect is found to be quite influential as well. Compositions evaluated directly after a good composition tended to receive lower marks and compositions evaluated directly after a bad composition tended to receive higher marks. However, not only does the rating of the previous composition affect the following compositions' score, the countable features of the composition such as the proportion of kanji characters and the composition's length, might also influence the score of the composition which is evaluated directly after it. These findings suggest that, in order for there to be fair evaluations, students' compositions should not be marked in a particular order such as alphabetical order. This solid, reliable information on why raters give the marks they do will help build a firm foundation for the further development of rater training.
Description
Keywords
Citation
Collections
Source
Type
Thesis (MPhil)
Book Title
Entity type
Access Statement
Open Access
License Rights
Restricted until
Downloads
File
Description