:: دانشگاه خوارزمی برای مشاهده سایت قبلی اینجا را کلیک کنید Kharazmi University ::
   [Home ] [Archive]   [ فارسی ]  
:: Main :: About :: Current Issue :: Archive :: Search :: Submit :: Contact ::
:: Volume 16, Number 1 (3-2013) ::
IJAL 2013, 16(1): 25-57 Back to browse issues page
Using an Analytic Dichotomous Evaluation Checklist to Increase Inter- and Intra-rater Reliability of EFL Writing Evaluation
Masoumeh Ahmadi Shirazi
Abstract:   (2018 Views)
The present study reports the processes of development and use of an Analytic Dichotomous Evaluation Checklist (ADEC) which aims at enhancing both inter- and intra-rater reliability of writing evaluation. The ADEC consists of a total of 68 items that comprises five subscales of content, organization, grammar, vocabulary, and mechanics. Eight raters assessed the writing performance of 20 Iranian EFL students using the ADEC. Also, the raters were asked to rate the same sample of essays holistically based on Test of Written English (TWE) scale. To examine the inter-rater and intra-rater reliability of the ADEC, multiple approaches were employed including correlation coefficient, the dichotomous Rasch Model, and many-faceted Rasch measurement (MFRM). The findings of the study confirmed that the ADEC introduces higher reliability into scoring procedure compared with holistic scoring. Future research with greater number of raters and examinees may provide robust evidence to use analytic scale rather than holistic one.   
Keywords: Analytic scoring, Inter-rater reliability, Intra-rater reliability, EFL writing evaluation
Full-Text [PDF 264 kb]   (1330 Downloads)    
Type of Study: Research |
Add your comments about this article
Your username or email:

Write the security code in the box >



XML   Persian Abstract   Print


Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Ahmadi Shirazi M. Using an Analytic Dichotomous Evaluation Checklist to Increase Inter- and Intra-rater Reliability of EFL Writing Evaluation. IJAL. 2013; 16 (1) :25-57
URL: http://ijal.khu.ac.ir/article-1-1587-en.html
Volume 16, Number 1 (3-2013) Back to browse issues page
Iranian Journal of Applied Linguistics
Persian site map - English site map - Created in 0.138 seconds with 819 queries by yektaweb 3460