中文摘要：

本研究以概化理论为视角，搜集了新HSK五级模拟书写题的作答和评分数据，估算了试题、评卷员、评阅速度效应及其交互效应的方差分量，考察了五级书写成绩的可靠性。基于概化理论和规划求解的数据分析发现了题量的调整方案及题型、题量、评卷人数的可行组合方案。本研究对评阅速度的分析属于前沿性的理论探索，而其他数据分析结果，则可能有益于旨在改进该测试质量的决策实践。

英文摘要：

Writing test scores of the New HSK, similar to those of any other language proficiency tests, are most vulnerable to reliability criticism. The study collected writing samples （n = 89） of two mock tests of writing of the New HSKS. Data analysis was performed from the perspective of Generalizability Theory. Variance components were estimated for effects of items, raters, and rating speeds. Phi was also estimated for various settings of the test. Major findings are： （a） according to the current test setting, the descending order of the Phi coefficients for each item type is the ordering of the inner- sentence components, writing based on the keywords given, and writing based on the photo given; （b） to keep Phi at least . 8 for each item type, ordering items needs to increase to 20 while each of the other two needs to increase 2 and 3 items ; （c） with current allocation of item quantities for each item type, if calculation of the comprehensive score of writing uses weight propor- tional to the raw scores, then the Phi coefficient for the writing test could marginally reach the level of. 74. The study explored various approaches reaching a Phi coefficient at least . 85 with relatively lower costs （for details, please refer to section 3.3.2 of this paper）. To do this, the analysis applied solver functions of Microsoft Excel ; （d） the study did not find a significant effect of rating speed. How- ever, this conclusion was limited to the two different speeds investigated： each rater＇ s comfortable speed, and a speed under which each rater felt a little rush but still had confidence about his/her rating quality. Effect of rating speeds needs to be investigated with more rigorous designs. The authors also called for more attention to reliability issues of writing test.