基于神经网络模型的汉语文本难度分级

Summary

The paper addresses automatic readability assessment for Chinese: given a text, predict the school level of readers it suits. Chinese lacks a widely adopted grading scale like Lexile, and English readability formulas applied to Chinese reached only about 26% accuracy. The authors build a hierarchical neural classifier. A CNN stage first extracts n-gram features. It uses a variable-length convolutional layer, where kernel size and stride grow along the sequence to cope with very uneven text lengths, followed by downsampling blocks. A BiLSTM with attention then produces a document representation. Training uses cross-entropy with label smoothing, since neighbouring difficulty levels overlap. The data come from primary and secondary textbooks of more than ten publishers, with poetry and classical Chinese removed. The authors also built a separate expert-labelled test set. On a five-level scheme based on school stages, the full model achieves 75.4% overall accuracy. This beats an SVM with rich handcrafted features as well as TextCNN, TextRNN, FastText, Transformer and BERT baselines. The work shows that tailored neural architectures can grade Chinese texts close to human-expert quality.

Key points

• Data and splits. The corpus combines textbooks from BNU, PEP, Jiangsu Education, the unified national edition and other publishers, split 8:1:1 into train, dev and test.
  • The five-level scheme follows the curriculum-standard stages: grades 1–2, 3–4, 5–6, 7–9, and senior high.
  • The eight-level scheme uses primary grades 1–6, plus junior high and senior high as single classes.
• Expert test set. It contains 200 articles labelled double-blind by three primary-school teachers, with Kendall's coefficient of concordance of 0.946.
• Five-level results. The full model (VBCNN+BiLSTM+Att+LS) reaches:
  • 0.741 accuracy on the textbook test set, versus 0.669 for the SVM;
  • 0.775 on the expert set;
  • 0.990 overall adjacent (±1) accuracy;
  • for comparison, BERT scored 0.692 overall.
• Eight-level results. These are much lower, at roughly 0.56–0.57 overall accuracy. Middle primary grades (4–6) are the hardest to separate.
• Comparison with human experts. The model's agreement with the experts is close to that of expert R3, at 0.775 versus 0.790 accuracy. Its Spearman correlations with the experts range from 0.815 to 0.868 (p<0.01).
• Released tool. A free Chinese graded-reading website built on the model was released.