📄 SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check

Summary

Chinese Spelling Check (CSC) means finding and fixing misused characters in Chinese text. Many such errors involve characters that sound or look alike but differ completely in meaning. Earlier systems used confusion sets only to narrow down candidate replacements. They did not model how similar characters relate to one another. The authors propose SpellGCN, which builds two graphs over characters from an open-source confusion set (Wu et al., 2013): one for pronunciation and one for shape. A lightweight GCN with no nonlinearity runs over these graphs, starting from BERT character embeddings. Attention merges the outputs of the two graphs. The resulting vectors serve as character classifiers applied to BERT's contextual representations, so the whole system trains end to end. On three human-annotated benchmarks (SIGHAN 2013, 2014 and 2015), the authors report clear gains over BERT alone and over previous models. Case studies show that SpellGCN picks corrections matching the phonetic or visual form, whereas plain BERT often picks fitting but unrelated characters. The approach shows how symbolic similarity knowledge can be built into a pretrained model's output space.

Key points

• Motivation: earlier work (Liu et al., 2010) estimates that roughly 83% of Chinese spelling errors involve phonological similarity and 48% involve visual similarity.
• The graphs cover up to about 5K characters. Adding a ReLU after the graph convolution hurt performance, so it was dropped to stay in BERT's embedding space.
• Ablation on SIGHAN 2013 (10K training samples, character-level C-F) compared ways to combine the two graphs:
  • baseline without SpellGCN: 67.0
  • sum pooling: 66.3
  • mean pooling: 67.5
  • attentive combination with β=3 (best): 68.2
• Performance improves with up to 3 SpellGCN layers and declines with more, which the authors attribute to over-smoothing.
• False positive rate is slightly lower than BERT's: 14.1% vs 15.3% on SIGHAN 14, and 13.2% vs 13.6% on SIGHAN 15. Code and data are released at github.com/ACL2020SpellGCN/SpellGCN.