📄 A Phrase-Based Statistical Model for SMS Text Normalization

Summary

SMS messages are full of ad-hoc short-forms, phonetic spellings, dropped words and missing punctuation, so off-the-shelf machine translation handles them poorly. Rather than retuning each MT system for this style, the authors clean up English SMS text into standard English first and leave the MT engine unchanged. They frame this normalization as monolingual translation from "SMS language" to English. Their system pairs a bigram language model with a phrase-based lexical mapping model, which allows multi-word to multi-word mappings and a null word for deletions. They built a parallel corpus of 5000 raw messages with manual normalizations. In 5-fold cross validation, the system reaches 0.8070 BLEU, compared with a baseline of about 0.70. Using it as a preprocessing step for English-to-Chinese translation of 402 messages nearly doubles average BLEU, from 0.1926 to 0.3770. This shows that a separate normalization module can adapt general MT systems to noisy text without modifying them.

Key points

• Data: The source collection has 55,000 English SMS messages from a chat room and from university students' correspondence. The 5000-message parallel corpus (raw text plus manual normalization) was used for training and testing.

• Corpus analysis: In a sample of 700 messages, 621 (88.71%) needed changes, totaling 2300 edits. Substitutions made up 86.43% of the edits, insertions 8.09% and deletions 5.48%.

• Normalization results: The phrase-based model averaged 0.8070 BLEU (3-gram) across 5 folds, with individual folds ranging from 0.7908 to 0.8236. The paper reports the baseline as 0.6958 in some places and 0.6985 in others. The learning curve levels off at around 3000 training messages.

• MT impact (402 messages, two references, 3-gram BLEU):

  Input	Systran	I2R	Average
  Raw messages	0.1219	0.2633	0.1926
  Dictionary lookup	0.1690	0.3485	0.2588
  Proposed normalization	0.3116	0.4423	0.3770

• Limitations and future work: The system does not yet restore missing punctuation. The authors plan to use pronunciation information to handle OOV words caused by phonetic spelling.

Interesting ideas

• I1. Normalize before translating, and treat MT as a black box. Separating noisy-text handling from the downstream system means one module can serve MT, text-to-speech, search and other applications. This is useful whenever you cannot or do not want to retrain the main model, for example when processing informal Ukrainian or surzhyk social media text.
• I2. Cast normalization as monolingual phrase-based translation. Contiguous multi-word mappings plus a null token cover all three edit types: substitution, deletion and insertion. Merged forms like "lemme" or "duno" show why word-by-word dictionary replacement is not enough.
• I3. Profile the edit types before designing the model. Sampling a few hundred messages and counting substitutions, insertions and deletions showed which phenomena the model must cover. This kind of cheap analysis can guide the design of any normalization or correction task.
• I4. Evaluate both intrinsically and extrinsically. Besides BLEU against manual normalizations, the authors measure the gain in downstream MT across two different engines. They also include a dictionary-lookup baseline, which shows how much context-aware modeling adds over a simple lingo list.
• I5. Use a learning curve to check data sufficiency. Plotting score against training size, which plateaued at about 3000 pairs, gives a practical estimate of how much parallel data to annotate for a similar task.