Elamur Research⌕
☰
home / papers / 2015 / SMT Errors Requiring Grammatical Knowledge for Prevention

SMT Errors Requiring Grammatical Knowledge for Prevention

Summary

The paper presents a qualitative analysis of errors in English-to-Japanese statistical machine translation (SMT). The author manually tested the online system Google Language Tools on individual sentences. She identified three classes of errors that, in her view, cannot be avoided without grammatical knowledge. The first class concerns words that have a negative meaning but no negative form (few, little, seldom, hardly, etc.). In Japanese, such sentences require explicit negation in the predicate. The system, however, often translates these words as "a few" or "almost", which reverses the meaning. The second class arises from inserted adverbials that break up fixed constructions, so the system no longer recognizes them. The third class is related to ellipsis. Constituents omitted in the English sentence must be restored when Japanese omission rules or a different Head-Complement order require it. The paper contains no quantitative evaluation. The author concludes that such cases require syntactic analysis and grammatical rules, and that statistics alone are not enough. The paper shows the weak points of SMT for language pairs with very different grammars.

Key points

  • The object of study is English-Japanese translation in the Google Language Tools service. The method is manual analysis of examples, without a corpus or metrics.
  • Words with implicit negation: the determiners few and little, and the adverbs little, seldom, rarely, scarcely, hardly and barely. The part of speech must be determined, since few can be a determiner, a pronoun or an adjective. The system translated sentences with little attention and was hardly surprising with the opposite meaning. By contrast, it translated could/can hardly constructions almost correctly.
  • Adverbials such as so far or long inside template phrases like "unable to …" or "have been geared" spoil the translation. When the insertion is removed, the result becomes correct. The author mentions the controlled skip parser (Yamada, 1996), but cautions that negative adverbs must not be skipped.
  • Ellipsis: the omitted "those who" must be restored in the Japanese translation. The elliptical comparisons as much and half as long also proved problematic.
  • Conclusion: adding new templates for each case is impractical. Syntactic parsing and rules for recognition and generation are needed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alam2015smt,
  title = {{SMT Errors Requiring Grammatical Knowledge for Prevention}},
  author = {Alam, Yukiko Sasaki},
  booktitle = {Proceedings of the 12th International Conference on Natural Language Processing},
  year = {2015},
  url = {https://aclanthology.org/W15-5923/}
}

References

11 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bunt, H. and A Van Horck. 1996. Discontinuous Constituency. Mouton De Gruyter. find
  2. Condon, Sherri, Dan Parvaz, John Aberdeen, Christy Doran, Andrew Freeman, and Marwan Awad. Evaluation of Machine Translation Errors in English and Iraqi Arabic. The MITRE Corporation. 7 pages. 2010 find
  3. Elliot, Debbie, Anthony Hartley, and Eric Atwell. A Fluency Error Categorization Scheme to Guide Automated Machine Translation Evaluation. AMTA-2004. 64-73, 2004. find
  4. Farrús, Mireia, Marta R. Costa-jussa, Jose B. Marino and Jose A. R. Fonollosa. Linguistic-based Evaluation Criteria to Identify Statistical Machine Translation Erros. Annual Conference of the European Association for Machine Translation. 167-173, 2010. find
  5. Farrús, Mireia, Marta R. Costa-jussa, Jose B. Marino, Marc Posh, Adolfo Hernandez, Carlos Henriquez, and Jose A. R. Fonollosa. Overcoming statistical machine translation limitations: error analysis and proposed solutions for the Catalan-Spanish language pair. Language Resources and Evaluation (Sprin… find
  6. Flanagan, Mary A. Error classification for MT evaluation. -1994. 65-72, 1994. find
  7. Popović, Maja and Aljoscha Burchardt. From Human to Automatic Error Classification for Machine Translation Output. Proceedings of the 15th Conference of the European Association for Machine Translation. 265-272, 2011. find our card
  8. Stymne, Sara and Lars Ahrenberg. On the practice of error analysis for machine translation evaluation. LREC. 1785-1790. 2012. find
  9. Talbot, David, Hideto Kazawa, Hiroshi Ichikawa, Jason Katz-Brown, Mranz J. Och. A Lightweight Evaluation Framework for Machine Translation Reordering. Proceedings of the 6 th Workshop on Statistical Machine Translation. 12-21. 2011. find
  10. Vilar, David, Jia Xu, Luis Fernaudo D’Haro, and Hermann Ney. Error Analysis of Statistical Machine Translation Output. Proceedings of the LREC. 697-702. 2006. find our card
  11. Yamada, Ken. 1996. A controlled Skip Parser. Proceedings of the 2 nd AMTA Conference. find

This page in Markdown →