Elamur Research⌕
☰
home / papers / 2015 / Grammar Engineering for a Customer: a Case Study with Five Languages

Grammar Engineering for a Customer: a Case Study with Five Languages

Paper · 2015 · Aarne Ranta, Christina Unger and Daniel Vidal Hussey

Summary

The paper describes a commercial project. The company Digital Grammars AB, commissioned by the Swedish web service Tillgänglighetsdatabasen, built a grammar-based machine translation system. The service describes how accessible various places are for people with disabilities. The system translates from Swedish into English, Finnish, German, and Spanish. Previous applications of GF (Grammatical Framework) worked with a controlled language. Here, the input texts were written by different authors over several years, so they contain errors and HTML markup. The authors chose the "embedded CNL" approach. A domain-specific controlled grammar is built on the Resource Grammar Library and embedded in a general wide-coverage grammar (WCT), which serves as a fallback. Probabilistic weights give preference to domain parses, and the resulting translations are then post-edited by humans. The domain grammar parsed 95% of the sentences, and the fallback mode covered the rest. BLEU against the post-edited translations was 73–74, compared with 28–37 for Google Translate. The whole project took 138 of the planned 200 hours. The work is valuable because it assesses the real cost and commercial viability of grammar-based translation beyond research prototypes.

Key points

  • Corpus: about 1,900 short, formulaic texts. After markup removal, 1,185 unique texts remained: 7,309 words, 980 lemmas according to SALDO. The grammar distinguishes 13 types of variables. The Swedish lexicon contains 72 multiword constructions, the English one 274.
  • Quality: BLEU was 73 for Finnish, 73 for German, and 74 for Spanish; for Google, 28, 37, and 28 respectively. Without post-editing, 46%, 42%, and 32% of the translations were correct. Caveat: the grammar was evaluated on the same data on which it was built, and Google only on a small sample.
  • Effort: the shared components (abstract syntax, Swedish and English grammars, data preparation) took 48 hours, and this cost is recouped with each new language. The grammar itself took 14 hours for Finnish, 18 for German, and 38 for Spanish; the Spanish grammar was written by a student with no prior experience with GF.
  • Code size: approximately 1,290 lines of GF per language, 8,056 lines in total. The robustness module is 70 lines long and identical for all languages.
  • Economics: post-editing proceeded at a rate of 900–1 200 words per hour. However, once grammar development is taken into account, the overall rate was about 160 words per hour, i.e., on par with human translation. The benefit emerges when new texts of the same type are translated later.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of GothenburgGothenburg · Sweden34 works · 1,625 citations🎓Bielefeld UniversityBielefeld · Germany35 works · 1,196 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Syntax & grammar formalisms · 214 works

Cite

Cite as Aarne Ranta, Christina Unger and Daniel Vidal Hussey. 2015. Grammar Engineering for a Customer: a Case Study with Five Languages. Proceedings of the Grammar Engineering Across Frameworks (GEAF) 2015 Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3301

@inproceedings{ranta2015grammar,
  title = {{Grammar Engineering for a Customer: a Case Study with Five Languages}},
  author = {Aarne Ranta, Christina Unger and Daniel Vidal Hussey},
  booktitle = {Proceedings of the Grammar Engineering Across Frameworks (GEAF) 2015 Workshop},
  year = {2015},
  doi = {10.18653/v1/w15-3301},
  url = {https://doi.org/10.18653/v1/w15-3301}
}

References

24 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alfred V. Aho and Jeffrey D. Ullman. 1969. Syntax directed translations and the pushdown assembler. Journal of Computer and System Sciences, 3(1):37 – 56. find
  2. Krasimir Angelov and Peter Ljunglöf. 2014. Fast statistical parsing with parallel multiple context-free grammars. In EACL’14, pages 368–376. find
  3. Krasimir Angelov, Björn Bringert, and Aarne Ranta. 2014. Speech-enabled hybrid multilingual translation for mobile devices. EACL’14, pages 41–44. find
  4. Lars Borin, Markus Forsberg, and Lennart Lönngren. 2013. Saldo: a touch of yin to wordnet’s yang. Language resources and evaluation, 47(4):1191–1211. find
  5. Miriam Butt, Helge Dyvik, Tracy Holloway King, Hiroshi Masuichi, and Christian Rohrer. 2002. The parallel grammar project. In Proceedings of the 2002 workshop on Grammar engineering and evaluation-Volume 15, pages 1–7. Association for Computational Linguistics.
  6. Haskell B. Curry. 1961. Some logical aspects of grammatical structure. In Roman Jakobson, editor, Structure of Language and its Mathematical Aspects: Proceedings of the Twelfth Symposium in Applied Mathematics, pages 56–68. American Mathematical Society. find
  7. Mariana Damova, Dana Dannélls, Ramona Enache, Maria Mateva, and Aarne Ranta. 2014. Multilingual natural language interaction with semantic web knowledge bases and linked open data. In Towards the Multilingual Semantic Web, pages 211– 226. Springer Berlin Heidelberg. find
  8. Brian Davis, Ramona Enache, Jeroen van Grondelle, and Laurette Pretorius. 2012. Multilingual verbalisation of modular ontologies using gf and lemon. In Tobias Kuhn and Norbert Fuchs, editors, Controlled Natural Language, volume 7427 of LNCS/LNAI, pages 167–184. Springer. find
  9. Grégoire Détrez and Aarne Ranta. 2012. Smart paradigms and the predictability and complexity of inflectional morphology. In EACL (European Association for Computational Linguistics), Avignon, April. Association for Computational Linguistics. find
  10. Marc Dymetman, Veronika Lux, and Aarne Ranta. 2000. XML and multilingual document authoring: Convergent trends. In Proc. Computational Linguistics COLING, Saarbrücken, Germany, pages 243–249. International Committee on Computational Linguistics. find
  11. Ramona Enache, Inari Listenmaa, and Prasanth Kolachina. 2014. Handling non-compositionality in multilingual CNLs. In Controlled Natural Language - 4th International Workshop, CNL 2014, Galway, Ireland, August 20-22, 2014. Proceedings, volume 8625 of LNCS. find
  12. Norbert E. Fuchs, Kaarel Kaljurand, and Tobias Kuhn. 2008. Attempto Controlled English for Knowledge Representation. In Cristina Baroglio, Piero A. Bonatti, Jan Małuszyński, Massimo Marchiori, Axel Polleres, and Sebastian Schaffert, editors, Reasoning Web, Fourth International Summer School 2008, n… find
  13. Meritxell Gonzàlez, Jesús Giménez, and Lluı́s Màrquez. 2012. A Graphical Interface for MT Evaluation and Error Analysis. In The 50th Annual Meeting of the Association for Computational Linguistics. find
  14. Thomas Hallgren, Aarne Ranta, John Camilleri, Grégoire Détrez, and Ramona Enache. 2012. Grammar Tools and Best Practices. MOLTO Deliverable D2.3, June. find
  15. Richard Montague. 1974. Formal Philosophy. Yale University Press, New Haven. Collected papers edited by Richmond Thomason. find
  16. Aarne Ranta, Ramona Enache, and Grégoire Détrez. 2012. Controlled Language for Everyday Use: the MOLTO Phrasebook. In Tobias Kuhn and Norbert Fuchs, editors, Controlled Natural Language, volume 7427 of LNCS/LNAI. Springer. find
  17. Aarne Ranta. 2009. The GF Resource Grammar Library. Linguistics in Language Technology, 2:1–65. find
  18. Aarne Ranta. 2011. Grammatical Framework: Programming with Multilingual Grammars. CSLI Publications, Stanford. find
  19. Aarne Ranta. 2014. Embedded controlled languages. In Controlled Natural Language - 4th International Workshop, CNL 2014, Galway, Ireland, August 20-22, 2014. Proceedings, volume 8625 of LNCS. find
  20. Jussi Rautio and Maarit Koponen. 2013. Deliverable 9.2: Molto evaluation and assessment report. find
  21. Manny Rayner, David Carter, Pierrette Bouillon, Vassilis Digalakis, and Mats Wirén. 2000. The Spoken Language Translator. Cambridge University Press, Cambridge. find
  22. Manny Rayner, Beth Ann Hockey, and Pierrette Bouillon. 2006. Putting Linguistics into Speech Recognition: The Regulus Grammar Compiler. CSLI Publications. find
  23. M. T. Rosetta. 1994. Kluwer, Dordrecht. find
  24. Compositional Translation. Jordi Saludes and Sebastian Xambó. 2011. The gf mathematics library. In Pedro Quaresma and Ralph-Johan Back, editors, Proceedings First Workshop on CTP Components for Educational Software (THedu’11), number 79, pages 102–110. Electronic Proceedings in Theoretical Computer… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ranta-2015-grammar-engineering-customer-case-study/