Elamur Research⌕
☰
home / papers / 2018 / Neural Surface Realization for Italian

Neural Surface Realization for Italian

Summary

The paper addresses surface realization for Italian, that is, generating a sentence from an unordered dependency tree in UD format. The authors split the task into two independent parts: word ordering and word form generation. Each part is handled by a separate neural network, and the outputs of the two are combined at the end. For word ordering, the tree is divided into single-level subtrees, each consisting of a head and its dependents. Each subtree is ordered with the ListNet ranking algorithm, and the order of the whole sentence is assembled from these local orders. Word forms are generated by the seq2seq model of Aharoni and Goldberg with hard attention, which takes as input the lemma, the morphological features and the syntactic relation. The DipInfo-UniTo system was previously submitted to SRST18. In this paper, the authors examine how training data and hyperparameters affect it. To the "gold" Italian treebanks they added a "silver" corpus from ItWaC, automatically annotated with UDPipe. They attribute the weak result in the shared task to an overly large learning rate. After tuning, BLEU-4 rose by about 12 points, and on NIST the system surpassed the best SRST18 participant.

Key points

  • Data. The Gold-SRST18 corpus consists of four Italian UD treebanks: 12 838 sentences and 270 703 tokens. The Silver-WaC corpus is a random sample from ItWaC annotated with UDPipe: 9 427 sentences and 274 115 tokens.
  • Features for ListNet. The model uses UPOS, the language-specific POS tag, morphological features and a flag indicating whether the word is the head of the subtree. Content words are represented with Polyglot embeddings, and function words with one-hot vectors. The code is published in the ud2ln and listnet repositories on GitHub.
  • Learning rate. With LR = 5·10⁻⁵, as in the official SRST18 submission, the Kendall τ curve is unstable and "jumps". With LR = 5·10⁻⁶ it reaches a plateau after about 100 epochs.
  • Results. BLEU-4 rose from 24.61 to 36.40–36.60, and NIST from 8.25 to 9.30. For comparison, the best SRST18 system had a NIST of 9.11. At the same time, DIST dropped from 36.11 to about 32.7, and the system is still far from the leader's scores (BLEU-4 44.16, DIST 58.61). Adding Silver-WaC changes BLEU-4 by at most 0.2 points.
  • Limitations. Because of the projectivity assumption, the system cannot correctly order non-projective sentences. The morphological module gains almost nothing from the "silver" data. The authors attribute this either to errors in the automatic annotation or to the model itself plateauing early.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Treebanks · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{basile2018neural,
  title = {{Neural Surface Realization for Italian}},
  author = {Basile, Valerio and Mazzei, Alessandro},
  booktitle = {Proceedings of the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018)},
  year = {2018},
  doi = {10.4000/books.aaccademia.3079},
  url = {https://doi.org/10.4000/books.aaccademia.3079}
}

References

12 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Roee Aharoni and Yoav Goldberg. 2017. Morphological inflection generation with hard monotonic attention. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, pages 2004–2015. ↗ our card
  2. Rami Al-Rfou’, Bryan Perozzi, and Steven Skiena. 2013. Polyglot: Distributed word representations for multilingual nlp. In CoNLL, pages 183–192. ACL. https://cloud.garr.it ↗ our card
  3. Marco Aldinucci, Sergio Rabellino, Marco Pironti, Filippo Spiga, Paolo Viviani, Maurizio Drocco, Marco Guerzoni, Guido Boella, Marco Mellia, Paolo Mar-gara, Idillio Drago, Roberto Marturano, Guido Marchetto, Elio Piccolo, Stefano Bagnasco, Stefano Lusso, Sara Vallero, Giuseppe Attardi, Alex Barchiesi, Alberto Colla, and Fulvio Galeazzi. 2018. Hpc4ai, an ai-on-demand federated platform endeavour. In ACM Computing Frontiers, Ischia, Italy, May. find
  4. Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide web: a collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation, 43(3):209–226, September. ↗
  5. Valerio Basile and Alessandro Mazzei. 2018. The dipinfo-unito system for srst 2018. In Proceedings of the First Workshop on Multilingual Surface Realisation, pages 65–71. Association for Computational Linguistics. find
  6. Valerio Basile. 2015. From Logic to Language : Natural Language Generation from Logical Forms. Ph.D. thesis, University of Groningen, Netherlands. find
  7. Bernd Bohnet, Anders Björkelund, Jonas Kuhn, Wolfgang Seeker, and Sina Zarrieß. 2012. Generating non-projective word order in statistical linearization. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 928–939. Association for Computational Linguistics. find our card
  8. Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: From pairwise approach to listwise approach. In Proceedings of the 24th International Conference on Machine Learning, ICML ’07, pages 129–136, New York, NY, USA. ACM. find
  9. M. G. Kendall. 1938. A new measure of rank correlation. Biometrika, 30(1/2):81–93. ↗
  10. Simon Mille, Anja Belz, Bernd Bohnet, Yvette Graham, Emily Pitler, and Leo Wanner. 2018. The first multilingual surface realisation shared task (sr’18): Overview and evaluation results. In Proceedings of the First Workshop on Multilingual Surface Realisation, pages 1–12. Association for Computational Linguistics. find our card
  11. Ehud Reiter and Robert Dale. 2000. Building Natural Language Generation Systems. Cambridge University Press, New York, NY, USA. ↗
  12. Milan Straka and Jana Straková. 2017. Tokenizing, pos tagging, lemmatizing and parsing ud 2.0 with udpipe. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 88–99, Vancouver, Canada, August. Association for Computational Linguistics. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/basile-2018-neural-surface-realization-italian/