Elamur Research⌕
☰
home / papers / 2020 / Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training

Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training

Paper · 2020 · Ernie Chang, David Ifeoluwa Adelani, Xiaoyu Shen et al.

Summary

West African Pidgin English, mainly the Nigerian variety, has tens of millions of speakers. It still has almost no parallel corpora, so generating Pidgin text from structured data is largely unexplored. An earlier unsupervised translation system, PidginUNMT, was trained on news and Bible material and transfers poorly to other domains.

The authors reuse English data-to-text generation and add a translation step into Pidgin that fits the target domain without any parallel pairs. The English side of the E2E restaurant corpus serves as the pivot. They train FastText cross-lingual embeddings and an unsupervised NMT model between this in-domain English and an out-of-domain monolingual Pidgin corpus, giving model_unsup. They then self-train: the model's own Pidgin outputs become pseudo-parallel pairs for further updates, giving model_self.

Two native speakers judged the outputs. The self-trained model was much more relevant than PidginUNMT and about as fluent. The results are not yet usable in practice, but they offer a starting recipe for NLG in languages without annotated data.

Key points

  • Speaker estimates cited: about 75 million in Nigeria and over 5 million in Ghana. The prior PidginUNMT system (Ogueji & Ahia, 2019) reported a BLEU of 5.18 for English→Pidgin.
  • Data used:
    • E2E (Novikova et al., 2017), roughly 42k training samples in the restaurant domain.
    • A monolingual Pidgin corpus of 56,695 sentences with 32,925 unique words.
  • Pipeline:
    • FastText cross-lingual embeddings trained on the combined Pidgin-English text.
    • Unsupervised NMT in the style of Lample et al. and Artetxe et al. (model_unsup).
    • Self-training following He et al. (2019), using pseudo-parallel pairs plus the monolingual data (model_self).
  • Human evaluation covered 630 E2E test instances, rated by 2 native speakers. Relevance was scored 0/1 and fluency 0–2. Relevance was 0.038 for PidginUNMT, 0.319 for model_unsup and 0.434 for model_self. Fluency was 0.827, 0.788 and 0.814 respectively.
  • Planned future work is to bring in phrase-based statistical MT.

Interesting ideas

  • I1. Keep a mature English generator and add only a thin translation layer, trained on English from the same domain the generator produces, instead of building a generator for each language.
  • I2. Self-training mixes the model's own pseudo-parallel pairs with the original non-parallel data, so the model learns from its guesses while staying anchored to real text.
  • I3. Domain match mattered more than system novelty: the out-of-domain translator gave fluent text that was almost never relevant to the input.
  • I4. Human evaluation scored relevance (binary) and fluency (graded) separately, which exposes output that reads well but says the wrong thing.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Mova's graph→English realizer could feed a translation layer trained on Mova's own in-domain English output, so the translated side matches the vocabulary and style Mova actually generates.
  • I2 (learn): when parser or realizer output is recycled as silver data, keep it blended with genuine treebank or corpus material, and check the gain against a negative control trained without the silver portion.
  • I3 (answer): measure readers on the actual target genre, such as fables or school tests, rather than trusting scores from general text, since a mismatch can stay hidden behind fluent output.
  • I4 (back): the text → graph → text compressor could report faithfulness to the meaning graph and surface fluency as separate scores, so a fluent but unfaithful paraphrase is never counted as a success.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Saarland UniversitySaarbrücken · Germany102 works · 17,447 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 3Research subjects and languages the work is about.Machine translation · 958 worksEnglish · 1,373 worksText generation · 145 works

Cite

Cite as Ernie Chang, David Ifeoluwa Adelani, Xiaoyu Shen and Vera Demberg. 2020. Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training. arXiv:2003.08272.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2003.08272 · arXiv 2003.08272

@misc{chang2020unsupervised,
  title = {{Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training}},
  author = {Ernie Chang, David Ifeoluwa Adelani, Xiaoyu Shen and Vera Demberg},
  year = {2020},
  doi = {10.48550/arxiv.2003.08272},
  eprint = {2003.08272},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2003.08272}
}

References

15 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041, 2017. ↗
  2. Anja Belz. Automatic generation of weather forecast texts using comprehensive probabilistic generation-space models. Natural Language Engineering, 14(4):431–455, 2008. ↗
  3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146, 2017. ISSN 2307-387X. ↗
  4. Ernie Chang, Vera Demberg, and Alex Marin. Jointly improving language understanding and generation with quality-weighted weak supervision of automatic labeling. Proceedings of EACL 2021, 2021a. find
  5. Ernie Chang, Xiaoyu Shen, Dawei Zhu, Vera Demberg, and Hui. Su. Neural data-to-text generation with lm-based text augmentation. Proceedings of EACL 2021, 2021b. find
  6. José GC de Souza, Michael Kozielski, Prashant Mathur, Ernie Chang, Marco Guerini, Matteo Negri, Marco Turchi, and Evgeny Matusov. Generating e-commerce product titles and predicting their quality. In Proceedings of the 11th International Conference on Natural Language Generation, pp. 233–243, 2018. find
  7. Yinpeng Guo, Yi Liao, Xin Jiang, Qing Zhang, Yibo Zhang, and Qun Liu. Zero-shot paraphrase generation with multilingual language models. arXiv preprint arXiv:1911.03597, 2019. ↗
  8. Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio Ranzato. Revisiting self-training for neural sequence generation. arXiv preprint arXiv:1909.13788, 2019. ↗
  9. Xudong Hong, Ernie Chang, and Vera Demberg. Improving language generation from feature-rich tree-structured data with relational graph convolutional encoders. In Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019), pp. 75–80, 2019. find
  10. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. Unsupervised machine translation using monolingual corpora only. arXiv preprint arXiv:1711.00043, 2017. ↗
  11. Rémi Lebret, David Grangier, and Michael Auli. Neural text generation from structured data with application to the biography domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1203–1213, 2016.
  12. Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. The e2e dataset: New challenges for end-to-end generation. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pp. 201–206, 2017.
  13. Kelechi Ogueji and Orevaoghene Ahia. Pidginunmt: Unsupervised neural machine translation from west african pidgin to english. arXiv preprint arXiv:1912.03444, 2019. ↗
  14. Somayajulu Sripada, Ehud Reiter, Jim Hunter, and Jin Yu. Sumtime-meteo: Parallel corpus of naturally occurring forecast texts and weather data. Computing Science Department, University of Aberdeen, Aberdeen, Scotland, Tech. Rep. AUCS/TR0201, 2002. find
  15. WenLi Zhuang and Ernie Chang. Neobility at semeval-2017 task 1: An attention-based sentence similarity model. arXiv preprint arXiv:1703.05465, 2017. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chang-2020-unsupervised-pidgin-text-generation-pivoting/