Elamur Research⌕
☰
home / papers / 2014 / POS Tagging of English-Hindi Code-Mixed Social Media Content

POS Tagging of English-Hindi Code-Mixed Social Media Content

Summary

The paper addresses part-of-speech tagging of social media posts in which English and Hindi are mixed, often within one sentence. The text also contains Romanized Hindi, irregular spellings and informal grammar, and very little annotated data exists for this setting. The authors collected Facebook comments from popular celebrity pages and the BBC Hindi page. They annotated a subset in four layers: matrix language, word origin, normalization/back-transliteration, and Universal POS tags with named-entity marks. They then built a baseline pipeline from off-the-shelf parts. Each word first gets a language label. Words labeled Hindi are converted back to Devanagari. Finally, each same-language chunk goes to a monolingual tagger for that language. Experiments that progressively remove gold information show that errors accumulate across stages. Overall accuracy falls from about 79% with gold inputs to about 65% when the whole pipeline runs automatically. Language identification and Hindi transliteration turn out to be the main bottlenecks. The study helps define the problem and supplies an early annotated resource for code-mixed, transliterated text.

Key points

  • Raw corpus: 6,983 posts and comments with 113,578 words, taken from the Facebook pages of Amitabh Bachchan, Shahrukh Khan, Narendra Modi and BBC Hindi.
  • Annotated data: 1,062 posts (10,171 words) were annotated. After posts shorter than 5 words were removed, 381 posts with 4,135 words remained for the experiments.
  • Code-mixing statistics: about 17.2% of the annotated posts show code-mixing or switching. More than 40% of all words are Indian-language words written in Roman script.
  • Language identification: the King and Abney logistic regression classifier was trained on 3,201 English SMS words and 3,218 Hindi transliterations.
    • It labels 84.6% of tokens correctly.
    • Hindi precision is 0.959 but recall only 0.672, because contracted Hindi forms such as "h" for "hai" get mistaken for English.
  • POS tagging setup and results:
    • Taggers: a CRF++ Hindi tagger plus the Twitter POS tagger or the Stanford tagger for English, with all outputs mapped to the Universal tagset.
    • Total token accuracy is 79.02 for Models 1a/1b (gold language and normalization), 74.87 for Model 2 (gold language only) and 65.39 for Model 3 (fully automatic).
    • In the fully automatic setting, Hindi token accuracy drops to 44.74.

Interesting ideas

  • I1. Ablation by removing oracle inputs one stage at a time. Comparing the gold-everything, gold-language-only and fully automatic settings shows how much error each pipeline stage adds. This is a cheap way to decide where to invest effort in any multi-step NLP system.
  • I2. POS labels based on a word's function in context rather than its dictionary category. An embedded foreign word often takes on a different grammatical role in the host language. This choice would also suit annotating borrowings or mixed Ukrainian–Russian (surzhyk) text.
  • I3. Chunk-and-route tagging. The text is split into maximal same-language spans, and each span goes to an existing monolingual tagger. This gives a quick baseline for mixed-language data without any new training.
  • I4. A multi-layer annotation scheme: matrix language, word origin, normalized or native-script form, and POS. It separates distinct phenomena so each can be studied or modeled separately, and it could be reused for other bilingual or transliterated corpora.
  • I5. Chunk-level accuracy alongside token accuracy. Chunk-level scores reveal whole spans that are mis-tagged, for example a Hindi span wrongly identified as English. Token-level averages can hide such failures.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

77 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 74 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/d14-1105

@inproceedings{vyas2014pos,
  title = {{POS Tagging of English-Hindi Code-Mixed Social Media Content}},
  author = {Vyas, Yogarshi and Gella, Spandana and Sharma, Jatin and Bali, Kalika and Choudhury, Monojit},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1105},
  url = {https://doi.org/10.3115/v1/d14-1105}
}

References

27 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter Auer. 1984. The Pragmatics of Code-Switching: A Sequential Approach. Cambridge University Press. find
  2. Kalika Bali, Yogarshi Vyas, Jatin Sharma, and Monojit Choudhury. 2014. ‘’i am borrowing ya mixing?” an analysis of English-Hindi code mixing in Facebook. In Proceedings of the First Workshop on Computational Approaches to Code Switching, EMNLP. find
  3. Mónica Stella Cardenas-Claros and Neny Isharyanti. 2009. Code-switching and code-mixing in internet chatting: Between yes, ya, and si a case study. In The JALT CALL Journal, 5. find
  4. Simon Carter, Wouter Weerkamp, and Manos Tsagkias. 2013. Microblog language identification: Overcoming the limitations of short, unedited and idiomatic text. Language Resources and Evaluation Journal, 47:195–215. find
  5. Monojit Choudhury, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. IJDAR, 10(3-4):157–174. find
  6. David Crystal. 2001. Language and the Internet. Cambridge University Press. find
  7. Brenda Danet and Susan Herring. 2007. The Multilingual Internet: Language, Culture, and Communication Online. Oxford University Press., New York. find
  8. Spandana Gella, Jatin Sharma, and Kalika Bali. 2013. Query word labeling and back transliteration for indian languages: Shared task system description. In FIRE Working Notes. find
  9. Kevin Gimpel, N. Schneider, B. O’Connor, D. Das, D. Mills, J. Eisenstein, M. Heilman, D. Yogatama, J. Flanigan, and N. A. Smith. 2011. Part-of-speech tagging for twitter: Annotation, features, and experiments. In Proceedings of ACL. find our card
  10. Moises Goldszmidt, Marc Najork, and Stelios Papari-zos. 2013. Boot-strapping language identifiers for short colloquial postings. In Machine Learning and Knowledge Discovery in Databases, volume 8189 of Lecture Notes in Computer Science, pages 95–111. find
  11. John J. Gumperz. 1964. Hindi-punjabi code-switching in Delhi. In Proceedings of the Ninth International Congress of Linguistics. Mouton:The Hague. find
  12. John J. Gumperz. 1982. Discourse Strategies. Oxford University Press. find
  13. Kanika Gupta, Monojit Choudhury, and Kalika Bali. 2012. Mining Hindi-English transliteration pairs from online Hindi lyrics. In Proceedings of LREC. find
  14. Parth Gupta, Kalika Bali, Rafael E. Banchs, Monojit Choudhury, and Paolo Rosso. 2014. Query expansion for mixed-script information retrieval. In Proc. of SIGIR, pages 677–686. ACM Association for Computing Machinery. find
  15. Susan Herring, editor. 2003. Media and Language Change. Special issue of Journal of Historical Pragmatics 4:1. find
  16. Ben King and Steven Abney. 2013. Labeling the languages of words in mixed-language documents using weakly supervised methods. In Proceedings of NAACL-HLT, pages 1110–1119. find
  17. Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Computational linguistics, 19(2):313–330. find our card
  18. Carol Myers-Scotton. 1993. Dueling Languages: Grammatical Structure in Code-Switching. Claredon, Oxford. find
  19. Dong Nguyen and A. Seza Dogruoz. 2013. Word level language identification in online multilingual communication. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 857–862. find
  20. Olutobi Owoputi, Brendan OConnor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A. Smith. 2013. Improved part-of-speech tagging for online conversational text with word clusters. In Proceedings of NAACL. find our card
  21. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2011. A universal part-of-speech tagset. arXiv preprint arXiv:1104.2086. find
  22. Rishiraj Saha Roy, Monojit Choudhury, Prasenjit Majumder, and Komal Agarwal. 2013. Overview and datasets of fire 2013 track on transliterated search. In FIRE Working Notes. find
  23. Bhaskaran Sankaran, Kalika Bali, Monojit Choudhury, Tanmoy Bhattacharya, Pushpak Bhattacharyya, Girish Nath Jha, S. Rajendran, K. Saravanan, L. Sobha, and K. V. Subbarao. 2008. A common parts-of-speech tagset framework for indian languages. In Proceedings of LREC. find
  24. Thamar Solorio and Yang Liu. 2008a. Learning to predict code-switching points. In Proceedings of the Empirical Methods in natural Language Processing. find
  25. Thamar Solorio and Yang Liu. 2008b. Parts-of-speech tagging for English-Spanish code-switched text. In Proceedings of the Empirical Methods in natural Language Processing. find
  26. V. B. Sowmya, Monojit Choudhury, Kalika Bali, Tirthankar Dasgupta, and Anupam Basu. 2010. Resource creation for training and testing of transliteration systems for indian languages. In Proceedings of the Language Resource and Evaluation Conference (LREC). find
  27. Kristina Toutanova, Dan Klein, Christopher Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of HLT-NAACL. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vyas-2014-pos-tagging-english-hindi-code/