Elamur Research⌕
☰
home / papers / 2014 / “I am borrowing ya mixing ?” An Analysis of English-Hindi Code Mixing in Facebook

“I am borrowing ya mixing ?” An Analysis of English-Hindi Code Mixing in Facebook

Paper · 2014 · Kalika Bali, Jatin Sharma, Monojit Choudhury et al.

Summary

The paper investigates how English and Hindi are combined within single utterances by bilingual users on Facebook. Such text is hard to process: it mixes scripts, often renders Hindi in Roman transliteration, and contains contractions and non-standard spelling. The authors gathered comment threads from four public pages (Amitabh Bachchan, Shahrukh Khan, Narendra Modi, BBC Hindi), producing a corpus of 6,983 posts and 113,578 words. They then annotated a random subset of 1,062 posts. Each post was split into fragments with a matrix language, and annotators marked word origin, normalized spelling, POS tags from the Universal Tagset, and named entities. The analysis finds that English insertions into Hindi frames are far more common than the reverse. Using monolingual corpus frequencies, the authors sorted embedded English words into three groups: clear code-mixing, clear borrowing, and an ambiguous middle. Many words fall into that middle zone. This matters for NLP because words in each group, along with named entities, probably need different treatment by language identification, normalization and downstream tools.

Key points

  • The corpus comes from 40 manually chosen Facebook posts (Oct 22–28, 2013) with long comment threads (50+ comments). After cleaning, it contains 6,983 posts and 113,578 words, mostly in Roman script, with some Devanagari and a few other Indian scripts.
  • The annotation layers are:
    • matrix-language fragments (En/Hin) based on Myers-Scotton's Matrix/Embedding framework;
    • word origin (En/Hin/Ot);
    • normalization, including Devanagari spellings for romanized Hindi;
    • POS tags from the 12-tag Universal Tagset (Petrov et al.);
    • named entities typed as person, location or organization.
  • The annotated sample has 398 Hindi, 698 English and 6 Other single-language matrices. 45 posts switch matrix inside one utterance, so code-switching accounts for no less than 4.2% of the data.
  • 23.7% of Hindi matrices contain at least one English embedding, while only 7.2% of English matrices contain Hindi. About 17.2% of posts, covering nearly a quarter of all words, show some code-mixing. In 86% of English matrices, Hindi insertions appear only once or twice.
  • The appendix sorts the English words found in Hindi matrices (δ = 1, α = 1000):
    • borrowings, e.g. police, cricket, party, vote;
    • ambiguous words, e.g. love, sir, football;
    • code-mixed words, e.g. affection, divine, spokesperson.

Interesting ideas

  • I1. Frequency-based borrowing vs. mixing split. The authors compare how often a foreign word appears in monolingual corpora to place it on a borrowing–mixing continuum, rather than forcing a yes/no decision. The same approach could be used for Ukrainian–Russian or Ukrainian–English mixing, where loanwords and ad-hoc insertions also blur together.
  • I2. POS by function, not by dictionary class. Embedded words were tagged by the role they play in the host sentence, since their original category often disappears. This is a useful convention for any annotation of mixed or noisy text.
  • I3. Language-neutral tagset to skip label mapping. The coarse Universal Tagset could be applied to both languages without cross-language mapping, and a small corpus did not justify finer tags anyway. Starting coarse is a pragmatic choice for low-resource bilingual corpora.
  • I4. Normalization into the native script as an annotation layer. Each romanized or misspelled token is paired with its standard form, for Hindi in Devanagari. This yields data for training transliteration and normalization models alongside the linguistic analysis.
  • I5. Deliberately scaling down an over-ambitious scheme. The team first tried rich, layered socio-pragmatic annotation, found it too costly, and fell back to four core layers. This is a reminder to pilot annotation schemes before committing resources.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Maryland, College ParkCollege Park · United States83 works · 23,784 citations🏢Microsoft ResearchBengaluru · India9 works · 3,523 citations🏢Microsoft (India)Hyderabad · India6 works · 2,123 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 1 English · 1,373 works

Cite

Cite as Kalika Bali, Jatin Sharma, Monojit Choudhury and Yogarshi Vyas. 2014. “I am borrowing ya mixing ?” An Analysis of English-Hindi Code Mixing in Facebook. Proceedings of the First Workshop on Computational Approaches to Code Switching.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-3914

@inproceedings{bali2014i,
  title = {{“I am borrowing ya mixing ?” An Analysis of English-Hindi Code Mixing in Facebook}},
  author = {Kalika Bali, Jatin Sharma, Monojit Choudhury and Yogarshi Vyas},
  booktitle = {Proceedings of the First Workshop on Computational Approaches to Code Switching},
  year = {2014},
  doi = {10.3115/v1/w14-3914},
  url = {https://doi.org/10.3115/v1/w14-3914}
}

References

25 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Beatrice Alex. 2008. Automatic Detection of English Inclusions in Mixed-lingual Data with an Application to Parsing, Doctor of Philosophy Thesis, School of Informatics, University of Edinburgh, Edinburgh, UK. find
  2. Celso Alvarez-Cáccamo. 2011. "Rethinking conversational code-switching: codes, speech varieties, and contextualization." Proceedings of the Annual Meeting of the Berkeley Linguistics Society. Vol. 16. find
  3. Peter Auer. 1984. The Pragmatics of Code-Switching: A Sequential Approach. Cambridge University Press. find
  4. Abdelali Bentahila and Eirlys E. Davies. 1991. "Constraints on code-switching: A look beyond grammar. Papers for the symposium on code-switching in blingual studies: Theory, significance and perspectives. Strasbourg: European Science Foundation. find
  5. MS Cardenas-Claros and N Isharyanti. 2009. Code-switching and code-mixing in internet chatting: Between yes, ya, and si- a case study. In The JALT CALL Journal, 5 find
  6. David Crystal. 2001. Language and the Internet. Cambridge University Press. find
  7. B. Danet and S. Herring. 2007. The Multilingual Internet: Language, Culture, and Communication Online. Oxford University Press, New York. find
  8. Frederic Field. 2002. Linguistic borrowing in bilingual contexts. Amsterdam: Benjamins. find
  9. Penelope Gardner-Chloros. 2009. find
  10. Code-Switching. Cambrudge University Press J. Gumperz. 1964. Hindi-Punjabi code-switching in Delhi. In Proceedings of the Ninth International Congress of Linguistics, Mouton: The Hague. find
  11. J. Gumperz. 1982. Discourse Strategies. Oxford University Press. find
  12. S. Herring. 2003. Media and Language Change: Special Issue. find
  13. Jeff MacSwan. 2012." Code-Switching and Grammatical Theory." In The Handbook of Bilingualism and Multilingualism (2012). 323. find
  14. Carol Myers-Scotton. 1993. Duelling Languages: Grammatical Structure in Code-switching. Claredon. Oxford. find
  15. Carol Myers-Scotton. 2002. Contact linguistics: Bilingual encounters and grammatical outcomes. Oxford University Press. find
  16. Pieter Muysken. 2000. Bilingual speech: A typology of code-mixing. Cambridge University Press. find
  17. John C. Paolillo. 2011. “Conversational” codeswitching on Usenet and Internet Relay Chat. In Language@Internet, 8, article 3. find
  18. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2011. A universal part-of-speech tagset. arXiv preprint arXiv:1104.2086 our card
  19. Shana Poplack, D. Sankoff, and C. Miller. 1988. The social correlates and linguistic processes of lexical borrowing and assimilation. Linguistics 26:47-104. find
  20. Shana Poplack and Nathalie Dion. 2012. "Myths and facts about loanword development." in Language Variation and Change 24, 3. find
  21. David Sankoff, Shana Poplack, and Swathi Vanniarajan. 1990. The case of the nonce loan in Tamil. Language Variation and Change, 2 (1990), 71-101. Cambridge University Press. find
  22. V.B. Sowmya, M. Choudhury, K. Bali, T. Dasgupta, and A. Basu. 2010. Resource creation for training and transliteration systems for Indian languages. In Proceedings of Language Resource and Evaluations Conference (LREC 2010). find
  23. Sarah G. Thomason. 2003. Contact as a Source of Language Change. In R.D. Janda & B. D. Joseph (eds), A handbook of historical linguistics, Oxford: Blackwell. find
  24. Paola Virga and Sanjeev Khudanpur. 2003. Transliteration of proper names in cross-lingual information retrieval. Proceedings of the ACL 2003 workshop on Multilingual and mixed-language named entity recognition-Volume 15. Association for Computational Linguistics. our card
  25. Donald Winford. 2003. An Introduction to Contact Linguistics. Malden, MA: Blackwell. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bali-2014-i-am-borrowing-ya-mixing/