Automatic Language Identification in Texts: A Survey

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Computing & GPUs

Cited by

101 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 100 not yet.

• GlotLID: Language Identification for Low-Resource Languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1613/jair.1.11675 · arXiv 1804.08186

@article{jauhiainen2019automatic,
  title = {{Automatic Language Identification in Texts: A Survey}},
  author = {Jauhiainen, Tommi and Lui, Marco and Zampieri, Marcos and Baldwin, Timothy and Lindén, Krister},
  journal = {Journal of Artificial Intelligence Research 65},
  year = {2019},
  doi = {10.1613/jair.1.11675},
  eprint = {1804.08186},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1613/jair.1.11675}
}

References

446 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kheireddine Abainia, Siham Ouamour, and Halim Sayoud. Robust Language Identification of Noisy Texts - Proposal of Hybrid Approaches. In 25th International Workshop on Database and Expert Systems Applications (DEXA), pages 228–232, Munich, Germany, 2014. find
2. Kheireddine Abainia, Siham Ouamour, and Halim Sayoud. Effective Language Identification of Forum Texts Based on Statistical Approaches. Information Processing and Management, 52:491–512, 2016. find
3. Steven Abney and Steven Bird. The Human Language Project: Building a Universal Corpus of the World’s Languages. In Proceedings of Human Language Technologies: The 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), pages 88–97, Los… find
4. Judit Ács, László Grad-Gyenge, Thiago Bruno, and Rodriguez de Rezende Oliveira. A Two-level Classifier for Discriminating similar Languages. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 73–77, Hissar, Bulgari… find
5. Gary Adams and Philip Resnik. A Language Identification Application Built on the Java Client/server Platform. In Proceedings of the ACL/EACL’97 Workshop on From Research to Commercial Applications: Making NLP Work in Practice, pages 43–47, Madrid, Spain, 1997. find
6. Wafia Adouane. Automatic Detection of Underresourced Languages: Dialectal Arabic Short Texts. Master’s thesis, University of Gothenburg, Gothenburg, Sweden, 2016. find
7. Wafia Adouane and Simon Dobnik. Identification of Languages in Algerian Arabic Multilingual Documents. In Proceedings of The Third Arabic Natural Language Processing Workshop (WANLP 2017), pages 1–8, Valencia, Spain, 2017. find
8. Wafia Adouane, Nasredine Semmar, and Richard Johansson. Romanized Berber and Romanized Arabic Automatic Language Identification Using Machine Learning. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 53–61, Osaka, Japan, 2016a. find
9. Wafia Adouane, Nasredine Semmar, and Richard Johansson. Arabicized and Romanized Berber Automatic Identification. In Proceedings of the International Conference on Information and Communication Technologies for Amazingh (TICAM 2016), Rabat, Morocco, 2016b. IRCAM. find
10. Wafia Adouane, Nasredine Semmar, and Richard Johansson. ASIREM Participation at the Discriminating Similar Languages Shared Task 2016. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 163– 169, Osaka, Japan, 2016c. find
11. Wafia Adouane, Nasredine Semmar, Richard Johansson, and Victoria Bobicev. Automatic Detection of Arabicized Berber and Arabic Varieties. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 63–72, Osaka, Japan, 2016d. find
12. Bashir Ahmed, Sung-Hyuk Cha, and Charles Tappert. Language Identification from Text Using N-gram Based Cumulative Frequency Addition. In Proceedings of Student/Faculty Research Day, pages 12.1–12.8, CSIS, Pace University, New York, USA, 2004. find
13. Malepati Bala Siva Sai Akhil and J. Abhishek. Language Identification, Transliteration and Resolving Common Words Ambiguity in a Pair of Languages: Shared Task on Transliterated Search. In Working Notes of Shared Task on Transliterated Search at Forum for Information Retrieval Evaluation (FIRE’14),… find
14. Liliya Akhtyamova, John Cardiff, and Andrey Ignatov. Twitter Author Profiling Using Word Embeddings and Logistic Regression - Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and Workshop… find
15. Nicholas Akosu and Ali Selamat. A Dynamic Model Selection Algorithm for Language Identification of Under-resourced Languages. International Journal of Digital Content Technology and its Applications (JDCTA), 8, 2014. find
16. Mohamed Al-Badrashiny and Mona T. Diab. LILI: A Simple Language Independent Approach for Language Identification. In Proceedings of the 26th International Conference on Computational Linguistics (COLING 2016): Technical Papers, pages 1211–1219, Osaka, Japan, 2016. find
17. Mohamed Al-Badrashiny, Heba Elfardy, and Mona Diab. AIDA2: A Hybrid Approach for Token and Sentence Level Dialect Identification in Arabic. In Proceedings of the 19th Conference on Computational Language Learning, pages 42–51, Beijing, China, 2015. find
18. Beatrice Alex. An Unsupervised System for Identifying English Inclusions in German Text. In Proceedings of the Student Research Workshop, ACL-05, pages 133–138, Ann Arbor, Michigan, USA, 2005. find
19. Beatrice Alex, Amit Dubey, and Frank Keller. Using Foreign Inclusion Detection to Improve Parsing Performance. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning 2007 (EMNLP-CoNLL 2007), pages 151–160, Prague, Czech… find
20. David Alfter. Language Segmentation. Master’s thesis, Universität Trier, Trier, Germany, 2015. find
21. Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell, and Steve Renals. Automatic Dialect Detection in Arabic Broadcast Speech. In Proceedings of Interspeech 2016, pages 2934–2938, San Francisco, USA, 2016. find
22. Ahmed Ali, Stephan Vogel, and Steve Renals. Speech Recognition challenge in the Wild: Arabic MGB-3. arXiv preprint, arXiv:1709.07276, 2017. find
23. Khaled Alrifai, Ghaida Rebdawi, and Nada Ghneim. Arabic Tweeps Gender and Dialect Prediction – Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and Workshop, Dublin, Ireland, September 20… find
24. AOO Alshutayri and Eric Atwell. Exploring Twitter as a Source of an Arabic Dialect Corpus. International Journal of Computational Linguistics (IJCL), 8(2):37–44, 2017. find
25. Areej Alshutayri, Eric Atwell, AbdulRahman Alosaimy, James Dickins, Michael Ingleby, and Janet Watson. Arabic Language WEKA-Based Dialect Classifier for Arabic Automatic Speech Recognition Transcripts. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial… find
26. Abdelmalek Amine, Zakaria Elberrichi, and Michel Simonet. Automatic Language Identification: An Alternative Unsupervised Approach Using a New Hybrid Algorithm. International Journal of Computer Science and Applications, 7(1):94–107, 2010. find
27. Supriya Anand. Language Identification for Transliterated Forms of Indian Language Queries. In Working Notes of Forum for Information Retrieval Evaluation (FIRE), Bangalore, India, 2014. find
28. Stefanie Anstein. Computational Approaches to the Comparison of Regional Variety Corpora: Prototyping a Semi-automatic System for German. PhD thesis, University of Stuttgart, 2013. find
29. Olga Artemenko and Margaryta Shramko. Entwicklung eines Werkzeugs zur Sprachiden-tifikation in mono- und multilingualen Texten. Master’s thesis, Universität Hildesheim, 2005. find
30. A. S. Babu and P. Kumar. Comparing Neural Network Approach with N-Gram Approach for Text Categorization. International Journal on Computer Science and Engineering, 2 (1):80–83, 2010. find
31. Ivana Balažević, Mikio Braun, and Klaus-Robert Müller. Language Detection For Short Text Messages In Social Media. arXiv preprint, arXiv:1608.08515, 2016. — [arXiv:1608.08515](https://arxiv.org/abs/1608.08515) find
32. Timothy Baldwin and Marco Lui. Multilingual Language Identification: ALTW 2010 Shared Task Dataset. In Proceedings of the Australasian Language Technology Workshop 2010 (ALTW 2010), pages 5–7, Melbourne, Australia, 2010a. find
33. Timothy Baldwin and Marco Lui. Language Identification: The Long and the Short of the Matter. In Proceedings of Human Language Technologies: The 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), pages 229–237, Los Angeles, USA, 20… find
34. Somnath Banerjee, Aniruddha Roy, Alapan Kuila, Sudip Kumar Naskar, Sivaji Bandyopadhyay, and Paolo Rosso. A Hybrid Approach for Transliterated Word-Level Language Identification: CRF with Post Processing Heuristics. In Proceedings of the Sixth Workshop of the Forum for Information Retrieval Evaluati… find
35. Kfir Bar and Nachum Dershowitz. The Tel Aviv University System for the Code-Switching Workshop Shared Task. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 139–143, Doha, Qatar, 2014. find
36. Adrien Barbaresi. An Unsupervised Morphological Criterion for Discriminating Similar Languages. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 212–220, Osaka, Japan, 2016. find
37. Adrien Barbaresi. Discriminating between Similar Languages using Weighted Subword Features. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 184–189, Valencia, Spain, 2017. find
38. Utsab Barman, Amitava Das, Joachim Wagner, and Jennifer Foster. Code Mixing: A Challenge for Language Identification in the Language of Social Media. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 127–132, Doha, Qatar, 2014a. find
39. Utsab Barman, Joachim Wagner, Grzegorz Chrupala, and Jennifer Foster. DCU-UVT: Word-Level Language Classification with Code-Mixed Data. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 13–23, Doha, Qatar, 2014b. find
40. Angelo Basile, Gareth Dwyer, Maria Medvedeva, Josine Rawee, Hessel Haagsma, and Malvina Nissim. N-GrAM: New Groningen Author-profiling Model—Notebook for PAN at CLEF 2017. In Working Notes Papers of CLEF 2017 Evaluation Labs and Workshop, Dublin, Ireland, 2017. find
41. Eleanor Olds Batchelder. A Learning Experience: Training an Artificial Neural Network to Discriminate Languages. Technical report, 1992. find
42. Eda Baykan, Monika Henzinger, and Ingmar Weber. Web page language identification based on URLs. In Proceedings of the 34th International Conference on Very Large Data Bases (VLDB 2008), pages 176–187, Auckland, New Zealand, 2008. find
43. Kenneth R. Beesley. Language Identifier: A Computer Program for Automatic Natural-Language Identification of On-line Text. In Proceedings of the 29th Annual Conference of the American Translators Association: Languages at Crossroads, pages 47–54, Seattle, USA, 1988. find
44. Božo Bekavac, Kristina Kocijan, and Marko Tadić. Near Language Identification using NooJ. In Formalising Natural Languages with NooJ 2014: Selected papers from the NooJ 2014 International Conference, pages 152–166. find
45. Cambridge Scholars Publishing, Sassari, Italy, 2014. find
46. Dario Benedetto, Emanuele Caglioti, and Vittorio Loreto. Language Trees and Zipping. Physical Review Letters, 88(4), 2002. find
47. Shane Bergsma, Paul McNamee, Mossaab Bagdouri, Clayton Fink, and Theresa Wilson. Language Identification for Creating Language-specific Twitter Collections. In Proceedings of the Second Workshop on Language in Social Media (LSM2012), pages 65–74, Montréal, Canada, 2012. find
48. Yves Bestgen. Improving the Character Ngram Model for the DSL Task with BM25 Weighting and Less Frequently Used Feature Sets. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 115–123, Valencia, Spain, 2017. find
49. Aditya Bhargava and Grzegorz Kondrak. Language Identification of Names with SVMs. In Proceedings of Human Language Technologies: The 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), pages 693–696, Los Angeles, California, USA, Ju… find
50. Rupal Bhargava, Yashvardhan Sharma, Shubham Sharma, and Abhinav Baid. Query Labeling for Indic Languages Using a Hybrid Approach. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2015), pages 42–44, Gandhinagar, India, 2015. find
51. S. Nagesh Bhattu and Vadlamani Ravi. Language Identification in Mixed Script Social Media Text. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2015), pages 39–31, Gandhinagar, India, 2015. find
52. Chris Biemann and Sven Teresniak. Disentangling from Babylonian confusion — Unsupervised Language Identification. In Alexander Gelbukh, editor, Proceedings of the 6th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2005), pages 773–784, Mexico City, Mex… find
53. Enikö Beatrice Bilcu and Jaakko Astola. A Hybrid Neural Network for Language Identification from Text. In Proceedings of the 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing, pages 253–258, Maynooth, Ireland, 2006. find
54. Arnold Binas. Markovian Time Series Models for Language Identification. Project Report, 2005. find
55. Johannes Bjerva. Byte-based Language Identification with Deep Convolutional Networks. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 119–126, Osaka, Japan, 2016. find
56. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993–1022, 2003. find our card
57. Su Lin Blodgett, Johnny Tian-Zheng Wei, and Brendan O’Connor. A Dataset and Classifier for Recognizing Social Media English. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 56–61, Copenhagen, Denmark, 2017. find
58. Victoria Bobicev. Discriminating between Similar Languages Using PPM. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 59–65, Hissar, Bulgaria, 2015. find
59. Gerrit Botha, Victor Zimu, and Etienne Barnard. Text-based Language Identification for South African Languages. Transactions of the South African Institute of Electrical Engineers, 98(4):141–148, 2007. find
60. Gerrit Reinier Botha. Text-Based Language Identification for The South African Languages. Master’s thesis, University of Pretoria, Hatfield, Pretoria, South Africa, 2008. find
61. Gerrit Reinier Botha and Etienne Barnard. Factors that Affect the Accuracy of Text-based Language Identification. In J. R. Tapamo and F. Nicolls, editors, Proceedings of the Eighteenth Annual Symposium of the Pattern Recognition Association of South Africa, pages 7–12, Pietermaritzburg, South Africa… find
62. Gerrit Reinier Botha and Etienne Barnard. Factors that Affect the Accuracy of Text-based Language Identification. Computer Speech and Language, 26(5):307–320, October 2012. find
63. Houda Bouamor, Nizar Habash, and Kemal Oflazer. A Multidialectal Parallel Corpus of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1240–1245, Reykjavik, Iceland, 2014. find
64. Matko Bošnjak, Eduarda Mendes Rodrigues, and Luis Sarmento. Robust Language Identification with RapidMiner: A Text Mining Use Case. In Markus Hofmann and Ralf Klinkenberg, editors, RapidMiner: Data Mining Use Cases and Business Analytics Applications, pages 213–239. Chapman and Hall/CRC, 2013. find
65. Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. Class-Based n-gram Models of Natural Language. Computational Linguistics, 18(4): 467–479, 1992. find
66. Ralf D. Brown. Finding and Identifying Text in 900+ Languages. Digital Investigation, 9: S34–S43, 2012. find
67. Ralf D. Brown. Selecting and Weighting N-grams to Identify 1100 Languages. In Proceedings of the 16th International Conference on Text, Speech and Dialogue (TSD 2013), pages 475–483, Plzeň, Czech Republic, 2013. find
68. Ralf D. Brown. Non-linear Mapping for Improved Identification of 1300+ Languages. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pages 627–632, Doha, Qatar, 2014. find
69. Brian O. Bush. Language Identification of Tweets Using LZW Compression. In 3rd Pacific Northwest Regional NLP Workshop (NW-NLP 2014), Redmond, USA, 2014. find
70. Paul van Cann. Dialect Identification on Twitter: A Research About the Detection of the Limburgian Dialect from Twitter messages. Master’s thesis, University of Tilburg, 2015. find
71. Simon Carter, Wouter Weerkamp, and Manos Tsagkias. Microblog Language Identification: Overcoming the Limitations of Short, Unedited and Idiomatic text. Language Resources and Evaluation, 47(1):195–215, 2013. find
72. Dayvid Castro, Ellen Souza, and Adriano L. I. de Oliveira. Discriminating between Brazilian and European Portuguese National Varieties on Twitter Texts. In Proceedings of the 5th Brazilian Conference on Intelligent Systems (BRACIS 2016), pages 265–270, Recife, Pernambuco, Brazil, 2016. IEEE. find
73. Dayvid W. Castro, Ellen Souza, Douglas Vitório, Diego Santos, and Adriano L. I. Oliveira. Smoothed N-gram Based Models for Tweet Language Identification: A Case Study of the Brazilian and European Portuguese National Varieties. Applied Soft Computing, 61: 1160–1172, 2017. find
74. William B. Cavnar and John M. Trenkle. N-Gram-Based Text Categorization. In Proceedings of SDAIR-94, Third Annual Symposium on Document Analysis and Information Retrieval, pages 161–175, Las Vegas, USA, 1994. find
75. Jordan Cazamias, Chinmayi Dixit, and Martina Marek. Large-Scale Language Classification - Writing a Detector for 200 Languages on Twitter. Stanford course report, 2015. find
76. Çagri Çöltekin and Taraka Rama. Discriminating Similar Languages: Experiments with Linear SVMs and Neural Networks. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 15–24, Osaka, Japan, 2016. find
77. Çagri Çöltekin and Taraka Rama. Tübingen System in VarDial 2017 Shared Task: Experiments with Language Identification and Cross-lingual Parsing. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 146–155, Valencia, Spain, 2017. find
78. Ebru Celikel. Language Discrimination via PPM Model. In Henry Selvaraj and Pradip K. Srimani, editors, Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC’05), volume 1, pages 57–62, Las Vegas, Nevada, USA, 2005. find
79. Hakan Ceylan and Yookyung Kim. Language Identification of Search Engine Queries. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 1066–1074, Suntec, Singapore, 2009. doi: 10.3115… find
80. Arunavha Chanda, Dipankar Das, and Chandan Mazumdar. Unraveling the English-Bengali Code-Mixing Phenomenon. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 80–89, Austin, TX, USA, 2016a. find
81. Arunavha Chanda, Dipankar Das, and Chandan Mazumdar. Columbia-Jadavpur submission for EMNLP 2016 Code-Switching Workshop Shared Task: System Description. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 112–115, Austin, TX, USA, 2016b. find
82. Joseph Chee Chang and Chu-Cheng Lin. Recurrent-neural-network for Language Detection on Twitter Code-Switching Corpus. arXiv preprint, arXiv:1412.4314, 2014. — [arXiv:1412.4314](https://arxiv.org/abs/1412.4314) find
83. Stanley F. Chen and Joshua Goodman. An empirical study of smoothing techniques for language modeling. Computer Speech and Language, 13(4):359–394, 1999. find
84. Stanley F Chen and Benoı̂t Maison. Using Place Name Data to Train Language Identification Models. In 8th European Conference on Speech Communication and Technology EUROSPEECH 2003 - INTERSPEECH 2003, pages 1349–1352, Geneva, Switzerland, 2003. find
85. Tianqi Chen and Carlos Guestrin. XGBoost: Reliable Large-sclae Tree Boosting System. arXiv preprint, 1603.02754:1–6, 2016. find
86. Yining Chen, Jiali You, Min Chu, Yong Zhao, and Jinlin Wang. Identifying Language Origin of Person Names with N-grams of Different Units. In Proceedings of the 2006 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2006), volume 1, pages 729–732, Toulouse, France, 2006… find
87. Yew Choong Chew, Yoshiki Mikami, Chandrajith Ashuboda Marasinghe, and S. Turrance Nandasara. Optimizing n-gram Order of an n-gram Based Language Identification Algorithm for 68 Written Languages. International Journal on Advances in ICT for Emerging Regions (ICTer), 02(02):21–28, 2009. find
88. Yew Choong Chew, Yoshiki Mikami, and Robin Lee Nagano. Language Identification of Web Pages Based on Improved N-gram Algorithm. International Journal of Computer Science Issues, 8(3):47–58, 2011. find
89. Gokul Chittaranjan, Yogarshi Vyas, Kalika Bali, and Monojit Choudhury. Word-level Language Identification using CRF: Code-switching Shared Task Report of MSR India System. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 73–79, Doha, Qatar, 2014. find
90. Kenneth Church. Stress Assignment in Letter to Sound Rules for Speech Synthesis. In Proceedings of the 23rd Annual Meeting of the Association for Computational Linguistics, pages 246–253, Chicago, Illinois, USA, 1985. find
91. Andre Cianflone and Leila Kosseim. N-gram and Neural Language Models for Discriminating Similar Languages. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 243–250, Osaka, Japan, 2016. find
92. Alina Maria Ciobanu and Liviu P Dinu. A Computational Perspective on the Romanian Dialects. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 3281–3285, Portorož, Slovenia, may 2016. find
93. Alina Maria Ciobanu, Marcos Zampieri, Shervin Malmasi, and Liviu P Dinu. Including Dialects and Language Varieties in Author Profiling. arXiv preprint, arXiv:1707.00621, 2017. — [arXiv:1707.00621](https://arxiv.org/abs/1707.00621) find
94. Simon Clematide and Peter Makarov. CLUZH at VarDial GDI 2017: Testing a Variety of Machine Learning Tools for the Classification of Swiss German Dialects. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 170–177, Valencia, Spain, 2017. find
95. Michael Clyne. Pluricentric Languages: Different Norms in Different Nations. CRC Press, Boca Raton, USA, 1992. find
96. Peter Constable and Gary Simons. Language identification and IT: Addressing Problems of Linguistic Diversity on a Global Scale. SIL Electronic Working Papers 2000-001, SIL International, Dallas, USA, 2000. find
97. Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest. Introduction to Algorithms. The MIT Press, Cambridge, Massachusetts, 1990. find
98. Ryan Cotterell and Chris Callison-Burch. A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic. In Proceedings of the 9th Language Resources and Evaluation Conference (LREC 2014), Reykjavik, Iceland, 2014. find
99. Jim Cowie, Yevgeny Ludovik, and Ron Zacharski. Language Recognition for Mono- and Multi-lingual Documents. In Proceedings of the VexTal Conference, pages 209–214, Venice, Italy, 1999. find
100. Mathias Creutz. Unsupervised Segmentation of Words Using Prior Distributions of Morph Length and Frequency. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL 2003), volume 1, pages 280–287, Sapporo, Japan, 2003. find
101. Marcelo Criscuolo and Sandra Maria Aluı́sio. Discriminating between Similar Languages with Word-level Convolutional Neural Networks. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 124–130, Valencia, Spain, 2017. find
102. Joaquim Ferreira da Silva and Gabriel Pereira Lopes. Identification of Document Language is Not yet a Completely Solved Problem. In Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation, 2006 and International Conference on Intelligent Agents… find
103. Joaquim Ferreira da Silva and Gabriel Pereira Lopes. Identification of Document Language in Hard Contexts. In ACM-SIGIR 2006 New Directions in Multilingual Information Access Proceedings of the Workshop, pages 40–48, Seattle, USA, 2006b. find
104. Joaquim Ferreira da Silva and Gabriel Pereira Lopes. Using Covariance as a Similarity Measure for Document Language Identification in Hard Contexts. Pliska Studia Mathematica Bulgarica, 18:341–360, 2007. find
105. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. Verifiably Effective Arabic Dialect Identification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pages 1465–1468, Doha, Qatar, 2014. find
106. Amitava Das and Björn Gambäck. Code-Mixing in Social Media Text: The Last Language Identification Frontier? Traitement Automatique des Langues, 54(3):41–64, 2013. find
107. Amitava Das and Björn Gambäck. Identifying Languages at the Word Level in Code-Mixed Indian Social Media Text. In Proceedings of the 11th International Conference on Natural Language Processing, pages 169–178, Goa, India, 2014. find
108. Franca Debole and Fabrizio Sebastiani. An Analysis of the Relative Hardness of Reuters-21578 Subsets. Journal of the American Society for Information Science and technology, 56(6):584–596, 2005. find
109. Persi Diaconis and Ronald L. Graham. Spearmam’s Footrule as a Measure of Disarray. Journal of the Royal Statistical Society. Series B (Methodological), 39(2):262–268, 1977. find
110. Sascha Diwersy, Stefan Evert, and Stella Neumann. A Weakly Supervised Multivariate Approach to the Study of Language Variation. In Benedikt Szmrecsanyi and Bernhard Wälchli, editors, Aggregating Dialectology, Typology, and Register Analysis. Linguistic Variation in Text and Speech. De Gruyter, Berl… find
111. Nina Dongen. Analysis and Prediction of Dutch-English Code-switching in Dutch Social Media Messages. Master’s thesis, Universiteit van Amsterdam, Amsterdam, Netherlands, 2017. find
112. Yerai Doval, David Vilares, and Jesús Vilares. Automatic Language Identification in Twitter: Adapting State-of-the-Art Identifiers to the Iberian Context. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference of the Spanish Society for Natural Language Pr… find
113. Gabriel Doyle. Mapping Dialectal Variation by Querying Social Media. In Proceedings of the 14th Conference of the EACL (EACL 2014), pages 98–106, Gothenburg, Sweden, 2014. find
114. Ted Dunning. Statistical Identification of Language. Technical Report MCCS 940-273, Computing Research Laboratory, New Mexico State University, 1994. find
115. Sukanya Dutta, Tista Saha, Somnath Banerjee, and Sudip Kumar Naskar. Text Normalization in Code-Mixed Social Media Text. In 2nd International Conference on Recent Trends in Information Systems (ReTIS), pages 378–382, Kolkata, India, 2015. find
116. Bernardt Duvenhage, Mfundo Ntini, and Phala Ramonyai. Improved Text Language Identification for the South African Languages. In Proceedings of the 28th Annual Symposium of the Pattern Recognition Association of South Africa (PRASA 2017), Bloemfontein, South Africa, 2017. find
117. Hisham El-Shishiny, Alexander Troussov, D. J. McCloskey, Mayo Takeuchi, Alex Nevidomsky, and Pavel Volkov. Word Fragments Based Arabic Language Identification. In Proceedings from NEMLAR International Conference on Arabic Language Resources and Tools, Cairo, Egypt, 2004. find
118. Mohamed Eldesouki, Fahim Dalvi, Hassan Sajjad, and Kareem Darwish. QCRI DSL 2016: Spoken Arabic Dialect Identification Using Textual Features. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 221–226, Osaka, Japan, 2016. find
119. Heba Elfardy and Mona Diab. Token Level Identification of Linguistic Code Switching. In Martin Kay and Christian Boitet, editors, Proceedings of COLING 2012: Posters, pages 287–296, Mumbai, India, December 2012. find
120. The COLING 2012 Organizing Committee. find
121. Heba Elfardy and Mona Diab. Sentence Level Dialect Identification in Arabic. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 456–461, Sofia, Bulgaria, 2013. find
122. Heba Elfardy, Mohamed Al-Badrashiny, and Mona Diab. Code Switch Point Detection in Arabic. In E. Métais, F. Meziane, M. Sararee, V. Sugumaran, and S. Vadera, editors, Proceedings of the Natural Language Processing and Information Systems - 18th International Conference on Applications of Natural La… find
123. Heba Elfardy, Mohamed Al-Badrashiny, and Mona Diab. AIDA: Identifying Code Switching in Informal Arabic Text. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 94–101, Doha, Qatar, 2014. find
124. Hani A. Elgabou and Dimitar Kazakov. Building Dialectal Arabic Corpora. In The Proceedings of the First Workshop on Human-Informed Translation and Interpreting Technology (HiT-IT), pages 52–57, Varna, Bulgaria, 2017. find
125. David Elworthy. Language Identification with Confidence Limits. In Proceedings of the 6th Annual Workshop on Very Large Corpora, pages 94–101, Montréal, Canada, 1998. find
126. Guy Emerson, Liling Tan, Susanne Fertmann, Alexis Palmer, and Michaela Regneri. SeedLing: Building and Using a Seed corpus for the Human Language Project. In Proceedings of the 2014 Workshop on the Use of Computational Methods in the Study of Endangered Languages, pages 77–85, Baltimore, USA, June 2… find
127. Ramy Eskander, Mohamed Al-Badrashiny, Nizar Habash, and Owen Rambow. Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 1–12, Doha, Qatar, 2014. find
128. Alexandra Espichán-Linares and Arturo Oncevay-Marcos. A Low-Resourced Peruvian Language Identification Model. In Juan Antonio Lossio-Ventura and Hugo Alatrista-Salas, editors, Proceedings of the 4th Annual International Symposium on Information Management and Big Data (SIMBig 2017), pages 57–63, Li… find
129. Alexandra Espichán-Linares and Arturo Oncevay-Marcos. Language Identification with Scarce Data: A Case Study from Peru. In Proceedings of the 4th Annual International Symposium on Information Management and Big Data (SIMBig 2017), Revised Selected Papers, Lima, Peru, 2018. find
130. Raül Fabra-Boluda, Francisco Rangel, and Paolo Rosso. NLEL UPV Autoritas participation at Discrimination between Similar Languages DSL 2015 Shared Task. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 52–58, Hissa… find
131. Hector-Hugo Franco-Penya and Liliana Malmani Sanchez. Tuning Bayes Baseline for Dialect Detection. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 227–234, Osaka, Japan, 2016. find
132. Marc Franco-Salvador, Francisco Rangel, Paolo Rosso, Mariona Taulé, and M. Antònia Martı́t. Language Variety Identification Using Distributed Representations of Words and Documents. In Proceedings of the 6th International Conference of the CLEF Association (CLEF’15): Experimental IR Meets Multilin… find
133. Marc Franco-Salvador, Paolo Rosso, and Francisco Rangel. Distributed Representations of Words and Documents for Discriminating Similar Languages. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 11–16, Hissar, Bulga… find
134. Marc Franco-Salvador, Greg Kondrak, and Paolo Rosso. Bridging the Native Language and Language Variety Identification Tasks. In Proceedings of the 21st International Conference Knowledge-Based and Intelligent Information and Engineering Systems (KES-2017), volume 112, pages 1554–1561, Marseille, Fra… — [doi:10.1016/j.procs.2017.08.068](https://doi.org/10.1016/j.procs.2017.08.068) find
135. Marc Franco-Salvador, Nataliia Plotnikova, Neha Pawar, and Yassine Benajiba. Subword-based Deep Averaging Networks for Author Profiling – Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs… find
136. Yoav Freund and Robert E. Schapire. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55: 119–139, 1997. find
137. Pablo Gamallo, Marcos Garcia, and Susana Sotelo. Comparing Ranking-based and Naive Bayes Approaches to Language Detection on Tweets. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference of the Spanish Society for Natural Language Processing (SEPLN 2014),… find
138. Pablo Gamallo, José Ramom Pichel, Iñaki Alegria, and Manex Agirrezabal. Comparing two Basic Methods for Discriminating Between Similar Languages and Varieties. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 170–177, Osaka, Japan, 2016. find
139. Pablo Gamallo, Jose Ramom Pichel, and Iñaki Alegria. A Perplexity-Based Method for Similar Languages Discrimination. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 109–114, Valencia, Spain, 2017. find
140. Archana Garg, Vishal Gupta, and Manish Jindal. A Survey of Language Identification Techniques and Applications. Journal of Emerging Technologies in Web Intelligence, 6 (4):388–400, 2014. find
141. Satanu Ghosh, Souvick Ghosh, and Dipankar Das. Labeling of Query Words using Conditional Random Field. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2015), pages 31–34, Gandhinagar, India, 2015. find
142. Emmanuel Giguet. Categorization according to Language: A step toward combining Linguistic Knowledge and Statistic Learning. In Proceedings of the International Workshop on Parsing Technologies (IWPT’95), Prague - Karlovy Vary, Czech Republic, 1995. find
143. Emmanuel Giguet. Méthode pour l’Analyse Automatique de Structures Formelles sur Documents Multilingues. PhD thesis, Université de Caen, 1998. find
144. Oluwapelumi Giwa. Language Identification for Proper Name Pronunciation. PhD thesis, North-West University, Vaal Triangle, 2016. find
145. Oluwapelumi Giwa and Marelie H. Davel. N-Gram based Language Identification of Individual Words. In Philip Robinson, editor, Proceedings of the 24th Annual Symposium of the Pattern Recognition Association of South Africa, pages 15–22, Johannesburg, South Africa, 2013. find
146. Oluwapelumi Giwa and Marelie H. Davel. Language Identification of Individual Words with Joint Sequence Models. In Proceedings of Interspeech 2014, Singapore, 2014. find
147. Moises Goldszmidt, Marc Najork, and Stelios Paparizos. Boot-Strapping Language Identifiers for Short Colloquial Postings. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2013), Part II, pages 95–111, Prague, Cze… find
148. Thomas Gottron and Nedim Lipka. A Comparison of Language Identification Approaches on Short, Query-style Texts. In Advances in Information Retrieval - Proceedings of the 32nd annual European Conference on Information Retrieval Research (ECIR 2010), pages 611–614, Milton Keynes, UK, 2010. Springer. find
149. Cyril Goutte and Serge Léger. Experiments in Discriminating Similar Languages. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 78–84, Hissar, Bulgaria, 2015. find
150. Cyril Goutte and Serge Léger. Advances in Ngram-based Discrimination of Similar Languages. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 178–184, Osaka, Japan, 2016. find
151. Cyril Goutte, Serge Léger, and Marine Carpuat. The NRC System for Discriminating Similar Languages. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects (VarDial), pages 139–145, Dublin, Ireland, 2014. find
152. Cyril Goutte, Serge Léger, Shervin Malmasi, and Marcos Zampieri. Discriminating Similar Languages: Evaluations and Explorations. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), may 2016. find
153. Gregory Grefenstette. Comparing Two Language Identification Schemes. In Proceedings of the 3rd International conference on Statistical Analysis of Textual Data (JADT 1995), Rome, Italy, 1995. find
154. Cyril Grouin, Dominic Forest, Lyne Da Sylva, Patrick Paroubek, and Pierre Zweigenbaum. Présentation et Résultats du Défi Fouille de Texte DEFT2010 Où et Quand un Article de Presse a-t-il Été Écrit? In Actes du sixième Défi Fouille de Textes, pages 3–14, Montpellier, France, 2011. find
155. Imène Guellil and Faiçal Azouaou. Arabic Dialect Identification With an Unsupervised Learning (based on a lexicon). Application case: Algerian Dialect. In Proceedings of the 2016 IEEE International Conference on Computational Science and Engineering and IEEE International Conference on Embedded an… find
156. Bhumika Gupta, Gaurav Bhatt, and Ankush Mittal. Language Identification and Disambiguation in Indian Mixed-Script. In Nikolaj Bjørner, Sanjiva Prasad, and Laxmi Parida, editors, Distributed Computing and Internet Technology, pages 113–121. Springer, 2016. find
157. Deepak Kumar Gupta, Shubham Kumar, and Asif Ekbal. Machine Learning Approach for Language Identification & Transliteration: Shared Task Report of IITP-TS. In Forum for Information Retrieval Evaluation (FIRE), pages 60–64, Bangalore, India, 2014. find
158. Gualberto A. Guzmán, Joseph Ricard, Jacqueline Serigos, Barbara Bullock, and Almeida Jacqueline Toribio. Moving Code-switching Research Toward More Empirically Grounded Methods. In Thierry Declerck and Sandra Kübler, editors, Proceedings of the Workshop on Corpora in the Digital Humanities (CDH 20… find
159. Helena Gómez-Adorno, Ilia Markov, Jorge Baptista, Grigori Sidorov, and David Pinto. Discriminating between Similar Languages Using a Combination of Typed and Untyped Character N-grams and Words. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 137–14… find
160. Nizar Habash and Fatiha Sadat. Arabic Preprocessing Schemes for Statistical Machine Translation. In Proceedings of the 7th Meeting of the North American Chapter of the Association for Computational Linguistics/Human Language Technologies Conference (HLT-NAACL06), pages 49–52, New York, NY, USA, 2006… find
161. Andrea Haţegan, Bogdan Bârligǎ, and Ioan Tǎbuş. Language Identification of Individual Words in a Multilingual Automatic Speech Recognition System. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2009), pages 4357–4360, Taipei, Taiwan, 2009.… find
162. Harald Hammarström. A Fine-Grained Model for Language Identification. In Proceedings of Improving Non English Web Searching (iNEWS-07) Workshop at SIGIR 2007, pages 14–20, Amsterdam, Netherlands, 2007. find
163. Amir Hamzah. Deteksi bahasa untuk dokumen teks berbahasa Indonesia. In Seminar Nasional Informatika 2010 (semnasIF 2010), pages A5–A13, Jakarta, Indonesia, 2010. find
164. Abualsoud Hanani, Aziz Qaroush, and Stephen Taylor. Classifying ASR Transcriptions According to Arabic Dialect. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 126–134, Osaka, Japan, 2016. find
165. Abualsoud Hanani, Aziz Qaroush, and Stephen Taylor. Identifying Dialects with Textual and Acoustic Cues. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 93–101, Valencia, Spain, 2017. find
166. Farheen Hanif, Fouzia Latif, and M. Sikandar Hayat Khiyal. Unicode Aided Language Identification across Multiple Scripts and Heterogeneous Data. Information Technology Journal, 6(4):534–540, 2007. find
167. Maimaitiyiming Hasimu and Wushouer Silamu. Three-stage Short Text Language Identification Algorithm. Journal of Digital Information Management, 15(6):354–371, December 2017. find
168. Katia Hayati. Language Identification on the World Wide Web. Master’s thesis, University of California Santa Cruz, Santa Cruz, California, USA, 2004. find
169. Sengül Bayrak Hayta, Hidayet Takçı, and Mübariz Eminli. Language Identification Based On N-Gram Feature Extraction Method By Using Classifiers. IU-Journal of Electrical and Electronics Engineering, 13(2):1629–1639, 2013. find
170. Junqing He, Zhen Zhang, Xuemin Zhao, Peijia Li, and Yonghong Yan. Similar Language Identification for Uyghur and Kazakh on Short Spoken Texts. In Proceedings of the 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC 2016), volume 2, pages 496–499, Hangzhou, Chin… find
171. Robert Hecht-Nielsen. Theory of the Backpropagation Neural Network. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 1989), pages I593–I605, Washington, DC, USA, 1989. doi: 10.1109/IJCNN.1989.118638. — [doi:10.1109/ijcnn.1989.118638](https://doi.org/10.1109/ijcnn.1989.118638) find
172. Peter Henrich. Language Identification for the Automatic Grapheme-to-phoneme Conversion of Foreign Words in a German Text-to-speech System. In First European Conference on Speech Communication and Technology, pages 2220–2223, Paris, France, 1989. find
173. Ondřej Herman, Vı́t Suchomel, Vı́t Baisa, and Pavel Rychlý. DSL Shared task 2016: Perfect Is The Enemy of Good Language Discrimination Through Expectation-Maximization and Chunk-based Language Model. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 1… find
174. Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Computation, 9:1735–1780, 1997. find
175. Nora Hollenstein and Noëmi Aepli. A Resource for Natural Language Processing of Swiss German Dialects. In Proceedings of GSCL, pages 108–109, 2015. find
176. Arthur S. House and Edward P. Neuburg. Toward Automatic Identification of the Language of an Utterance. I. Preliminary Methodological Considerations. The Journal of the Acoustical Society of America, 62(3):708–713, 1977. find
177. Chu-Ren Huang and Lung-Hao Lee. Contrastive Approach towards Text Source Classification based on Top-Bag-of-Word Similarity. In Proceedings of the 22nd Pacific Asia Conference on Language, Information and Computation, pages 404–410, Cebu City, Philippines, November 2008. find
178. Fei Huang. Improved Arabic Dialect Classification with Social Media Data. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pages 2118–2126, Lisbon, Portugal, 2015. URL https://aclweb.org/anthology/D/D15/D15-1254. ↗
179. Baden Hughes, Timothy Baldwin, Steven Bird, Jeremy Nicholson, and Andrew MacKinlay. Reconsidering Language Identification for Written Language Resources. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC 2006), pages 485–488, Genoa, Italy, 2006. find
180. Lluı́s-F. Hurtado, Ferran Pla, Mayte Giménez, and Emilio Sanchis. ELiRF-UPV at Tweet-LID: Twitter Language Identification. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference of the Spanish Society for Natural Language Processing (SEPLN 2014), pages 35–… find
181. Juha Häkkinen and Jilei Tian. N-gram and Decision Tree Based Language Identification for Written Words. In Conference Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU 2001), pages 335–338, Madonna di Campiglio, Italy, 2001. find
182. K. Indhuja, M. Indu, C. Sreejith, and P. C. Reghu Raj. Text Based Language Identification System for Indian Languages Following Devanagiri Script. International Journal of Engineering Reseach and Technology, 3(4):327–331, 2014. find
183. Radu Tudor Ionescu. Local rank distance. In Nikolaj Björner, Viorel Negru, Tetsuo Ida, Tudor Jebelean, Dana Petcu, Stephen Watt, and Daniela Zaharie, editors, Proceedings of the 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC 2013), pages 219–226, Ti… find
184. Radu Tudor Ionescu and Andrei M. Butnaru. Learning to Identify Arabic and German Dialects using Multiple Kernels. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 200–209, Valencia, Spain, 2017. find
185. Radu Tudor Ionescu and Marius Popescu. UnibucKernel: An Approach for Arabic Dialect Identification based on Multiple String Kernels. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 135–144, Osaka, Japan, 2016. find
186. Aaron Jaech, George Mulcaire, Shobhit Hathi, Mari Ostendorf, and Noah A. Smith. Hierarchical Character-Word Models for Language Identification. In Proceedings of the Fourth International Workshop on Natural Language Processing for Social Media, pages 84–93, Austin, TX, USA, 2016a. find
187. Aaron Jaech, George Mulcaire, Shobhit Hathi, Mari Ostendorf, and Noah A. Smith. A Neural Model for Language Identification in Code-Switched Tweets. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 60–64, Austin, TX, USA, 2016b. find
188. Devanshu Jain. DA-IICT in FIRE 2015 Shared Task on Mixed Script Information Retrieval. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2015), pages 53–56, Gandhinagar, India, 2015. find
189. Radwan Jalam. Apprentissage Automatique et Catégorisation de Textes Multilingues. PhD thesis, Université Lumière Lyon 2, 2003. find
190. Radwan Jalam and Olivier Teytaud. Kernel-based Text Categorization. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’01), volume 3, pages 1891–1896, Washington, DC, USA, 2001a. find
191. Radwan Jalam and Olivier Teytaud. Identification de la Langue et Catégorisation de Textes Basées sur les N-grammes. In Henri Briand and Fabrice Guillet, editors, Journées Francophones d’extraction et de gestion de connaissances (EGC’2001), pages 227–238, Nantes, France, 2001b. find
192. Heidi Jauhiainen, Tommi Jauhiainen, and Krister Lindén. The Finno-Ugric Languages and The Internet Project. Septentrio Conference Series, 0(2):87–98, 2015a. ISSN 2387-3086. doi: 10.7557/5.3471. URL http://septentrio.uit.no/index.php/SCS/article/view/3471. — [doi:10.7557/5.3471](https://doi.org/10.7557/5.3471) ↗
193. Tommi Jauhiainen. Tekstin kielen automaattinen tunnistaminen. Master’s thesis, University of Helsinki, Helsinki, 2010. find
194. Tommi Jauhiainen, Heidi Jauhiainen, and Krister Lindén. Discriminating Similar Languages with Token-based Backoff. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects, LT4VarDial ’15, pages 44–51, Hissar, Bulgaria, 2015b. find
195. Tommi Jauhiainen, Krister Lindén, and Heidi Jauhiainen. Language Set Identification in Noisy Synthetic Multilingual Documents. In Proceedings of the Computational Linguistics and Intelligent Text Processing 16th International Conference, CICLing 2015, pages 633– 643, Cairo, Egypt, 2015c. find
196. Tommi Jauhiainen, Krister Lindén, and Heidi Jauhiainen. HeLI, a Word-Based Backoff Method for Language Identification. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 153–162, Osaka, Japan, 2016. find
197. Tommi Jauhiainen, Krister Lindén, and Heidi Jauhiainen. Evaluation of Language Identification Methods Using 285 Languages. In Proceedings of the 21st Nordic Conference on Computational Linguistics (NoDaLiDa 2017), pages 183–191, Gothenburg, Sweden, 2017a. Linköping University Electronic Press. find
198. Tommi Jauhiainen, Krister Lindén, and Heidi Jauhiainen. Evaluating HeLI with Non-Linear Mappings. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 102–108, Valencia, Spain, 2017b. find
199. Harsh Jhamtani, Suleep Kumar Bhogi, and Vaskar Raychoudhury. Word-level Language Identification in Bi-lingual Code-switched Texts. In 28th Pacific Asia Conference on Language, Information and Computation, pages 348–357, Phuket, Thailand, 2014. find
200. Jay J. Jiang and David W. Conrath. Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy. In Proceedings of International Conference on Research on Computational Linguistics (ROCLING), Taiwan, 1997. find
201. Taeho Jo. Neural Text Categorizer for Exclusive Text Categorization. Journal of Infomra-tion Processing Systems, 4:77–86, 2008. find
202. Hardik Joshi, Apurva Bhatt, and Honey Patel. Transliterated Search using Syllabification Approach. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2013), New Delhi, India, 2013. find
203. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of Tricks for Efficient Text Classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2017), volume 2, pages 427–431, Valencia, Spain, 2017. find
204. Pierre Jourlin. Entity Recognition and Language Identification with FELTS. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and Workshop, Dublin, Ireland, September 2017. CEUR-WS.org. URL http://ceur-ws.org/Vol-1866/. ↗
205. P. Juola. Language Identification, Automatic. In Encyclopedia of Language and Linguistics, volume 6, pages 508—-510. Elsevier, Amsterdam, Netherlands, 2006. find
206. David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. Incorporating Dialectal Variability for Socially Equitable Language Identification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Short Papers), volume 2, pages 51–57, Vancouver, Canada, 2017. find
207. Said Kadri and Abdelouahab Moussaoui. An Effective Method to Recognize the Language of a Text in a Collection of Multilingual Documents. In Proceedings of the International Conference on Electronics, Computer and Computation (ICECCO 2013), pages 208–211, Ankara, Turkey, 2013. find
208. Slava M. Katz. Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer. IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-35(3):400–401, March 1987. find
209. Thomas Kerwin. Classification of Natural Language Based on Character Frequency. Ohio Supercomputer Center, 2006. find
210. Guillaume Kheng, Laporte Léa, and Michael Granitzer. INSA LYON and UNI PASSAU’s participation at PAN@CLEF’17: Author Profiling task - Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and… find
211. Gen-Itiro Kikui. Identifying the Coding System and Language of Online Documents on the Internet. In Proceedings of the 16th International Conference on Computational Linguistics (COLING ’96), pages 652–657, Copenhagen, Denmark, 1996. find
212. Seungbeom Kim and Jongsoo Park. Automatic Detection of Character Encoding and Language. Technical report, Stanford University, 2007. find
213. Ben King and Steven Abney. Labeling the Languages of Words in Mixed-Language Documents using Weakly Supervised Methods. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1110–1119, Atlanta, USA, Ju… find
214. Ben King, Dragomir Radev, and Steven Abney. Experiments in Sentence Language Identification with Groups of Similar Languages. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 146–154, Dublin, Ireland, 2014a. find
215. Josh King and Jon Dehdari. An N-gram Based Language Identification System. The Ohio State University, 2008. find
216. Levi King, Eric Baucom, Timur Gilmanov, Sandra Kübler, Daniel Whyatt, Wolfgang Maier, and Paul Rodrigues. The IUCL+ System: Word-Level Language Identification via Extended Markov Models. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 102–106, Doha, Qatar,… find
217. Levi King, Sandra Kübler, and Wallace Hooper. Word-level language identification in The Chymistry of Isaac Newton. Digital Scholarship in the Humanities, 30(4):532–540, 2015. find
218. Tom Kocmi and Ondr̆ej Bojar. LanideNN: Multilingual Language Identification on Character Window. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Long Papers, volume 1, pages 927–936, Valencia, Spain, 2017. find
219. Don Kodiyan, Florin Hardegger, Stephan Neuhaus, and Mark Cieliebak. Author Profiling with Bidirectional RNNs using Attention with GRUs - Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs a… find
220. Stasinos Konstantopoulos. What’s in a Name? In Proceedings of the 2007 Conference on Recent Advances in Natural Language Processing (RANLP-07), Borovets, Bulgaria, 2007. find
221. Anett Kralisch and Thomas Mandl. Barriers to Information Access Across Languages on the Internet: Network and Language Effects. In Proceedings of the 39th Annual Hawaii International Conference on System Sciences, volume 3, page 54b, Kauai, USA, 2006. find
222. Canasai Kruengkrai, Prapass Srichaivattana, Virach Sornlertlamvanich, and Hitoshi Isahara. Language identification based on string kernels. In Proceedings of the 5th International Symposium on Communications and Information Technologies (ISCIT-2005), volume 2, pages 896–899, Beijing, China, 2005. find
223. Stan Kulikowski. Language Identification of Short Texts. Newsgroup article, Educational Research and Development Center, The University of West Florida, 1991. find
224. Rahul Venkatesh RM Kumar, Anand M Kumar, and KP Soman. AmritaCEN NLP @ FIRE 2015 Language Identification for Indian Languages in Social Media Text. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2015), pages 28–30, Gandhinagar, India, 2015. find
225. Gustavo Laboreiro, Matko Bošnjak, Luı́s Sarmento, Eduarda Mendes Rodrigues, and Eugénio Oliveira. Determining Language Variant in Microblog Messages. In Proceedings of the 28th Annual ACM Symposium on Applied Computing (SAC’13), pages 902–907, Coimbra, Portugal, 2013. find
226. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, pages 282– 289, San Francisco, CA, USA, 2001. Morgan Kauf… find
227. Priyadarshini Lamabam and Kunal Chakma. A Language Identification System for Code-Mixed English-Manipuri Social Media Text. In Proceedings of the IEEE International Conference on Engineering and Technology (ICETECH 2016), pages 79–83, Coimbatore, Tamil Nadu, India, 2016. find
228. Stefan Langer. Natural Languages and the World Wide Web. Bulletin de Linguistique Appliquée et Générale (BULAG), 26:89–100, 2001. find
229. Sebastian Leidig. Single and Combined Features for the Detection of Anglicisms in German and Afrikaans. Bachelor’s Thesis, Karlsruhe Institute of Technology, 2014. find
230. William D Lewis and Fei Xia. Developing ODIN: A Multilingual Repository of Annotated Language Data for Hundreds of the World’s Languages. Literary and Linguistic Computing, 25(3):303–319, 2010. find
231. Shanjian Li and Katsuhiko Momoi. A Composite Approach to Language/Encoding Detection. In Nineteenth International Unicode Conference (IUC19), San Jose, California, USA, 2001. find
232. Yitong Li, Trevor Cohn, and Timothy Baldwin. What’s in a domain? learning domain-robust text representations using adversarial training. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics — Human Language Technologies (NAACL HLT 2018)… find
233. Chu-Cheng Lin, Waleed Ammar, Lori Levin, and Chris Dyer. The CMU Submission for the Shared Task on Language Identification in Code-Switched Data. In Proceedings of First Workshop on Computational Approaches to Code Switching, pages 80–86, Doha, Qatar, 2014. find
234. Dekang Lin. An Information-Theoretic Definition of Similarity. In Jude W. Shavlik, editor, Proceedings of the 15th International Conference on Machine Learning (ICML-98), pages 296–304, Madison, Wisconsin, USA, 1998. find
235. Yoseph Linde, Andrés Buzo, and Robert M. Gray. An Algorithm for Vector Quantizer Design. IEEE Transactions on Communications, COM-28(1):84–95, 1980. find
236. Wang Ling, Guang Xiang, Chris Dyer, Alan Black, and Isabel Trancoso. Microblogs as Parallel Corpora. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 176–186, Sofia, Bulgaria, August 2013. find
237. Rafael Dueire Lins and Paulo Jr. Gonçalves. Automatic Language Identification of Written Texts. In Proceedings of the 2004 ACM symposium on Applied Computing (SAC 2004), pages 1128–1133, Nicosia, Cyprus, 2004. find
238. Nick Littlestone. Learning Quickly When Irrelevant Attributes Abound: A New Linear-threshold Algorithm. Machine learning, 2(4):285–318, 1987. find
239. Nikola Ljubešić and Denis Kranjcić. Discriminating between VERY Similar Languages among Twitter Users. In Proceedings of the 9th Language Technologies Conference, pages 90–94, Ljubljana, Slovenia, 2014. find
240. Nikola Ljubešić and Denis Kranjcić. Discriminating Between Closely Related Languages on Twitter. Informatica, 39, 2015. find
241. Nikola Ljubešić, Nives Mikelić, and Damir Boras. Language Identification: How to Distinguish Similar Languages? In Proceedings of the 29th International Conference on Information Technology Interfaces (ITI 2007), pages 541–546, Cavtat/Dubrovnik, Croatia, 2007. find
242. Man Lu and Moustafa Mohamed. LAHGA: Arabic Dialect Classifier. Unpublished Report. December 2011. find
243. Yevgeny Ludovik and Ron Zacharski. Multilingual Document Language Recognition for Creating Corpora. Technical report, New Mexico State University, 1999. find
244. Marco Lui. Generalized Language Identification. PhD thesis, The University of Melbourne, 2014. find
245. Marco Lui and Timothy Baldwin. Cross-domain Feature Selection for Language Identification. In Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP 2011), volume 1, pages 553—-561, Chiang Mai, Thailand, 2011. find
246. Marco Lui and Timothy Baldwin. langid.py: An Off-the-shelf Language Identification Tool. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012) Demo Session, pages 25–30, Jeju, Republic of Korea, 2012. find our card
247. Marco Lui and Timothy Baldwin. Accurate Language Identification of Twitter Messages. In Proceedings of the 5th Workshop on Language Analysis for Social Media (LASM), pages 17–25, Gothenburg, Sweden, 2014. Association for Computational Linguistics. find
248. Marco Lui and Paul Cook. Classifying English Documents by National Dialect. In Proceedings of Australasian Language Technology Association Workshop 2013 (ALTA 2013), pages 5–15, Brisbane, Australia, December 2013. find
249. Marco Lui, Jey Han Lau, and Timothy Baldwin. Automatic Detection and Language Identification of Multilingual Documents. Transactions of the Association for Computational Linguistics, 2:27–40, 2014a. ISSN 2307-387X. URL https://tacl2013.cs.columbia.edu/ojs/index.php/tacl/article/view/86. ↗
250. Marco Lui, Ned Letcher, Oliver Adams, Long Duong, Paul Cook, and Timothy Baldwin. Exploring Methods and Resources for Discriminating Similar Languages. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 129–138, Dublin, Ireland, August 2014… find
251. Shane MacNamara, Pádraig Cunningham, and John Byrne. Neural Networks for Language Identification: a Comparative Study. Information processing and management, 34(4): 395–403, 1998. find
252. Wolfgang Maier and Carlos Gómez-Rodrı́guez. Language Variety Identification in Spanish Tweets. In Proceedings of the EMNLP’2014 Workshop on Language Technology for Closely Related Languages and Language Variants (LT4CloseLang 2014), pages 25–35, Doha, Qatar, October 2014. Association for Computatio… find
253. Martin Majliš. Large Multilingual Corpus. Master’s thesis, Charles University in Prague, Prague, 2011. find
254. Martin Majliš. Yet Another Language Identifier. In Proceedings of the Student Research Workshop at the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 46–54, Avignon, France, 2012. find
255. Shervin Malmasi. Open-Set Language Identification. arXiv preprint, arXiv:1707.04817, 2017. — [arXiv:1707.04817](https://arxiv.org/abs/1707.04817) find
256. Shervin Malmasi and Mark Dras. Automatic Language Identification for Persian and Dari Texts. In Proceedings of the 14th Conference of the Pacific Association for Computational Linguistics, PACLING’15, pages 59–64, Bali, Indonesia, 2015a. find
257. Shervin Malmasi and Mark Dras. Language Identification using Classifier Ensembles. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects, LT4VarDial’15, pages 35–43, Hissar, Bulgaria, 2015b. find
258. Shervin Malmasi and Mark Dras. Feature Hashing for Language and Dialect Identification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 399–403, Vancouver, Canada, 2017. find
259. Shervin Malmasi and Marcos Zampieri. Arabic Dialect Identification in Speech Transcripts. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 106–113, Osaka, Japan, 2016. find
260. Shervin Malmasi and Marcos Zampieri. German Dialect Identification in Interview Transcriptions. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 164–169, Valencia, Spain, 2017a. find
261. Shervin Malmasi and Marcos Zampieri. Arabic Dialect Identification Using iVectors and ASR Transcripts. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 178–183, Valencia, Spain, 2017b. find
262. Shervin Malmasi, Eshrag Refaee, and Mark Dras. Arabic Dialect Identification using a Parallel Multidialectal Corpus. In Proceedings of the 14th Conference of the Pacific Association for Computational Linguistics, PACLING’15, pages 209–217, Bali, Indonesia, 2015. find
263. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. Discriminating Between Similar Languages and Arabic Dialect Identification: A Report on the Third DSL Shared Task. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dial… find our card
264. Soumik Mandal, Somnath Banerjee, Sudip Kumar Naskar, Paolo Rosso, and Sivaji Bandyopadhyay. Adaptive Voting in Multiple Classifier Systems for Word Level Language Identification. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2015), pages 49–52, Gandhinagar, India, 2015. find
265. Thomas Mandl, Margaryta Shramko, Olga Tartakovski, and Christa Womser-Hacker. Language Identification in Multi-lingual Web-Documents. In Proceedings of the 11th International Conference on Applications of Natural Language to Information Systems (NLDB 2006), pages 153–163, Klagenfurt, Austria, 2006. find
266. Jean-Christophe Marcadet, Volker Fischer, and Claire Waast-Richard. A Transformation-based Learning Approach to Language Identification for Mixed-lingual Text-to-speech Synthesis. In Proceedings of the 6th Interspeech and 9th European Conference on Speech Communication and Technology (EUROSPEECH), p… find
267. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2): 313–330, 1993. find our card
268. Ilia Markov, Helena Gómez-Adorno, and Grigori Sidorov. Language- and Subtask-Dependent Feature Selection and Classifier Parameter Tuning for Author Profiling— Notebook for PAN at CLEF 2017. In Working Notes Papers of CLEF 2017 Evaluation Labs and Workshop, Dublin, Ireland, 2017. find
269. Puji Martadinata, Bayu Distiawan Trisedya, Hisar Maruli Manurung, and Mirna Adriani. Building Indonesian Local Language Detection Tools Using Wikipedia Data. In Yohei Murakami and Donghui Lin, editors, Worldwide Language Service Infrastructure, pages 113–123. Springer, 2016. find
270. Matej Martinc, Iza Škrjanec, Katja Zupan, and Senja Pollak. PAN 2017: Author Profiling - Gender and Language Variety Prediction—Notebook for PAN at CLEF 2017. In Working Notes Papers of CLEF 2017 Evaluation Labs and Workshop, Dublin, Ireland, 2017. find
271. Bruno Martins and Mário J. Silva. Language Identification in Web Pages. In Proceedings of the 2005 ACM symposium on Applied computing, pages 764–768, Santa Fe, USA, 2005. find
272. Laura A. Mather. A Linear Algebra Approach to Language Identification. In E. V. Munson, editor, Proceedings of the 4th International Workshop Principles of Digital Document Processing (PODDP’98), pages 92–103, Saint Malo, France, 1998. find
273. Priyank Mathur, Arkajyoti Misra, and Emrah Budur. LIDE: Language Identification from Text Documents. arXiv preprint, arXiv:1701.03682, 2017. — [arXiv:1701.03682](https://arxiv.org/abs/1701.03682) find
274. Uwe F. Mayer. Bootstrapped Language Identification for Multi-site Internet Domains. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2012), pages 579–585, Beijing, China, 2012. find
275. Paul McNamee. Language Identification: A Solved Problem Suitable for Undergraduate Instruction. Journal of Computing Sciences in Colleges, 20(3):94–101, 2005. find
276. Paul McNamee. Language and Dialect Discrimination Using Compression-Inspired Language Models. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 195–203, Osaka, Japan, 2016. find
277. Maria Medvedeva, Martin Kroon, and Barbara Plank. When Sparse Traditional Models Outperform Dense Neural Networks: the Curious Case of Discriminating between Similar Languages. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 156–163, Valencia, Spain,… find
278. Iosu Mendizabal, Jeroni Carandell, and Daniel Horowitz. TweetSafa: Tweet Language Identification. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference of the Spanish Society for Natural Language Processing (SEPLN 2014), pages 21–25, Girona, Spain, 2014. find
279. Ismael Mendoza and Julia Mendelsohn. Exploring Techniques in Distinguishing Similar Languages. Stanford course project, 2017. find
280. Tomas Mikolov, Kai Chen, Greg Corrado, and Dean Jeffrey. Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781 [cs.CL], 2013. find
281. Rachel Mary Milne, Richard A. O’Keefe, and Andrew Trotman. A study in Language Identification. In Proceedings of the Seventeenth Australasian Document Computing Symposium, pages 88–95, Dunedin, New Zealand, 2012. find
282. Akshay Minocha and Francis M. Tyers. Subsegmental Language Detection in Celtic Language Text. In Proceedings of the First Celtic Language Techonology Workshop (CLTW 2014), pages 76–80, Dublin, Ireland, 2014. find
283. Yasuhide Miura, Tomoki Taniguchi, Motoki Taniguchi, Shotaro Misawa, and Tomoko Ohkuma. Using Social Networks to Improve Language Variety Identification with Neural Networks. In Proceedings of the 8th International Joint Conference on Natural Language Processing, pages 263–270, Taipei, Taiwan, 2017. find
284. Serguei A. Mokhov. Complete Complimentary Results Report of the MARF’s NLP Approach to the DEFT 2010 Competition. CoRR, abs/1006.3787, 2010a. — [arXiv:1006.3787](https://arxiv.org/abs/1006.3787) find
285. Serguei A Mokhov. A MARF Approach to DEFT 2010. In Proceedings of the 6th DEFT Workshop (DEFT’10), pages 35–49, 2010b. find
286. Giovanni Molina, Nicolas Rey-Villamizar, Thamar Solorio, Fahad AlGhamdi, Mahmoud Ghoneim, Abdelati Hawwari, and Mona Diab. Overview for the Second Shared Task on Language Identification in Code-Switched Data. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages… find
287. Avashlin Moodley. Language Identification With Decision Trees: Identification Of Individual Words In The South African Languages. Bachelor’s Thesis, University of South Africa, 2016. find
288. Abhinav Mukherjee, Anirudh Ravi, and Kaustav Datta. Mixed-script Query Labelling Using Supervised Learning and Ad Hoc Retrieval Using Sub Word Indexing. In FIRE ’14 Proceedings of the Forum for Information Retrieval, pages 86–90, Bangalore, India, 2014. find
289. Kavi Narayana Murthy and G. Bharadwaja Kumar. Language Identification from Small Text Samples. Journal of Quantitative Linguistics, 13(1):57–80, January 2006. find
290. Seppo Mustonen. Multiple Discriminant Analysis in Linguistic Problems. Statistical Methods in Linguistics, 4:37–44, 1965. find
291. Yeshwant K Muthusamy and A Lawrence Spitz. Automatic Language Identification. In Ronald Cole, Joseph Mariani, Hans Uszkoreit, Giovanni Battista Varile, Annie Zaenen, Antonio Zampolli, and Victor Zue, editors, Web Edition: Survey of the State of the Art in Human Language Technology, pages 314–317. Ca… find
292. Yukio Nakamura. Identification of Languages with Short Sample Texts – A Linguometric Study. Library and information science, 9:459–481, 1971. find
293. Choon-Ching Ng and Ali Selamat. Improved Letter Weighting Feature Selection on Arabic Script Language Identification. In Proceedings of the 1st Asian Conference on Intelligent Information and Database Systems (ACIIDS 2009), pages 150–154, Dong Hoi, Vietnam, 2009. IEEE. find
294. Dong Nguyen and Leonie Cornips. Automatic Detection of Intra-Word Code-Switching. In Proceedings of the 14th Annual SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 82–86, Berlin, Germany, 2016. find
295. Dong Nguyen and A. Seza Dogruöz. Word Level Language Identification in Online Multilingual Communication. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), pages 857–862, Seattle, USA, 2013. find
296. Shiho Nobesawa and Ikuo Tahara. Language Identification for Person Names Based on Statistical Information. In Proceedings of the 19th Pacific Asia Conference on Language, Information and Computation (PACLIC 19), number 1027 in Y05, Taipei, Taiwan, December 2005. find
297. Norzaidah Md Noh, Mohd Rusydi Abdul Talib, Azlin Ahmad, Shamimi A. Halim, and Azlinah Mohamed. Malay Language Document Identification Using BPNN. In Nikos E. Mastorakis, Anca Croitoru, Valentina Emilia Balas, Eduard Son, and Valeri Mladenov, editors, Proceedings of the 10th WSEAS international confe… find
298. Daisuke Okanohara and Jun’ichi Tsujii. Text Categorization with All Substring Features. In Proceedings of the 2009 SIAM International Conference on Data Mining, pages 838–846, Miami, USA, 2009. find
299. Rodrigo Ribeiro Oliveira and Rosalvo Ferreira de Oliveira Neto. Using Character N-grams and Style Features for Gender and Language Variety Classification – Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 201… find
300. Gorkem Ozbek, Itamar Rosenn, and Eric Yeh. Language Classification in Multilingual Documents. Technical report, Stanford University, 2006. find
301. Leonid Panich. Comparison of Language Identification Techniques. Bachelor’s Thesis, Heinrich Heine Universität Düsseldorf, 2015. find
302. Evangelos E. Papalexakis, Dong Nguyen, and A. Seza Dogruöz. Predicting Code-Switching in Multilingual Communication for Immigrant Communities. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 42–50, Doha, Qatar, 2014. find
303. Kosuru Pavan, Niket Tandon, and Vasudeva Varma. Addressing Challenges in Automatic Language Identification of Romanized Text. In Proceedings of 8th International Conference on Natural Language Processing (ICON-2010), Kharagpur, India, 2010. find
304. Timo Pawelka and Elmar Jürgens. Is This Code Written in English? A Study of the Natural Language of Comments and Identifiers in Practice. In Proceedings of the 31st International Conference on Software Maintenance and Evolution (ICSME), Bremen, Germany, 2015. find
305. Yves Peirsman, Dirk Geeraerts, and Dirk Speelman. The Automatic Identification of Lexical Variation Between Language Varieties. Natural Language Engineering, 16(04):469–491, October 2010. find
306. Fuchun Peng and Dale Schuurmans. Combining Naive Bayes and n-Gram Language Models for Text Classification. In Proceedings of the 25th European Conference on IR Research, Advances in Information Retrieval: (ECIR 2003), pages 335–350, Pisa, Italy, 2003. Springer Berlin Heidelberg. ISBN 978-3-540-36618… — [doi:10.1007/3-540-36618-0\](https://doi.org/10.1007/3-540-36618-0\) find
307. Fuchun Peng, Fangfang Feng, and Andrew McCallum. Chinese Segmentation and New Word Detection Using Conditional Random Fields. In Proceedings of the 20th International Conference on Computational Linguistics (COLING 2004), pages 562–568, Geneva, Switzerland, 2004. find
308. Gergely Pethö and Eszter Mózes. An N-gram-based Language Identification Algorithm for Variable-length and Variable-language Texts. Argumentum, 10:56–82, 2014. find
309. Tuan Pham and Dat Tran. VQ-based Written Language Identification. In Proceedings of the Seventh International Symposium on Signal Processing and Its Applications (ISSPA 2003), volume 1, pages 513–516, Paris, France, 2003. find
310. Wikus Pienaar and Dirk Snyman. Spelling Checker-based Language Identification for the Eleven Official South African Languages. In Proceedings of the 21st Annual Symposium of the Pattern Recognition Association of South Africa, pages 213–217, Stellenbosch, South Africa, 2010. find
311. Mario Piergallini, Rouzbeh Shirvani, Gauri S. Gautam, and Mohamed Chouikha. Word-Level Language Identification and Predicting Codeswitching Points in Swahili-English Language Data. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 21–29, Austin, TX, USA, 2016… find
312. Mario Piergallini, Rouzbeh Shirvani, Gauri S. Gautam, and Mohamed Chouikha. The Howard University System Submission for the Shared Task in Language Identification in Spanish-English Codeswitching. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 116–120, Aus… find
313. Ferran Pla and Lluı́s-F. Hurtado. Language Identification in Twitter: A Study Case of Multiclass and Multilabel Text Classification Problem. International Journal of Computational Linguistics and Applications, 6(1):135–150, 2015. find
314. Ferran Pla and Lluı́s-F. Hurtado. Language Identification of Multilingual Posts from Twitter: A Case Study. Knowledge and Information Systems, 51(3):965–989, 2017. find
315. Sergi Plaza Cagigós. Catdetect, a framework for detecting Catalan tweets. Bachelor thesis, Universitat de Lleida, 2017. find
316. Jordi Porta. Twitter Language Identification using Rational Kernels and its Potential Application to Sociolinguistics. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference of the Spanish Society for Natural Language Processing (SEPLN 2014), pages 17–20, G… find
317. Jordi Porta and José-Luis Sancho. Using Maximum Entropy Models to Discriminate between Similar Languages and Varieties. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 120–128, Dublin, Ireland, 2014. find
318. Adam Poulston, Zeerak Waseem, and Mark Stevenson. Using TF-ID n-gram and Word Embedding Cluster Ensembles for Author Profiling – Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and Works… find
319. Arjen Poutsma. Applying Monte Carlo Techniques to Language Identification. Language and Computers, 45(1):179–189, 2002. find
320. John M. Prager. Linguini: Language Identification for Multilingual Documents. In Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences (HICSS-32), Maui, USA, 1999. find
321. M. A. Nejla Qafmolla. Automatic Language Identification. European Journal of Language and Literature Studies, 7(1):140–150, 2017. find
322. Shenglan Qiao and Daniel Lévy. Similar Language Detection. Stanford University, 2015. find
323. Will Radford and Matthias Gallé. Discriminating Between Similar Languages in Twitter Using Label Propagation. arXiv preprint, arXiv:1607.05408, 2016. — [arXiv:1607.05408](https://arxiv.org/abs/1607.05408) find
324. K. A. Rafidha Rehiman, A. S. Keerthy, K. S. Lakshmi, and A. Sreekumar. A Language Identification and Conversion System for Malayalam to Ensure Security. In 3rd National Conference on Indian Language Computing (NCILC 2013), Cochin, Kerala, India, 2013. find
325. Khyathi C. Raghavi, Manoj Chinnakotla, and Manish Shrivastava. “Answer ka type kya he?” Learning to Classify Questions in Code-Mixed Language. In WWW’15 Companion Proceedings of the 24th International Conference on World Wide Web, pages 853–858, Florence, Italy, 2015. find
326. Abhinav Raj and Sankha Karfa. A List-searching Based Approach for Language Identification in Bilingual Text: Shared Task Report by Asterisk. In Working Notes of the Shared Task on Transliterated Search at Forum for Information Retrieval Evaluation FIRE ’14, Bangalore, India, 2014. find
327. Carlos Ramisch. N-gram Models for Language Detection. Technical Report, 2008. find
328. Bali Ranaivo-Malançon. Automatic Identification of Close Languages – Case study: Malay and Indonesian. ECTI Transactions on Computer and Information Technology, 2(2): 126–134, 2006. find
329. Bali Ranaivo-Malançon and Pek Kuan Ng. Language Identifier for Bahasa Malaysia and Bahasa Indonesia. In Proceedings of the 1st Malaysian Software Engineering Conference (MySEC’05), pages 257–259, Penang, Maylaysia, 2005. find
330. Francisco Rangel, Marc Franco-Salvador, and Paolo Rosso. A Low Dimensionality Representation for Language Variety Identification. In Proceedings of the 17th International Conference on Computational Linguistics and Intelligent Text Processing (CICLING), 2017a. find
331. Francisco Rangel, Paolo Rosso, Martin Potthast, and Benno Stein. Overview of the 5th Author Profiling Task at PAN 2017: Gender and Language Variety Identification in Twitter. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation… find
332. Prakash Ranjan, Bharathi Raja, Ruba Priyadharshini, and Rakesh Chandra Balabantaray. A Comparative Study on Code-Mixed Data of Indian Social Media vs Formal Text. In S. K. Niranjan and V. N. Manjunatha Aradhya, editors, Proceedings of the 2nd International Conference on Contemporary Computing and In… find
333. Morton David Rau. Language Identification by Statistical Analysis. Master’s thesis, Naval Postgraduate School, Monterey, 1974. find
334. Radim Rehůřek and Milan Kolkus. Language Identification on the Web: Extending the Dictionary Method. In Proceedings of the 10th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2009), pages 357–368, Mexico City, Mexico, 2009. find
335. Matthew Richardson and Pedro Domingos. Markov logic networks. Machine Learning, 62: 107–136, 2006. find
336. Shruti Rijhwani, Royal Sequeira, Monojit Choudhury, Kalika Bali, and Chandra Sekhar Maddila. Estimating Code-Switching on Twitter with a Novel Generalized Word-Level Language Detection Technique. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1971–1… find
337. Paul Rodrigues. Processing Highly Variant Language Using Incremental Model Selection. PhD thesis, Indiana University, 2012. find
338. Harald Romsdorfer and Beat Pfister. Text Analysis and Language Identification for Polyglot Text-to-speech Synthesis. Speech communication, 49(9):697–724, 2007. find
339. Mike Rosner and Paulseph-John Farrugia. A Tagging Algorithm for Mixed Language Identification in a Noisy Domain. In INTERSPEECH-2007, 8th Annual Conference of the International Speech Communication Association, pages 190–193, Antwerp, Belgium, 2007. find
340. Neil C. Rowe, Riqui Schwamm, and Simson L Garfinkel. Language Translation for File Paths. Digital Investigation, 10, 2013. find
341. Rishiraj Saha Roy, Monojit Choudhury, Prasenjit Majumder, and Komal Agarwal. Overview of the FIRE 2013 Track on Transliterated Search. In Prasenjit Majumder, Mandar Mitra, Maghulika Agrawal, and Parth Mehta, editors, Proceedings of the 5th Forum on Information Retrieval Evaluation (FIRE ’13), New De… find
342. Graham Russell and Guy Lapalme. Automatic Identification of Language and Encoding. Technical report, Laboratoire de Recherche Appliquée en Linguistique Informatique (RALI), Université de Montréal, 2003. find
343. Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. Automatic Identification of Arabic Language Varieties and Dialects in Social Media. In Shou-de Lin, Lun-Wei Ku, Erik Cambria, and Tsung-Ting Kuo, editors, Proceedings of the Second Workshop on Natural Language Processing for Social Media (SocialNL… find
344. Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. Automatic Identification of Arabic Dialects in Social Media. In Proceedings of the first international workshop on Social media retrieval and analysis (SoMeRA 2014), pages 35–40, Gold Coast, QLD, Australia, 2014b. ACM. find
345. Navanath Saharia. Phone-based Identification of Language in Code-mixed Social Network Data. Journal of Statistics and Management Systems, 20(4):565–574, 2017. find
346. Wael Salloum, Heba Elfardy, Linda Alamir-Salloum, Nizar Habash, and Mona Diab. Sentence Level Dialect Identification for Machine Translation System Selection. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 772–778, Baltimore… find
347. Younes Samih. Dialectal Arabic Processing Using Deep Learning. PhD thesis, Heinrich-Heine-Universität Düsseldorf, Düsseldorf, Germany, 2017. find
348. Younes Samih and Wolfgang Maier. Detecting Code-Switching in Moroccan Arabic Social Media. In Proceedings of the 4th International Workshop on Natural Language Processing for Social Media (SocialNLP 2016 IJCAI), New York City, USA, 2016. find
349. Younes Samih, Suraj Maharjan, Mohammed Attia, Laura Kallmeyer, and Thamar Solorio. Multilingual Code-switching Identification via LSTM Recurrent Neural Networks. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 50–59, Austin, TX, USA, 2016. find
350. Miguel A. Sanchez-Perez, Ilia Markov, Helena Gómez-Adorno, and Sidorov Grigori. Comparison of Character N-grams and Lexical Features on Author, Gender, and Language Variety Identification on the Same Spanish News Corpus. In Gareth J. F. Jones, Séamus Lawless, Julio Gonzalo, Liadh Kelly, Lorraine G… find
351. Kevin P Scannell. The Crúbadán Project: Corpus Building for Under-resourced Languages. In Building and Exploring Web Corpora: Proceedings of the 3rd Web as Corpus Workshop, pages 5–15, Louvain-la-Neuve, Belgium, 2007. find
352. Nils Schaetti. UniNE at CLEF 2017: TF-IDF and Deep-Learning for Author Profiling - Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and Workshop, Dublin, Ireland, September 2017. CEUR-WS.… find
353. Yves Scherrer and Owen Rambow. Word-based Dialect Identification with Georeferenced Rules. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010), pages 1151–1161, Massachusetts, USA, 2010. Association for Computational Linguistics. find
354. Sarah Schulz and Mareike Keller. Code-Switching Ubique Est - Language Identification and Part-of-Speech Tagging for Historical Mixed Text. In Proceedings of the 10th SIGHUM Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH 2016), pages 43–51, Berlin, Germ… find
355. Fabrizio Sebastiani. Machine Learning in Automated Text Categorization. ACM Computing Surveys (CSUR), 34(1):1–47, 2002. find
356. Frank Seifart and Roger Mundry. Quantitative Comparative Linguistics based on Tiny Corpora: N-gram Language Identification of Wordlists of Known and Unknown Languages from Amazonia and Beyond. Journal of Quantitative Linguistics, 22(3), 2015. find
357. Ali Selamat and Nicholas Akosu. Word-length Algorithm for Language Identification of Under-resourced Languages. Journal of King Saud University - Computer and Information Sciences, 28:457–469, 2016. find
358. Ali Selamat and Choon-Ching Ng. Arabic Script Documents Language Identifications Using Fuzzy ART. In David Al-Dabass, Steve Turner, Gary Tan, and Ajith Abraham, editors, Proceedings of the Second Asia International Conference on Modeling & Simulation (AMS 2008), pages 528–533, Kuala Lumpur, Malaysia… find
359. Ali Selamat, Choon-Ching Ng, and Yoshiki Mikami. Arabic Script Web Documents Language Identification Using Decision Tree-ARTMAP Model. In Proceedings of the International Conference on Convergence Information Technology (ICCIT 2007), pages 717–722, Gyeongju, Korea, 2007. IEEE. find
360. Royal Sequeira, Monojit Choudhury, Parth Gupta, Paolo Rosso, Shubham Kumar, Somnath Banerjee, Sudip Kumar Naskar, Sivaji Bandyopadhyay, Gokul Chittaranjan, Amitava Das, and Kunal Chakma. Overview of FIRE-2015 Shared Task on Mixed Script Information Retrieval. In Working Notes of the Forum for Inform… find
361. Saatvik Shah, Vaibhav Jain, Sarthak Jain, Anshul Mittal, Jatin Verma, Shubham Tripathi, and Dr. Rajesh Kumar. Hierarchical classification for Multilingual Language Identification and Named Entity Recognition. In Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2015), pages 21–27… find
362. H. L. Shashirekha. Automatic Language Identification from Written Texts - An Overview. International Journal of Innovative Research in Computer and Communication Engineering, 2(5):156–160, 2014. find
363. Karen Shiells and Peter Pham. Unsupervised Clustering for Language Identification. Project Report, Stanford University, 2010. find
364. Prajwol Shrestha. Incremental N-gram Approach for Language Identification in Code-Switched Text. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 133–138, Doha, Qatar, 2014. find
365. Prajwol Shrestha. Codeswitching Detection via Lexical Features using Conditional Random Fields. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 121–126, Austin, TX, USA, 2016. find
366. Penelope Sibun and Jeffrey C. Reynar. Language Identification: Examining the Issues. In Proceedings of the 5th Annual Symposium on Document Analysis and Information Retrieval (SDAIR-96), pages 125–135, Las Vegas, USA, 1996. find
367. Sebastian Sierra, Manuel Montes-y Gómez, Thamar Solorio, and Fabio A. González. Convolutional Neural Networks for Author Profiling—Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and W… find
368. Utpal Kumar Sikdar and Björn Gambäck. Language Identification in Code-Switched Text Using Conditional Random Fields and Babelnet. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 127–131, Austin, TX, USA, 2016. find
369. Vasiliki Simaki, Panagiotis Simakis, Carita Paradis, and Andreas Kerren. Identifying the Authors’ National Variety of English in Social Media Texts. In Proceedings of the International Conference Recent Advances in Natural Language Processing (RANLP 2017), pages 671–678, Varna, Bulgaria, September 2… find
370. Alberto Simões, José João Almeida, and Simon D. Byers. Language Identification: a Neural Network Approach. In Maria João Varanda Pereira, José Paulo Leal, and Alberto Simões, editors, Proceedings of the 3rd Symposium on Languages, Applications and Technologies (SLATE 2014), pages 251–265, Brag… find
371. Gary F. Simons and Charles D. Fennig, editors. Ethnologue: Languages of the World, Twentieth Edition. SIL International, Dallas, USA, 2017. Online version: http://www.ethnologue.com. ↗
372. Anil Kumar Singh. Study of Some Distance Measures for Language and Encoding Identification. In Proceedings of the Workshop on Linguistic Distances, pages 63–72, Sydney, Australia, July 2006. find
373. Anil Kumar Singh. Modeling and Application of Linguistic Similarity. PhD thesis, International Institute of Information Technology, Hyderabad, 2010. find
374. Anil Kumar Singh and Jagadeesh Gorla. Identification of Languages and Encodings in a Multilingual Document. In Proceedings of the 3rd ACL SIGWAC Workshop on Web As Corpus (WAC3-2007), pages 95–108, Louvain-la-Neuve, Belgium, 2007. find
375. Anil Kumar Singh and Pratya Goyal. A Language Identification Method Applied to Twitter Data. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference of the Spanish Society for Natural Language Processing (SEPLN 2014), pages 26–29, Girona, Spain, 2014. find
376. Navneet Sinha and Gowri Srinivasa. Hindi-English Language Identification, Named Entity Recognition and Back Transliteration: Shared Task System Description. In Working Notes os Shared Task on Transliterated Search at Forum for Information Retrieval Evaluation FIRE ’14, Bangalore, India, 2014. find
377. Thamar Solorio, Elizabeth Blair, Suraj Maharjan, Steven Bethard, Mona Diab, Mahmoud Gohneim, Abdelati Hawwari, Fahad AlGhamdi, Julia Hirschberg, Alison Chang, and Pascale Fung. Overview for the First Shared Task on Language Identification in Code-Switched Data. In Proceedings of The First Workshop o… find
378. Clive Souter, Gavin Churcher, Judith Hayes, John Hughes, and Stephen Johnson. Natural Language Identification using Corpus-Based Models. Hermes, Journal of Linguistics, 13: 183–203, 1994. find
379. Aleksander Stensby, B. John Oommen, and Ole-Christoffer Granmo. Language Detection and Tracking in Multilingual Documents Using Weak Estimators. In Proceedings of the Joint IAPR International Workshop Structural, Syntactic, and Statistical Pattern Recognition (SSPR&SPR 2010), pages 600–609, Cesme, I… find
380. Erik Sterneberg. Language Identification of Person Names Using Cascaded SVMs. Bachelor’s Thesis, Uppsala University, Uppsala, 2012. find
381. Marija Stupar, Tereza Jurić, and Nikola Ljubešić. Language Identification of Web Data for Building Linguistic Corpora. In Proceedings of the 3rd International Conference on The Future of Information Sciences (INFuture 2011), pages 365–372, Zagreb, Croatia, 2011. find
382. Izumi Suzuki, Yoshiki Mikami, Ario Ohsato, and Yoshihide Chubachi. A Language and Character Set Determination Method Based on n-gram Statistics. ACM Transactions on Asian Language Information Processing (TALIP), 1(3):269–278, September 2002. find
383. Hidayet Takçı and Ekin Ekinci. Minimal Feature Set in Language Identification and Finding Suitable Classification Method with it. Procedia Technology, 1:444–448, January 2012. find
384. Hidayet Takçı and Tunga Güngör. A High Performance Centroid-based Classification Approach for Language Identification. Pattern Recognition Letters, 33(16):2077–2084, December 2012. find
385. Liling Tan, Marcos Zampieri, Nikola Ljubešić, and Jörg Tiedemann. Merging Comparable Data Sources for the Discrimination of Similar Languages: The DSL Corpus Collection. In Proceedings of the 7th Workshop on Building and Using Comparable Corpora (BUCC), Reykjavik, Iceland, 2014. find
386. William John Teahan. Text Classification and Segmentation Using Minimum Cross-Entropy. In Proceedings of the 6th International Conference Recherche d’Information Assistee par Ordinateur (RIAO’00), pages 943–961, Paris, France, 2000. find
387. Eric S. Tellez, Sabino Miranda-Jiménez, Mario Graff, and Daniela Moctezuma. Gender and Language-variety Identification with MicroTC – Notebook for PAN at CLEF 2017. In Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl, editors, Working Notes Papers of CLEF 2017 Evaluation Labs and… find
388. Samuel Thomas and Ashish Verma. Language Identification of Person Names using CF-IOF based Weighing Function. In 8th Annual Conference of the International Speech Communication Association (INTERSPEECH 2007), pages 1769–1772, Antwerp, Belgium, 2007. find
389. Jilei Tian and Janne Suontausta. Scalable Neural Network Based Language Identification from Written Text. In Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’03), volume 1, pages 48–51, Hong Kong, 2003. find
390. Jilei Tian, Juha Häkkinen, Søren Riis, and Kåre Jean Jensen. On Text-based Language Identification for Multilingual Speech Recognition Systems. In Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP2002), Denver, Colorado, USA, 2002. find
391. Jörg Tiedemann and Nikola Ljubešić. Efficient Discrimination Between Closely Related Languages. In Proceedings of the 24th International Conference on Computational Linguistics (COLING 2012), pages 2619–2634, Mumbai, India, 2012. find
392. Christoph Tillmann, Yaser Al-Onaizan, and Saab Mansour. Improved Sentence-Level Arabic Dialect Classification. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 110–119, Dublin, Ireland, 2014. find
393. Andrija Tomović and Predrag Janičić. A Variant of N-Gram Based Language Classification. In Proceedings of the 10th Congress of the Italian Association for Artificial Intelligence Artificial Intelligence and Human-Oriented Computing (AI* IA 2007), pages 410–421, Rome, Italy, 2007. find
394. Dat Tran and Dharmendra Sharma. Markov Models for Written Language Identification. In Proceedings of the 12th International Conference on Neural Information Processing, pages 67–70, Taipei, Taiwan, 2005. find
395. Giang Binh Tran, Dat Ba Nguyen, and Bin Thanh Kieu. n-gram based approach for multilingual language identification. Poster. available at http://comp.mq.edu.au/programming/task\_description/VILangTek.pdf, 2010. ↗
396. Stephen Tratz, Douglas Briesch, Jamal Laoudi, and Clare Voss. Tweet Conversation Annotation Tool with a Focus on an Arabic Dialect, Moroccan Darija. In Proceedings of the 7th Linguistic Annotation Workshop & Interoperability with Discourse, pages 135–139, Sofia, Bulgaria, 2013. find
397. Stephen C. Tratz. Accurate Arabic Script Language/Dialect Classification. Technical report, Army Research Laboratory, 2014. find
398. Dolf Trieschnigg, Djoerd Hiemstra, Mariët Theune, Franciska de Jong, and Theo Meder. An Exploration of Language Identification Techniques for the Dutch Folktale Database. In Proceedings of the LREC workshop Adaptation of Language Resources and Tools for Processing Cultural Heritage, pages 47–51, Is… find
399. Erik Tromp and Mykola Pechenizkiy. Graph-Based N-gram Language Identification on Short Texts. In Proceedings of the 20th Annual Belgian Dutch Conference on Machine Learning (Benelearn 2011), pages 27–34, The Hague, Netherlands, 2011. find
400. Yoshido Ueda and Seiichi Nakagawa. Prediction for Phoneme/Syllable/Word-Category and Identification of Language using HMM. In Proceedings of the 1990 International Conference on Spoken Language Processing, volume 2, volume 2, pages 1209–1212, Kobe, Japan, 1990. find
401. Edvin Ullman. Shibboleth - A Multilingual Language Identifier. Master’s thesis, Uppsala University, Uppsala, 2014. find
402. Chris van der Lee and Antal van den Bosch. Exploring Lexical and Syntactic Features for Language Variety Identification. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 190–199, Valencia, Spain, 2017. find
403. C. J. van Rijsbergen. Information Retrieval. Butterworths, London, UK, 1979. find
404. Tommi Vatanen, Jaakko J. Väyrynen, and Sami Virpioja. Language Identification of Short Text Segments with N-gram Models. In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010), pages 3423–3430, Valletta, Malta, 2010. find
405. Vinsensius Berlian Vega and Stéphane Bressan. Continuous-Learning Weighted-Trigram Approach for Indonesian Language Distinction: A Preliminary Study. In Proceedings of 19th International Conference on Computer Processing of Oriental Languages (ICCPOL 2001), Seoul, Korea, 2001a. Oriental Languages C… find
406. Vinsensius Berlian Vega and Stéphane Bressan. Indexing the Indonesian Web: Language Identification and Miscellaneous Issues. In Poster Proceedings of the Tenth International World Wide Web Conference (WWW 10), Hong Kong, 2001b. find
407. J. Vinosh Babu and S. Baskaran. Automatic Language Identification Using Multivariate Analysis. In Proceedings of the 6th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2005), pages 789–792, Mexico City, Mexico, 2005. find
408. Tony Vitale. An Algorithm for High Accuracy Name Pronunciation by Parametric Speech Synthesizer. Computational Linguistics, 17(3):257–276, 1991. find
409. John Vogel and David Tresner-Kirsch. Robust Language Identification in Short, Noisy Texts: Improvements to LIGA. In Martin Atzmueller and Hotho Andreas, editors, Proceedings of the 3rd International Workshop on Mining Ubiquitous and Social Environments (MUSE), pages 43–50, Bristol, UK, 2012. find
410. Clare Voss, Stephen Tratz, Jamal Laoudi, and Douglas Briesch. Finding Romanized Arabic Dialect in Code-Mixed Tweets. In Proceedings of the 9th International Conference on Language Resources and Evaluation, ELRA, pages 188–199, Reykjavik, Iceland, 2014. find
411. Ada Wan. Leveraging Data-Driven Methods in Word-Level Language Identification for a Multilingual Alpine Heritage Corpus. In Proceedings of the Workshop on Multilingual and Cross-lingual Methods in NLP, pages 45–54, San Diego, California, 2016. find
412. Martin Wechsler, Sheridan Páraic, and Peter Schäuble. Multi-language Text Indexing for Internet Retrieval. In Proceeding RIAO’97 Computer-Assisted Information Searching on Internet, pages 217–232, Montreal, Canada, 1997. find
413. Jennifer Williams and Charlie K. Dagli. Twitter Language Identification Of Similar Languages And Dialects Without Ground Truth. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 73–83, Valencia, Spain, 2017. find
414. Gergely Windisch and László Csink. Language Identification Using Global Statistics of Natural Languages. In Proceedings of the 2nd Romanian-Hungarian Joint Symposium on Applied Computational Intelligence (SACI), pages 243–255, Timisoara, Romania, 2005. find
415. Samantha Wray. Classification of Closely Related Sub-dialects of Arabic Using Support-Vector Machines. In Proceedings of Language Resources and Evaluation (LREC), Miyazaki, Japan, 2018. find
416. Alexandros Xafopoulos, Constantine Kotropoulos, George Almpanidis, and Ioannis Pitas. Language Identification in Web Documents Using Discrete HMMs. Pattern Recognition, 37(3):583–594, 2004. find
417. Fei Xia, William D. Lewis, and Hoifung Poon. Language ID in the Context of Harvesting Language Data off the Web. In +EACL2009, pages 870–878, Athens, Greece, 2009. find
418. Fei Xia, Carrie Lewis, and William D. Lewis. The Problems of Language Identification within Hugely Multilingual Data Sets. In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010), pages 2790–2797, Valletta, Malta, 2010. find
419. Meng Xuan Xia. Codeswitching Language Identification Using Subword Information Enriched Word Vectors. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 132–136, Austin, TX, USA, 2016. Association for Computational Linguistics. find
420. Fan Xu, Mingwen Wang, and Maoxi Li. Sentence-level Dialects Identification in the Greater China Region. International Journal on Natural Language Computing (IJNLC), 5(6), December 2016. find
421. Mohammad Ali Yaghan. “Arabizi”: A Contemporary Style of Arabic Slang. Design Issues, 24(2):39–52, 2008. find
422. Hiroshi Yamaguchi and Kumiko Tanaka-Ishii. Text Segmentation by Language Using Minimum Description Length. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012), pages 969–978, Jeju Island, Korea, July 2012. find
423. Xi Yang and Wenxin Liang. An N-Gram-and-Wikipedia joint approach to Natural Language Identification. In Proceedings of the 4th International Universal Communication Symposium (IUCS 2010), pages 332–339, Beijing, China, 2010. find
424. Noson S. Yanofsky. Towards a Definition of an Algorithm. Journal of Logic and Computation, 21(2):253–286, 2011. find
425. Yin-Lai Yeong and Tien-Ping Tan. Language Identification of Code Switching Malay-English Words Using Syllable Structure Information. In Proceedings of the 2nd International Workshop on Spoken Languages Technologies for Under-resourced Languages (SLTU’10), pages 142–145, Penang, Malaysia, 2010. find
426. Yin-Lai Yeong and Tien-Ping Tan. Applying Grapheme, Word, and Syllable Information for Language Identification in Code Switching Sentences. In Proceedings of the International Conference on Asian Language Processing (IALP 2011), pages 111–114, Penang, Malaysia, 2011. IEEE. find
427. Jia-Li You, Yi-Ning Chen, Min Chu, Frank K. Soong, and Jin-Lin Wang. Identifying Language Origin of Named Entity with Multiple Information Sources. Audio, Speech, and Language Processing, IEEE Transactions on, 16(6):1077–1086, 2008. find
428. Omar F. Zaidan and Chris Callison-Burch. The Arabic Online Commentary Dataset: An Annotated Dataset of Informal Arabic with High Dialectal Content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 37… find
429. Omar F. Zaidan and Chris Callison-Burch. Arabic Dialect Identification. Computational Linguistics, 40(1):171–202, 2014. find our card
430. Juglar Dı́az Zamora, Adrian Fonseca Bruzón, and Reynier Ortega Bueno. Tweets Language Identification Using Feature Weighting. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference of the Spanish Society for Natural Language Processing (SEPLN 2014), pages… find
431. Marcos Zampieri. Using Bag-of-words to Distinguish Similar Languages: How Efficient Are They? In Proceedings of the 2013 IEEE 14th International Symposium on Computational Intelligence and Informatics (CINTI), pages 37–41, Budapest, Hungary, 2013. find
432. Marcos Zampieri. Automatic Language Identification. In Working with Text: Tools, Techniques and Approaches for Text Mining, chapter 8, pages 189–205. Elsevier, 2016. find
433. Marcos Zampieri and Binyam Gebrekidan Gebre. Automatic Identification of Language Varieties: The Case of Portuguese. In Proceedings of The 11th Conference on Natural Language Processing (KONVENS 2012), pages 233–237, Vienna, Austria, 2012. find
434. Marcos Zampieri and Binyam Gebrekidan Gebre. VarClass: An Open-source Language Identification Tool for Language Varieties. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014), pages 3305–3308, Reykjavik, Iceland, 2014. find
435. Marcos Zampieri, Binyam Gebrekidan Gebre, and Sascha Diwersy. N-gram Language Models and POS Distribution for the Identification of Spanish Varieties. In Proceedings of la 20ème conférence du Traitement Automatique du Langage Naturel (TALN), pages 580– 587, Sables d’Olonne, France, 2013. find
436. Marcos Zampieri, Liling Tan, Nikola Ljubešić, and Jörg Tiedemann. A Report on the DSL Shared Task 2014. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 58–67, Dublin, Ireland, 2014. find
437. Marcos Zampieri, Binyam Gebrekidan Gebre, Hernani Costa, and Josef van Genabith. Comparing Approaches to the Identification of Similar Languages. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 66–72, Hissar, Bulga… find
438. Marcos Zampieri, Liling Tan, Nikola Ljubešić, Jörg Tiedemann, and Preslav Nakov. Overview of the DSL Shared Task 2015. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 1–9, Hissar, Bulgaria, 2015b. find
439. Marcos Zampieri, Shervin Malmasi, Octavia-Maria Sulea, and Liviu P Dinu. A Computational Approach to the Study of Portuguese Newspapers Published in Macau. In Proceedings of the Workshop on Natural Language Processing meets Journalism (NLPMJ 2016), pages 47–51, New York City, NY, USA, 2016. find
440. Marcos Zampieri, Shervin Malmasi, Nikola Ljubešic, Preslav Nakov, Ahmed Ali, Jörg Tiedemann, Yves Scherrer, and Noëmi Aepli. Findings of the VarDial Evaluation Campaign 2017. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 1–15, Valencia, Spain, 2… find our card
441. Andjelka Zecevic and Stasa Vujicic-Stankovic. The Mysterious Letter J. In Proceedings of the Workshop on Adaptation of Language Resources and Tools for Closely Related Languages and Language Variants, pages 40–44, Hissar, Bulgaria, 2013. find
442. Wei Zhang, Robert A. J. Clark, Yongyuan Wang, and Wen Li. Unsupervised Language Identification Based on Latent Dirichlet Allocation. Computer Speech and Language, 39: 47–66, 2016. find
443. Anna V. Zhdanova. Automatic Identification of European Languages. In Revised Papers of the Natural Language Processing and Information Systems - 6th International Conference on Applications of Natural Language to Information Systems, (NLDB 2002), pages 76–84, Stockholm, Sweden, 2002. find
444. Ayah Zirikly, Bart Desmet, and Mona Diab. The GW/LT3 VarDial 2016 Shared Task System for Dialects and Similar Languages Detection. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects, pages 33–41, Osaka, Japan, 2016. find
445. Arkaitz Zubiaga, Inaki San Vicente, Pablo Gamallo, José Ramom Pichel, Inaki Alegria, Nora Aranberri, Aitzol Ezeiza, and Vıctor Fresno. Overview of TweetLID: Tweet Language Identification at SEPLN 2014. In Proceedings of the Tweet Language Identification Workshop 2014 co-located with 30th Conference… find
446. Arkaitz Zubiaga, Iñaki San Vicente, Pablo Gamallo, José Ramom Pichel, Iñaki Alegria, Nora Aranberri, Aitzol Ezeiza, and Vı́ctor Fresno. TweetLID: A Benchmark for Tweet Language Identification. Language Resources and Evaluation, 50 (4):729–766, 2016. ISSN 1574-020X. doi: 10.1007/s10579-015-9317-4.… — [doi:10.1007/s10579-015-9317-4](https://doi.org/10.1007/s10579-015-9317-4) find