Elamur Research⌕
☰
home / papers / 2016 / UnibucKernel: An Approach for Arabic Dialect Identification Based on Multiple String Kernels

UnibucKernel: An Approach for Arabic Dialect Identification Based on Multiple String Kernels

Paper · 2016 · Radu Tudor Ionescu and Marius Popescu

Summary

The paper addresses Arabic Dialect Identification (ADI) in the closed track of the DSL 2016 Challenge. Systems had to assign each text to one of five classes: Modern Standard Arabic (MSA) or one of four dialects. The data are ASR transcripts of Broadcast News speech, so they may contain recognition errors. The authors deliberately avoid linguistic processing, including tokenization, and treat every document as a raw character string. They build three string kernels over character p-grams:

  • a presence-bits kernel,
  • a histogram intersection kernel,
  • a kernel derived from Local Rank Distance (LRD).

They sum these kernels in a multiple kernel learning setup and train Kernel Discriminant Analysis (KDA), which edged out Kernel Ridge Regression (KRR) in 10-fold cross-validation. Their best submission placed second of 18 teams, with a weighted F1 of 51.31%. After the competition they learned that the training and test sets came from different sources. Simply doubling the regularization then raised weighted F1 to 52.18%, above the winner's score. The work shows that shallow, language-independent character features stay competitive on noisy dialect data.

Key points

  • The task is 5-way classification (MSA plus 4 Arabic dialects) on ASR transcripts from the Broadcast News domain, in the ADI Closed Shared Task of DSL 2016. Eighteen teams submitted.
  • The features are blended character p-grams of lengths 3 to 6, used by three kernels: p-grams presence bits, intersection string kernel, and an LRD-based kernel.
  • In 10-fold CV on the training data:
    • LRD was the strongest single kernel, at 66.54% accuracy with KDA.
    • Summing all three kernels reached 67.37% with KDA and 66.98% with KRR.
    • Kernel combination added close to 1% accuracy.
  • On the official test set:
    • The submitted run3 (all kernels, KDA) scored 50.91% accuracy and 51.31% weighted F1. The winner was ahead by a tiny margin (51.32%).
    • After the challenge, KDA on presence bits plus intersection, with doubled regularization, reached 51.82% accuracy and 52.18% weighted F1.
    • The best KRR model reached 51.99% weighted F1.
  • The best model takes about 12 minutes on one CPU core to compute the kernels, train and predict.

Interesting ideas

  • I1. The intersection kernel sums the minimum of two p-gram counts, which sits between ignoring frequency (presence bits) and multiplying frequencies (p-spectrum); this damps the influence of repeated items without discarding counts entirely.
  • I2. Several similarity measures that see the data differently are combined simply by adding their kernel matrices, and every combination beat its individual parts.
  • I3. Cross-validation accuracy (~67%) was far above test accuracy (~51%) because the two sets came from different sources; the remedy was stronger regularization so the model fits less of the training distribution.
  • I4. A deliberately tokenization-free, linguistics-free character model reached near-top results, which makes it a cheap and honest reference point.
  • I5. Character p-grams tolerate noisy input such as ASR transcripts, where word-level units may be broken.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when comparing outputs of witness parsers or silver datasets by feature overlap, a min-of-counts similarity could stop a few frequent constructions from dominating the agreement score.
  • I2 (tree): agreement scores from the perceptron parser and gparse could be summed into a single combined similarity, rather than choosing one parser's view, when ranking candidate trees or treebank sentences.
  • I3 (learn): evaluation should always include an out-of-source test set, and the gap from in-domain CV should be read as a signal to regularize models trained on silver data harder.
  • I4 (learn): such a shallow character baseline could act as a negative control, showing whether Mova's explainable pipeline really beats surface statistics on a given test.
  • I5 (tokens): character n-gram cues could serve as a fallback signal in the tokenizer or tagger for misspelled or garbled tokens, without replacing the structured analysis.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BucharestBucharest · Romania21 works · 516 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Radu Tudor Ionescu and Marius Popescu. 2016. UnibucKernel: An Approach for Arabic Dialect Identification Based on Multiple String Kernels. Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ionescu2016unibuckernel,
  title = {{UnibucKernel: An Approach for Arabic Dialect Identification Based on Multiple String Kernels}},
  author = {Radu Tudor Ionescu and Marius Popescu},
  booktitle = {Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3)},
  year = {2016},
  url = {https://aclanthology.org/W16-4818/}
}

References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell, and Steve Renals. 2016. Automatic dialect detection in arabic broadcast speech. Proceedings of Interspeech, pages 2934–2938.
  2. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken Arabic Dialect Identification Using Phonotactic Modeling. Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages, pages 53–61.
  3. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. A Multidialectal Parallel Corpus of Arabic. Proceedings of LREC, pages 1240–1245, may.
  4. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. 2014. Verifiably Effective Arabic Dialect Identification. Proceedings of EMNLP, pages 1465–1468.
  5. Liviu P. Dinu and Florin Manea. 2006. An efficient approach for the rank aggregation problem. Theoretical Computer Science, 359(1–3):455–461. ↗
  6. Liviu P. Dinu, Radu Tudor Ionescu, and Alexandru I. Tomescu. 2014. A rank-based sequence aligner with applications in phylogenetic analysis. PLoS ONE, 9(8):e104006, 08. ↗
  7. Heba Elfardy and Mona T. Diab. 2013. Sentence Level Dialect Identification in Arabic. Proceedings of ACL, pages 456–461.
  8. Hugo Jair Escalante, Thamar Solorio, and Manuel Montes-y-Gómez. 2011. Local histograms of character n-grams for authorship attribution. Proceedings of ACL: HLT, 1:288–298. find
  9. Mehmet Gonen and Ethem Alpaydin. 2011. Multiple Kernel Learning Algorithms. Journal of Machine Learning Research, 12:2211–2268, July. find
  10. Cristian Grozea, Christian Gehl, and Marius Popescu. 2009. ENCOPLOT: Pairwise Sequence Matching in Linear Time Applied to Plagiarism Detection. Proceedings of 3rd PAN WORKSHOP, page 10. find
  11. Trevor Hastie and Robert Tibshirani. 2003. The Elements of Statistical Learning. Springer, corrected edition, July. find
  12. David Haussler. 1999. Convolution Kernels on Discrete Structures. Technical Report UCS-CRL-99-10, University of California at Santa Cruz, Santa Cruz, CA, USA. find
  13. Radu Tudor Ionescu, Marius Popescu, and Aoife Cahill. 2014. Can characters reveal your native language? A language-independent approach to native language identification. Proceedings of EMNLP, pages 1363–1373, October. find
  14. Radu Tudor Ionescu, Marius Popescu, and Aoife Cahill. 2016. String kernels for native language identification: Insights from behind the curtains. Computational Linguistics, 42(3):491–525. ↗
  15. Radu Tudor Ionescu. 2013. Local Rank Distance. Proceedings of SYNASC, pages 221–228. find
  16. Radu Tudor Ionescu. 2015. A Fast Algorithm for Local Rank Distance: Application to Arabic Native Language Identification. Proceedings of ICONIP, 9490:390–400.
  17. Rohit J. Kate and Raymond J. Mooney. 2006. Using String-kernels for Learning Semantic Parsers. Proceedings of ACL, pages 913–920.
  18. Huma Lodhi, Craig Saunders, John Shawe-Taylor, Nello Cristianini, and Christopher J. C. H. Watkins. 2002. Text Classification using String Kernels. Journal of Machine Learning Research, 2:419–444. find
  19. Subhransu Maji, Alexander C. Berg, and Jitendra Malik. 2008. Classification using intersection kernel support vector machines is efficient. Proceedings of CVPR. find
  20. Shervin Malmasi, Eshrag Refaee, and Mark Dras. 2015. Arabic Dialect Identification using a Parallel Multidialectal Corpus. Proceedings of PACLING, pages 209–217, May. find
  21. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. Discriminating between similar languages and arabic dialect identification: A report on the third dsl shared task. Proceedings of the 3rd Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (VarDial).
  22. Marius Popescu and Liviu P. Dinu. 2007. Kernel methods and string kernels for authorship identification: The federalist papers case. Proceedings of RANLP, September. find
  23. Marius Popescu and Cristian Grozea. 2012. Kernel methods and string kernels for authorship analysis. CLEF (Online Working Notes/Labs/Workshop), September. find
  24. Marius Popescu and Radu Tudor Ionescu. 2013. The Story of the Characters, the DNA and the Native Language. Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 270–278, June. find
  25. Marius Popescu. 2011. Studying translationese at the character level. Proceedings of RANLP, pages 634–639, September. find
  26. Conrad Sanderson and Simon Guenter. 2006. Short text authorship attribution via sequence kernels, markov chains and author unmasking: An investigation. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 482–491, July. find
  27. John Shawe-Taylor and Nello Cristianini. 2004. Kernel Methods for Pattern Analysis. Cambridge University Press. ↗
  28. Andrea Vedaldi and Andrew Zisserman. 2010. Efficient additive kernels via explicit feature maps. Proceedings of CVPR, pages 3539–3546. ↗
  29. Omar F. Zaidan and Chris Callison-Burch. 2011. The Arabic Online Commentary Dataset: An Annotated Dataset of Informal Arabic with High Dialectal Content. Proceedings of ACL: HLT, 2:37–41.
  30. Omar F. Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ionescu-2016-unibuckernel-approach-arabic-dialect-identification/