Elamur Research⌕
☰
home / papers / 2017 / Learning to Identify Arabic and German Dialects using Multiple Kernels

Learning to Identify Arabic and German Dialects using Multiple Kernels

Paper · 2017 · Radu Tudor Ionescu and Andrei Butnaru

Summary

The paper describes the UnibucKernel entry to two closed dialect-identification shared tasks at the DSL 2017 Challenge (VarDial). The first, ADI, is a 5-way task that separates Modern Standard Arabic from four Arabic dialects using ASR transcripts of Broadcast News speech plus i-vectors from the audio. The second, GDI, is a 4-way task over transcribed Swiss German from Basel, Bern, Lucerne and Zurich. The authors use three character p-gram string kernels: presence bits, histogram intersection, and a kernel derived from Local Rank Distance. For Arabic they add an RBF kernel over i-vectors. The kernels are simply summed and fed to Kernel Ridge Regression or Kernel Discriminant Analysis. Lengths and ranges of p-grams were tuned with cross-validation or on the development data. The system won ADI with 76.32% weighted F1 and placed fifth in GDI with 63.67%, only 2.57 points behind the winner. The work shows that a shallow kernel method can stay competitive and can merge evidence from text and audio in one framework.

Key points

  • ADI results: among 6 teams the system ranked 1st, with 76.27% accuracy and 76.32% weighted F1, which was 4.62 points ahead of second place.
  • GDI results: among 10 teams it ranked 5th, with 66.36% accuracy and 63.67% weighted F1. The majority-class baseline on the test set is 25.80%. The GDI data comes from the corpus of Samardzic et al. (2016).
  • Value of i-vectors on ADI:
    • The best single string kernel reached 52.55% on development data.
    • The i-vector kernel alone reached up to 58.99% with KDA.
    • Summing all four kernels gave 64.17% with KRR and 63.85% with KDA.
    • The authors attribute the weakness of the transcript kernels to ASR errors.
  • GDI tuning (5-fold CV on the training set):
    • With KRR and the intersection kernel, 4-grams were best among single lengths (82.87%).
    • Blended ranges worked better: 3–6 for the presence-bits and intersection kernels, 3–5 for LRD.
    • Combining all three kernels reached 84.39% with KRR and 84.49% with KDA.
  • Overall patterns:
    • KRR and KDA performed about the same, though the best submitted run in both tasks used KRR.
    • Every kernel combination beat each of its parts.
    • Lucerne was the dialect the model confused most often.
    • The final models were retrained on the training and development data together.

Interesting ideas

  • I1. Different views of the data, such as several p-gram similarity measures and an audio-based kernel, are combined by plain kernel addition, so a new evidence source can be added without redesigning the model.
  • I2. Recording only whether a p-gram occurs, instead of how often, turns out to be a competitive signal and dampens the effect of repeated material.
  • I3. Instead of fixing one n-gram length, the authors first sweep single lengths and then test blended ranges, with each kernel getting its own best range.
  • I4. Their analysis traces the transcript kernels' weakness to errors introduced upstream by ASR, and a different modality (i-vectors) works around that noise.
  • I5. A per-class confusion matrix shows that one class (Lucerne) is almost never separated from the others, a problem the headline score hides.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Agreement among witness parsers could be scored as a sum of similarity measures, one per witness, which lets a new witness join the silver-data vote cheaply and keeps each contribution inspectable.
  • I2 (tokens): When guessing tags or lemmas for unknown words from character n-grams, presence-only features may be more robust than counts to repetitive or noisy text.
  • I3 (tokens): Character-affix features in the tagger or morphology module could be tuned the same way, sweeping single lengths first and then checking contiguous ranges per feature type under cross-validation.
  • I4 (learn): Evaluation should track how upstream errors spread through the pipeline, for example by comparing performance on gold versus predicted trees, so that weak stages are found before downstream modules are blamed.
  • I5 (answer): Reporting confusion by question type or frame type on school tests and fable QA would expose categories that the readers systematically fail.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BucharestBucharest · Romania21 works · 516 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Radu Tudor Ionescu and Andrei Butnaru. 2017. Learning to Identify Arabic and German Dialects using Multiple Kernels. Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-1225

@inproceedings{ionescu2017learning,
  title = {{Learning to Identify Arabic and German Dialects using Multiple Kernels}},
  author = {Radu Tudor Ionescu and Andrei Butnaru},
  booktitle = {Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)},
  year = {2017},
  doi = {10.18653/v1/w17-1225},
  url = {https://doi.org/10.18653/v1/w17-1225}
}

References

34 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Radu Tudor Ionescu, Marius Popescu, and Aoife Cahill. 2014. Can characters reveal your native language? A language-independent approach to native language identification. Proceedings of EMNLP, pages 1363–1373, October. find
  2. Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell, and Steve Renals. 2016. Automatic dialect detection in arabic broadcast speech. Proceedings of Interspeech, pages 2934–2938.
  3. Radu Tudor Ionescu, Marius Popescu, and Aoife Cahill. 2016. String kernels for native language identification: Insights from behind the curtains. Computational Linguistics, 42(3):491–525. ↗
  4. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken Arabic Dialect Identification Using Phonotactic Modeling. Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages, pages 53–61.
  5. Radu Tudor Ionescu. 2013. Local Rank Distance. Proceedings of SYNASC, pages 221–228. find
  6. Radu Tudor Ionescu. 2015. A Fast Algorithm for Local Rank Distance: Application to Arabic Native Language Identification. Proceedings of ICONIP, 9490:390–400.
  7. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. A Multidialectal Parallel Corpus of Arabic. Proceedings of LREC, pages 1240–1245, may.
  8. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. 2014. Verifiably Effective Arabic Dialect Identification. Proceedings of EMNLP, pages 1465–1468.
  9. Rohit J. Kate and Raymond J. Mooney. 2006. Using String-kernels for Learning Semantic Parsers. Proceedings of ACL, pages 913–920.
  10. Liviu P. Dinu, Radu Tudor Ionescu, and Alexandru I. Tomescu. 2014. A rank-based sequence aligner with applications in phylogenetic analysis. PLoS ONE, 9(8):e104006, 08. ↗
  11. Huma Lodhi, Craig Saunders, John Shawe-Taylor, Nello Cristianini, and Christopher J. C. H. Watkins. 2002. Text Classification using String Kernels. Journal of Machine Learning Research, 2:419–444. find
  12. Heba Elfardy and Mona T. Diab. 2013. Sentence Level Dialect Identification in Arabic. Proceedings of ACL, pages 456–461.
  13. Subhransu Maji, Alexander C. Berg, and Jitendra Malik. 2008. Classification using intersection kernel support vector machines is efficient. Proceedings of CVPR. find
  14. Hugo Jair Escalante, Thamar Solorio, and Manuel Montes-y-Gómez. 2011. Local histograms of character n-grams for authorship attribution. Proceedings of ACL: HLT, 1:288–298. find
  15. Shervin Malmasi and Marcos Zampieri. 2016. Arabic Dialect Identification in Speech Transcripts. Proceedings of VarDial Workshop of COLING, pages 106–113.
  16. Cristian Grozea, Christian Gehl, and Marius Popescu. 2009. ENCOPLOT: Pairwise Sequence Matching in Linear Time Applied to Plagiarism Detection. Proceedings of 3rd PAN WORKSHOP, page 10. find
  17. Shervin Malmasi, Eshrag Refaee, and Mark Dras. 2015. Arabic Dialect Identification using a Parallel Multidialectal Corpus. Proceedings of PACLING, pages 209–217, May. find
  18. Trevor Hastie and Robert Tibshirani. 2003. The Elements of Statistical Learning. Springer, corrected edition, July. find
  19. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. Discriminating between similar languages and arabic dialect identification: A report on the third dsl shared task. Proceedings of the 3rd Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (VarDial).
  20. David Haussler. 1999. Convolution Kernels on Discrete Structures. Technical Report UCS-CRL-99-10, University of California at Santa Cruz, Santa Cruz, CA, USA. find
  21. Nora Hollenstein and Noëmi Aepli. 2015. A Resource for Natural Language Processing of Swiss German Dialects. Proceedings of GSCL, pages 108–109. find
  22. Marius Popescu and Cristian Grozea. 2012. Kernel methods and string kernels for authorship analysis. CLEF (Online Working Notes/Labs/Workshop), September. find
  23. Radu Tudor Ionescu and Marius Popescu. 2016a. Knowledge Transfer between Computer Vision and Text Mining. Advances in Computer Vision and Pattern Recognition. Springer International Publishing. ↗
  24. Marius Popescu and Radu Tudor Ionescu. 2013. The Story of the Characters, the DNA and the Native Language. Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 270–278, June. find
  25. Radu Tudor Ionescu and Marius Popescu. 2016b. UnibucKernel: An Approach for Arabic Dialect Identification based on Multiple String Kernels. Proceedings of VarDial Workshop of COLING, pages 135–144.
  26. Marius Popescu. 2011. Studying translationese at the character level. Proceedings of RANLP, pages 634– 639, September. find
  27. Tanja Samardzic, Yves Scherrer, and Elvira Glaser. 2016. ArchiMob–A corpus of spoken Swiss German. Proceedings of LREC, pages 4061–4066. find
  28. Conrad Sanderson and Simon Guenter. 2006. Short text authorship attribution via sequence kernels, markov chains and author unmasking: An investigation. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 482–491, July. find
  29. Yves Scherrer and Owen Rambow. 2010. Word-based dialect identification with georeferenced rules. Proceedings of EMNLP, pages 1151–1161. find
  30. John Shawe-Taylor and Nello Cristianini. 2004. Kernel Methods for Pattern Analysis. Cambridge University Press. ↗
  31. Andrea Vedaldi and Andrew Zisserman. 2010. Efficient additive kernels via explicit feature maps. Proceedings of CVPR, pages 3539–3546. ↗
  32. Omar F. Zaidan and Chris Callison-Burch. 2011. The Arabic Online Commentary Dataset: An Annotated Dataset of Informal Arabic with High Dialectal Content. Proceedings of ACL: HLT, 2:37–41.
  33. Omar F. Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗
  34. Marcos Zampieri, Shervin Malmasi, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, Jörg Tiedemann, Yves Scherrer, and Noëmi Aepli. 2017. Findings of the VarDial Evaluation Campaign 2017. Proceedings of VarDial Workshop of EACL.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ionescu-2017-learning-identify-arabic-german-dialects/