Elamur Research⌕
☰
home / papers / 2020 / Habibi - a multi Dialect multi National Arabic Song Lyrics Corpus

Habibi - a multi Dialect multi National Arabic Song Lyrics Corpus

Summary

Arabic dialect identification has mostly relied on tweets and forum comments. These are short and noisy, and they are usually labelled by crowd workers, which is costly and of uneven quality. El-Haj presents Habibi, which he says is the earliest corpus of Arabic lyrics to be released openly at no cost. It contains over 30,000 songs by singers from 18 countries, split into verses. Song metadata came from the web, and Google and Wikipedia supplied each singer's country of origin. Countries were then grouped into six dialect labels: Egyptian, Gulf, Levantine, Iraqi, Sudanese and Maghrebi. To show the corpus is usable, the author ran binary and multi-class experiments on two tasks, dialect identification and country identification. Classical classifiers were compared with several neural architectures, using either pretrained Arabic Wiki FastText vectors or CBOW embeddings trained on the lyrics themselves. The best binary dialect result was 93% test accuracy, from a word-level CNN with Habibi's own embeddings. Every model beat a majority-class baseline. The corpus and its embeddings provide a longer-context resource for dialect work and future authorship studies.

Key points

  • Corpus statistics:
    • 30,072 songs, 527,870 verse-sentences and over 3.5 million words.
    • 1,765 singers, 3,789 songwriters and 2,463 composers.
    • 18 countries and 6 dialects.
  • Labels come from the singer's country of origin, not from the lyrics' actual dialect. This introduces noise when, for example, a Lebanese singer such as Nancy Ajram performs in Egyptian.
  • The corpus is released as annotated txt, csv, json and xml. Missing writers or composers are marked "unknown". The 300-dimension CBOW embeddings are released as well.
  • Models tested:
    • Classical: Naïve Bayes, SVM and LR.
    • Neural: CNN, LSTM, BiGRU, BiLSTM and CLSTM.
    • Country experiments used nested subsets: top_2, top_3, top_4, top_5 and all_18 countries.
  • Findings:
    • The CNN was the strongest neural model throughout.
    • Naïve Bayes beat all neural models in every multi-class setting.
    • In-domain embeddings helped dialect identification, while FastText helped country identification.

Interesting ideas

  • I1. Instead of paying crowd annotators, the labels are inherited from external metadata (singer origin found via Wikipedia), and the paper openly names the resulting noise source. — For Mova (learn): Document-level metadata such as source collection or genre could act as cheap silver labels for register or style classifiers, provided the known mismatch cases are logged as a noise caveat.
  • I2. The country task is run over a ladder of nested label sets (2, 3, 4, 5, then all 18 classes), showing how accuracy decays as classes multiply and per-class data thins. — For Mova (learn): Evaluations could be reported over growing label inventories (e.g., coarse vs. fine semantic roles) to reveal where a component stops scaling, rather than giving a single number.
  • I3. A simple multinomial Naïve Bayes outperformed every neural model once there were many classes, and a most-frequent-class baseline anchored all results. — For Mova (learn): Every new learned component should be scored against a trivial majority baseline and a shallow classical model as negative controls, to show the added machinery earns its cost.
  • I4. Embeddings trained on the target domain helped one task, while general pretrained vectors helped a related one, so the embedding source was treated as a per-task choice. — For Mova (tokens): For tagging and lemmatising fables and school texts, Mova could compare domain-derived lexical features against general ones per subtask instead of fixing one resource globally.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

38 papers of our library cite it (of the 140884 whose bibliographies we read); 10 of them have a page here, 28 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{elhaj2020habibi,
  title = {{Habibi - a multi Dialect multi National Arabic Song Lyrics Corpus}},
  author = {El-Haj, Mahmoud},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.165/}
}

References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alsarsour, I., Mohamed, E., Suwaileh, R., and Elsayed, T. (2018). DART: A Large Dataset of Dialectal Arabic Tweets. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 7-12, 2018. European Language Resources Association (ELRA). find
  2. Aquil, R. (2012). Revisiting songs in language pedagogy. Journal of the National Council of Less Commonly Taught Languages, 11:75–95. find
  3. Bouamor, H., Hassan, S., and Habash, N. (2019). The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy, August. Association for Computational Linguistics. find our card
  4. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. ↗
  5. El-Haj, M., Rayson, P. E., Young, S. E., Walker, M., Moore, A., Athanasakou, V., and Schleicher, T. (2016). Learning tone and attribution for financial text mining. find
  6. El-Haj, M., Rayson, P., and Aboelezz, M. (2018). Arabic dialect identification in the context of bivalency and code-switching. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA). find
  7. Elaraby, M. and Abdul-Mageed, M. (2018). Deep models for Arabic dialect identification on benchmarked data. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 263–274, Santa Fe, New Mexico, USA, August. Association for Computational Linguistics. find
  8. Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Comput., 9(8):1735–1780, November. find
  9. Kilgarriff, A. and Grefenstette, G. (2001). Web as corpus. In Proceedings of Corpus Linguistics 2001, pages 342– 344. Corpus Linguistics. find
  10. Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746–1751, Doha, Qatar, October. Association for Computational Linguistics. find our card
  11. McCallum, A., Nigam, K., et al. (1998). A comparison of event models for naive bayes text classification. In AAAI-98 workshop on learning for text categorization, volume 752, pages 41–48. Citeseer. find
  12. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  13. Qing Chen, Shipper, T., and Khan, L. (2010). Tweets mining using wikipedia and impurity cluster measurement. In 2010 IEEE International Conference on Intelligence and Security Informatics, pages 141–143, May. find
  14. Ritter, A., Clark, S., Etzioni, O., et al. (2011). Named entity recognition in tweets: an experimental study. In Proceedings of the conference on empirical methods in natural language processing, pages 1524–1534. Association for Computational Linguistics. find our card
  15. Sainath, T. N., Vinyals, O., Senior, A., and Sak, H. (2015). Convolutional, long short-term memory, fully connected deep neural networks. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4580–4584. IEEE. find
  16. Soboh, L., Elkabani, I., and Osman, Z. (2017). Arabic cultural style based music classification. In 2017 International Conference on New Trends in Computing Sciences (ICTCS), pages 6–11, Oct. find
  17. Terkourafi, M. (2010). The Languages of Global Hip Hop. Continuum International Publishing Group, New York, USA. find
  18. Touma, H. H. and Touma, H. (2003). The music of the Arabs. Hal Leonard Corporation. find
  19. Zaidan, O. F. and Callison-Burch, C. (2014). Arabic dialect identification. Comput. Linguist., 40(1):171–202, March. find our card
  20. Zhou, C., Sun, C., Liu, Z., and Lau, F. C. M. (2015). A C-LSTM neural network for text classification. CoRR, abs/1511.08630. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/haj-2020-habibi-multi-dialect-multi-national/