Elamur Research⌕
☰
home / papers / 2026 / E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding

E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding

Summary

Headline accuracy on NLU benchmarks says little about which linguistic phenomena break a model. Existing diagnostics suites (FraCaS, the GLUE and ALUE diagnostics) also lack a shared naming scheme and an agreed list of phenomena. The authors respond in three ways. They propose an initial cross-lingual hierarchy of macro and micro categories for NLU error analysis. They describe a method for turning that hierarchy into a language-specific NLI framework, using Arabic as the case study. They release E-CONAN, a free, manually annotated set of Arabic premise–hypothesis pairs. Each pair is labelled entail, contradict or neutral and tagged with coarse and fine phenomenon categories, many of which come from Arabic grammar. With E-CONAN they probe 9 pretrained language models and 5 LLMs. Overall, the LLMs score about the same as the pretrained baselines, but their strengths differ. LLMs do better on world knowledge and common sense and worse on syntax. Reasoning is the hardest area for every model. The value of the resource is that it breaks a single accuracy figure into a diagnosable profile.

Key points

  • The macro-categories are Lexical, Lexico-Syntactic, Syntactic, Discourse, Logic, and Knowledge & Common Sense, following Bentivogli et al.'s phenomena list. The authors add a new Pragmatics category.
  • The Arabic hierarchy has many grammar-specific micro-categories:
    • annuller particles (kāna-type and inna-type)
    • negation, emphasis, conditional, jazm and accusative particles
    • accusative cases such as the absolute object, causal object, exclusion, ḥāl and tamyīz
    • followers such as adjective and apposition.
  • The authors claim E-CONAN is larger than any prior NLU diagnostics set in any language. For comparison, the GLUE diagnostics has 1104 pairs, ALUE has 1147 and FraCaS has 346 problems.
  • The evaluation is zero-shot and covers multilingual NLI checkpoints such as mDeBERTa-V3-Base-XNLI-Multilingual-NLI-2mil7, Ernie-M-Base-MNLI-XNLI and Multilingual-MiniLMv2 variants, plus 5 LLMs.
  • Syntactic phenomena are the easiest for pretrained models, while Lexico-Syntactic phenomena are the easiest for LLMs. Reasoning is the hardest area for both groups.

Interesting ideas

  • I1. Each NLI pair is tagged with a two-level phenomenon path, such as macro → micro, so one test set yields per-phenomenon error profiles instead of a single score. — For Mova (learn): Tag Mova's evaluation items with phenomenon paths so that each regression can be traced to a specific construction rather than shown as a drop in an aggregate number.
  • I2. Every pair includes a short human explanation of why its label holds, which turns the dataset into a set of small, checkable inference rules. — For Mova (answer): Store a rationale alongside each test answer, so a wrong reader output can be compared against the expected reasoning step and not just the final label.
  • I3. The pairs are minimal contrasts that hinge on verb class. A once-only event such as dying or building at two different places or by two different agents yields contradiction. A repeatable event such as travelling to two places yields neutral. — For Mova (world): Mark verb classes as unique per entity or repeatable in the compiled knowledge, so the world model can detect conflicting event states instead of simply adding both events.
  • I4. Adding a modifier or state, such as "came laughing", to the hypothesis gives neutral, while dropping it gives entailment. A different time or place adverbial for the same single event gives contradiction. — For Mova (frames): Treat optional modifiers in frames as restrictive slots that are monotone when deleted but unsupported when added, and use this as a simple entailment check between two frame graphs.
  • I5. The pragmatic items rely on implicit assumptions behind utterances. A request to close the door implies the door is still open, and an impossible-wish particle such as layta implies the wished-for state does not hold. — For Mova (frames): Give speech acts and wish/hope markers presupposition effects on the world state, for example a request implying that its target state is currently false, and test these effects with entail/contradict pairs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2609.33530 · arXiv 2609.33530

@misc{jallad2026e,
  title = {{E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding}},
  author = {Jallad, Khloud AL and Ghneim, Nada and Rebdawi, Ghaida},
  year = {2026},
  doi = {10.48550/arxiv.2609.33530},
  eprint = {2609.33530},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2609.33530}
}

References

55 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. L. Bentivogli, E. Cabrio, I. Dagan, D. Giampiccolo, M. Leggio, and B. Magnini, “Building Textual Entailment Specialized Data Sets: a Methodology for Isolating Linguistic Phenomena Relevant to Inference.,” Mar. 2010. find
  2. Y. Wilks, “Natural language inference.,” 1973. [Online]. Available: https://api.semanticscholar.org/CorpusID:60246043 ↗
  3. I. Dagan and O. Glickman, “PROBABILISTIC TEXTUAL ENTAILMENT: GENERIC APPLIED MODELING OF LANGUAGE VARIABILITY,” 2004. [Online]. Available: https://api.semanticscholar.org/CorpusID:17200692 ↗
  4. L. Bentivogli, B. Magnini, I. Dagan, H. T. Dang, and D. Giampiccolo, “The Fifth PASCAL Recognizing Textual Entailment Challenge,” in Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17, 2009, NIST, 2009. [Online]. Available: https://tac.nist.gov/publications/2009/additional.papers/RTE5_overview.proceedings.pdf ↗
  5. O. Glickman and I. Dagan, “Probabilistic textual entailment: Generic applied modeling of language variability,” in Proceedings of the Workshop on Learning Methods for Text Understanding and Mining, 2004. find
  6. L. Bentivogli, P. Clark, I. Dagan, and D. Giampiccolo, “The Seventh PASCAL Recognizing Textual Entailment Challenge,” Theory and Applications of Categories, 2011, [Online]. Available: https://api.semanticscholar.org/CorpusID:5791809 ↗
  7. D. Giampiccolo, H. T. Dang, B. Magnini, I. Dagan, E. Cabrio, and W. B. Dolan, “The Fourth PASCAL Recognizing Textual Entailment Challenge,” in Text Analysis Conference, 2008. [Online]. Available: https://api.semanticscholar.org/CorpusID:12381965 ↗
  8. L. Bentivogli, P. Clark, I. Dagan, and D. Giampiccolo, “The Sixth PASCAL Recognizing Textual Entailment Challenge,” in Text Analysis Conference, 2009. [Online]. Available: https://api.semanticscholar.org/CorpusID:858065 ↗
  9. L. Bentivogli, B. Magnini, I. Dagan, H. T. Dang, and D. Giampiccolo, “The Fifth PASCAL Recognizing Textual Entailment Challenge,” in Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17, 2009, NIST, 2009. [Online]. Available: https://tac.nist.gov/publications/2009/additional.papers/RTE5_overview.proceedings.pdf ↗
  10. D. Giampiccolo, B. Magnini, I. Dagan, and B. Dolan, “The Third PASCAL Recognizing Textual Entailment Challenge,” in Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, S. Sekine, K. Inui, I. Dagan, B. Dolan, D. Giampiccolo, and B. Magnini, Eds., Prague: Association for Computational Linguistics, Jun. 2007, pp. 1–9. [Online]. Available: https://aclanthology.org/W07-1401 ↗
  11. R. Bar-Haim et al., “The Second PASCAL Recognising Textual Entailment Challenge,” 2006. [Online]. Available: https://api.semanticscholar.org/CorpusID:13385138 ↗
  12. O. and M. B. Dagan Ido and Glickman, “The PASCAL Recognising Textual Entailment Challenge,” in Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, I. and M. B. and d’Alché-B. F. Quiñonero-Candela Joaquin and Dagan, Ed., Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 177–190. find
  13. I. Dagan, D. Roth, M. Sammons, and F. M. Zanzotto, “Recognizing Textual Entailment: Models and Applications,” Synthesis Lectures on Human Language Technologies, vol. 6, pp. 1–220, Jul. 2013, doi: 10.2200/S00509ED1V01Y201305HLT023. ↗
  14. D. Giampiccolo, H. T. Dang, B. Magnini, I. Dagan, E. Cabrio, and W. B. Dolan, “The Fourth PASCAL Recognizing Textual Entailment Challenge,” in Text Analysis Conference, 2008. [Online]. Available: https://api.semanticscholar.org/CorpusID:12381965 ↗
  15. H. J. Levesque, E. Davis, and L. Morgenstern, “The Winograd Schema Challenge,” in AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, 2011. [Online]. Available: https://api.semanticscholar.org/CorpusID:15710851 ↗
  16. N. and B. S. Williams Adina and Nangia, “A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Association for Computational Linguistics, 2018, pp. 1112–1122. [Online]. Available: http://aclweb.org/anthology/N18-1101 ↗
  17. S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning, “A large annotated corpus for learning natural language inference,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, L. Màrquez, C. Callison-Burch, and J. Su, Eds., Lisbon, Portugal: Association for Computational Linguistics, Sep. 2015, pp. 632–642. doi: 10.18653/v1/D15-1075. ↗
  18. A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. Bowman, “GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding,” in Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Brussels, Belgium: Association for Computational Linguistics, Nov. 2018, pp. 353– 355. doi: 10.18653/v1/W18-5446. ↗ our card
  19. A. Chowdhery et al., “PaLM: Scaling Language Modeling with Pathways,” ArXiv, vol. abs/2204.02311, 2022, [Online]. Available: https://api.semanticscholar.org/CorpusID:247951931 ↗
  20. Q. Zhong et al., “Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE,” ArXiv, vol. abs/2212.01853, 2022, [Online]. Available: https://api.semanticscholar.org/CorpusID:254246784 ↗
  21. Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” 2019. find
  22. Z. Zhang et al., “Semantics-aware BERT for Language Understanding,” in AAAI Conference on Artificial Intelligence, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:202539891 ↗
  23. Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V Le, “XLNet: Generalized Autoregressive Pretraining for Language Understanding,” in Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alché-Buc, E. Fox, and R. Garnett, Eds., Curran Associates, Inc., 2019. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2019/file/dc6a7e655d7e5840e66733e9ee67cc69-Paper.pdf ↗ our card
  24. S. Soltan et al., “AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model,” ArXiv, vol. abs/2208.01448, 2022, [Online]. Available: https://api.semanticscholar.org/CorpusID:251253416 ↗
  25. S. Wu et al., “BloombergGPT: A Large Language Model for Finance,” ArXiv, vol. abs/2303.17564, 2023, [Online]. Available: https://api.semanticscholar.org/CorpusID:257833842 ↗ our card
  26. S. Banerjee, A. Mahajan, A. Agarwal, and E. Singh, “First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI,” CoRR, vol. abs/2412.09263, 2024, doi: 10.48550/ARXIV.2412.09263. ↗
  27. L. Xue et al., “ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models,” Trans. Assoc. Comput. Linguist., vol. 10, pp. 291–306, 2022, doi: 10.1162/tacl_a_00461. ↗
  28. H. W. Chung, T. Févry, H. Tsai, M. Johnson, and S. Ruder, “Rethinking embedding coupling in pre-trained language models,” ArXiv, vol. abs/2010.12821, 2020, [Online]. Available: https://api.semanticscholar.org/CorpusID:225067567 ↗ our card
  29. O. Shliazhko, A. Fenogenova, M. Tikhonova, A. Kozlova, V. Mikhailov, and T. Shavrina, “mGPT: Few-Shot Learners Go Multilingual,” Trans. Assoc. Comput. Linguist., vol. 12, pp. 58–79, 2024, doi: 10.1162/tacl_a_00633. ↗ our card
  30. P. He, X. Liu, J. Gao, and W. Chen, “DeBERTa: Decoding-enhanced BERT with Disentangled Attention,” 2021. find our card
  31. M. Joshi, D. Chen, Y. Liu, D. S. Weld, L. Zettlemoyer, and O. Levy, “SpanBERT: Improving Pre-training by Representing and Predicting Spans,” Trans. Assoc. Comput. Linguist., vol. 8, pp. 64–77, 2020, doi: 10.1162/tacl_a_00300. ↗ our card
  32. F. Iandola, A. Shaw, R. Krishna, and K. Keutzer, “SqueezeBERT: What can computer vision teach NLP about efficient neural networks?,” in Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing, N. S. Moosavi, A. Fan, V. Shwartz, G. Glavaš, S. Joty, A. Wang, and T. Wolf, Eds., Online: Association for Computational Linguistics, Nov. 2020, pp. 124–135. doi: 10.18653/v1/2020.sustainlp-1.17. ↗
  33. V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” ArXiv, vol. abs/1910.01108, 2019, [Online]. Available: https://api.semanticscholar.org/CorpusID:203626972 ↗
  34. M. Alabbas, “A Dataset for Arabic Textual Entailment,” in Proceedings of the Student Research Workshop associated with RANLP 2013, I. Temnikova, I. Nikolova, and N. Konstantinova, Eds., Hissar, Bulgaria: INCOMA Ltd. Shoumen, BULGARIA, Sep. 2013, pp. 7–13. [Online]. Available: https://aclanthology.org/R13-2002 ↗
  35. K. Al Jallad and N. Ghneim, “ARNLI: ARABIC NATURAL LANGUAGE INFERENCE ENTAILMENT AND CONTRADICTION DETECTION,” Computer Science, vol. 24, no. 2, Mar. 2023, doi: 10.7494/csci.2023.24.2.4378. ↗ our card
  36. R. Obeidat, Y. Al-Harahsheh, M. Al-Ayyoub, and M. Gharaibeh, “ArEntail: manually-curated Arabic natural language inference dataset from news headlines,” Lang. Resour. Eval., 2024, doi: 10.1007/s10579-024-09731-1. ↗
  37. Ž. Agić and N. Schluter, “Baselines and Test Data for Cross-Lingual Inference,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), N. Calzolari, K. Choukri, C. Cieri, T. Declerck, S. Goggi, K. Hasida, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, S. Piperidis, and T. Tokunaga, Eds., Miyazaki, Japan: European Language Resources Association (ELRA), May 2018. [Online]. Available: https://aclanthology.org/L18-1614 ↗
  38. A. Conneau et al., “XNLI: Evaluating Cross-lingual Sentence Representations,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, Eds., Brussels, Belgium: Association for Computational Linguistics, Oct. 2018, pp. 2475–2485. doi: 10.18653/v1/D18-1269. ↗
  39. W. Yin, J. Hay, and D. Roth, “Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), K. Inui, J. Jiang, V. Ng, and X. Wan, Eds., Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 3914–3923. doi: 10.18653/v1/D19-1404. ↗ our card
  40. M. Lewis et al., “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds., Online: Association for Computational Linguistics, Jul. 2020, pp. 7871–7880. doi: 10.18653/v1/2020.acl-main.703. ↗ our card
  41. Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” ArXiv, vol. abs/1907.11692, 2019, [Online]. Available: https://api.semanticscholar.org/CorpusID:198953378 ↗
  42. W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou, “MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers,” ArXiv, vol. abs/2002.10957, 2020, [Online]. Available: https://api.semanticscholar.org/CorpusID:211296536 ↗
  43. P. He, J. Gao, and W. Chen, “DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing,” ArXiv, vol. abs/2111.09543, 2021, [Online]. Available: https://api.semanticscholar.org/CorpusID:244346093 ↗ our card
  44. M. Laurer, W. van Atteveldt, A. Casas, and K. Welbers, “Less annotating, more classifying: Addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT-NLI,” Political Analysis, vol. 32, no. 1, pp. 84–100, Jan. 2024, doi: 10.1017/pan.2023.20. ↗
  45. R. Cooper et al., “Using the framework. Technical Report,” 1996. find
  46. B. MacCartney and C. D. Manning, “An extended model of natural logic,” in Proceedings of the Eighth International Conference on Computational Semantics, in IWCS-8 ’09. USA: Association for Computational Linguistics, 2009, pp. 140– 156. find
  47. M.-Y. Day and Y.-J. Wang, “Analysis of identifying linguistic phenomena for recognizing inference in text,” Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration, IEEE IRI 2014, pp. 607–612, Mar. 2015, doi: 10.1109/IRI.2014.7051945. ↗
  48. A. Ravichander, A. Naik, C. Rose, and E. Hovy, “EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference,” Mar. 2019, pp. 349–361. doi: 10.18653/v1/K19-1033. ↗ our card
  49. S. Sugawara, Y. Kido, H. Yokono, and A. Aizawa, “Evaluation Metrics for Machine Reading Comprehension: Prerequisite Skills and Readability,” Mar. 2017, pp. 806–817. doi: 10.18653/v1/P17-1075. ↗
  50. G. Angeli and C. D. Manning, “NaturalLI: Natural Logic Inference for Common Sense Reasoning,” in Conference on Empirical Methods in Natural Language Processing, 2014. [Online]. Available: https://api.semanticscholar.org/CorpusID:2854390 ↗
  51. Y. Otmakhova, T. Baldwin, T. Cohn, K. Verspoor, and J. Lau, Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation. 2022. doi: 10.48550/arXiv.2210.03256. ↗
  52. A. Poliak, Y. Belinkov, J. Glass, and B. Van Durme, “On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), M. Walker, H. Ji, and A. Stent, Eds., New Orleans, Louisiana: Association for Computational Linguistics, Jun. 2018, pp. 513–523. doi: 10.18653/v1/N18-2082. ↗
  53. H. Seelawi et al., “ALUE: Arabic Language Understanding Evaluation,” in Proceedings of the Sixth Arabic Natural Language Processing Workshop, N. Habash, H. Bouamor, H. Hajj, W. Magdy, W. Zaghouani, F. Bougares, N. Tomeh, I. Abu Farha, and S. Touileb, Eds., Kyiv, Ukraine (Virtual): Association for Computational Linguistics, Apr. 2021, pp. 173–184. [Online]. Available: https://aclanthology.org/2021.wanlp-1.18 ↗
  54. “Madinah Arabic.” Accessed: Jan. 01, 2024. [Online]. Available: https://www.madinaharabic.com/arabic-language-course/lessons/يراصنلأا الله ديبع نب دمحم نب نمحرلا دبع and يرابنلأا نيدلا لامك تاكربلا وبأ , ةيبرعلا رارسأ . مقرلأا يبأ نب مقرلأا راد , 1999. Accessed: ↗
  55. Jul. 12, 2025. [Online]. Available: https://shamela.ws/book/7502 يئارماسلا حلاص لضاف , وحنلا يناعم , 1st ed. ندرلأا : عيزوتلاو رشنلاو ةعابطلل ركفلا راد , 2000. Accessed: Jul. 12, 2025. [Online]. Available: https://shamela.ws/book/17801 يئارماسلا حلاص لضاف , اهماسقأو اهفيلأت ةيبرعلا ةلمجلا . ءارتبلا قوس ( يريجحلا ) - ينيسحلا عماجلا ةحاس : نوعزومو نورشان ركفلا راد , 2007. “Mawdoo.” Accessed: Jan. 01, 2024. [Online]. Available: https://mawdoo3.com/ فينصت : دعاوق _ ةغللا _ ةيبرعلا [58] [59] “Loghate.” Accessed: Jan. 01, 2025. [Online]. Available: https://loghate.com/[60] P. Paroubek, S. Chaudiron, and L. Hirschman, “Principles of Evaluation in Natural Language Processing,” in Traitement Automatique des Langues, Volume 48, Numéro 1 : Principes de l’évaluation en Traitement Automatique des Langues [Principles of Evaluation in Natural Language Processing], P. Paroubek, S. Chaudiron, and L. Hirschman, Eds., France: ATALA (Association pour le Traitement Automatique des Langues), 2007, pp. 7–31. [Online]. Available: https://aclanthology.org/2007.tal-1.1 [61] K. Al Jallad, N. Ghneim, and G. Rebdawi, “Survey of NLU Benchmarks Diagnosing Linguistic Phenomena: Why not Standardize Diagnostics Benchmarks?,” 2025. [Online]. Available: https://arxiv.org/abs/2507.20419 [62] A. Ritter, S. Soderland, D. Downey, and O. Etzioni, It’s a Contradiction -- No, it’s Not: A Case Study using Functional Relations. 2008. [63] “Arabic Language Course Lessons,” Madinah Arabic FZE. يلع راتخم يلع نيدلا زع , “ هتلالادو هتاودأ يفنلا بولسأ ,” نيلينلا بادآ ةلجم , vol. 1, no. 3, pp. 259–300, 2011. [64] بيقرلا فسوي , “ وحنلا يف تابثلإاو يفنلا نيب ةلمجلا ماكحأ ,” Taif University, Taif, 2013. [65] [66] Mustafa Siddiqui, Arabic Grammar Unlocked: A complete study of the Ajurroomiyyah , 1st ed. Independently Published, 2019. [67] H. Seelawi et al., “ALUE: Arabic Language Understanding Evaluation,” in Proceedings of the Sixth Arabic Natural Language Processing Workshop, N. Habash, H. Bouamor, H. Hajj, W. Magdy, W. Zaghouani, F. Bougares, N. Tomeh, I. Abu Farha, and S. Touileb, Eds., Kyiv, Ukraine (Virtual): Association for Computational Linguistics, Apr. 2021, pp. 173–184. [Online]. Available: https://aclanthology.org/2021.wanlp-1.18 [68] G. Wenzek et al., “CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data,” in Proceedings of the Twelfth Language Resources and Evaluation Conference, N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, and S. Piperidis, Eds., Marseille, France: European Language Resources Association, May 2020, pp. 4003–4012. [Online]. Available: https://aclanthology.org/2020.lrec-1.494 [69] A. Conneau et al., “Unsupervised Cross-lingual Representation Learning at Scale,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds., Online: Association for Computational Linguistics, Jul. 2020, pp. 8440–8451. doi: 10.18653/v1/2020.acl-main.747. [70] M. S. Bari et al., “ALLaM: Large Language Models for Arabic and English,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=MscdsFVZrN [71] A. Yang et al., “Qwen2 Technical Report,” arXiv preprint arXiv:2407.10671, 2024. [72] A. Yang et al., “Qwen2.5 Technical Report,” arXiv preprint arXiv:2412.15115, 2024. [73] G. Team, “Gemma 3,” 2025, [Online]. Available: https://goo.gle/Gemma3Report [74] Y. Alnumay et al., “Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLM,” 2025. [Online]. Available: https://arxiv.org/abs/2503.14603 [75] DeepSeek-AI et al., “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning,” 2025. [Online]. Available: https://arxiv.org/abs/2501.12948 A PPENDIX Macro Category Level_1 Micro Category Identity/Mismatch Lexical Semantics Lexical Factivity Morphological Negation Unique Action Verbs Symmetry/Collectivity Redundancy Named Entities Quantifiers Genitives/Partitives (Idafa) Alternations Lexico-Syntactic Particles https://www.madinaharabic.com/arabic-language-course/lessons/L060_001.html Level_2 Micro Category Level_3 Micro Category Strings - Numbers - Dates - Format - Acronymy - Demonymy - Synonymy - Semantic Opposition (Contrast) - Hypernymy - Attitudes Verbs - Believe Verbs - Probability Verbs - - - - - - - - - - - - - Transparent head - With Modifier - Nominalization/Verbalization - Causative/Inchoative - Waw Letter واولا Ma Word ام La Word لا Annuller Particles 16 )ةخسانلا فورحلا( - - - Verbal Particles 17 اهتاوخأو ناك Comparative Phrases Superlative Phrases Argument Realization (Core Arguments) The Accusative Cases )تابوصنملا( Syntactic The followers 32 )عباوتلا( https://www.madinaharabic.com/arabic-language-course/lessons/L059_001.html https://www.madinaharabic.com/arabic-language-course/lessons/L060_002.html 19 https://www.madinaharabic.com/arabic-language-course/lessons/L043_001.html 20 https://www.madinaharabic.com/arabic-language-course/lessons/L031_007.html 21 https://www.madinaharabic.com/arabic-language-course/lessons/22 https://www.madinaharabic.com/arabic-language-course/lessons/23 https://www.madinaharabic.com/arabic-language-course/lessons/L036_006.html 24 https://www.madinaharabic.com/arabic-language-course/lessons/L049_001.html 25 https://arabikey.com/the-absolute-object-in-arabic/26 https://www.madinaharabic.com/arabic-language-course/lessons/L068_006.html 27 https://www.madinaharabic.com/arabic-language-course/lessons/L073_001.html 28 https://www.madinaharabic.com/arabic-language-course/lessons/L071_001.html 29 https://www.madinaharabic.com/arabic-language-course/lessons/L072_001.html 30 https://www.madinaharabic.com/arabic-language-course/lessons/L069_001.html 31 https://www.madinaharabic.com/arabic-language-course/lessons/L066_001.html 32 https://www.madinaharabic.com/arabic-language-course/lessons/L078_001.html 33 https://www.madinaharabic.com/arabic-language-course/lessons/L078_002.html Verb-Like Particles 18 اهتاوخأو نإ Interrogative Words 19 )ماهفتسلاا تاودأ( Negation Particles 20 )يفنلا تاودأ( Emphasis Particles 21 )ديكوتلا تاودأ( The Accusative Particles 22 )بصنلا تاودأ( The Jazm Particles 23 )مزجلا تاودأ( Conditional Particles 24 )طرشلا تاودأ( Redressing Particles )كاردتسلاا تاودأ( - - - - - - - - - - - - Absolute Object 25 (قلطم لوعفم) Causal Object 26 )هلجلأ لوعفم( The Exclusion 27 (ىنثتسم) Al-Hal (Status) 28 (لاحلا) Al-tamyīz (The Distinctive) 29 (زي ي ِ م ْ َتلا) Circumstantial Object 30 (هيف لوعفم) Direct Object 31 (هب لوعفم) Adjective 33 )تعنلا( Apposition 34 (Substitution 35 ) )لدبلا( - - - - - - - - Syntactic Semantic Attachment Ambiguity Modifier (ةفصلا/تعنلا) Alternations Anaphora/ Coreference Apposition Discourse Ellipsis/Implicits Propositional Structure Negation Structure Quantifiers Logic Logical Intervals Monotonicity https://corpus.quran.com/documentation/apposition.jsp https://www.fluentarabic.net/ajrumiyyah-arabic-grammar-5/36 https://www.madinaharabic.com/arabic-language-course/lessons/37 https://www.madinaharabic.com/arabic-language-course/lessons/L079_001.html Emphasis 36 )ديكوتلا( Coordinating 37 )فطعلا( Coordination Sentence Level Relative Clauses Prepositional Phrase Phrase Level Adverbial Phrase Intersectivity (عطاقتلا) Restrictivity (دييقتلا) - - Dative Alternation - Active/Passive Alternation - Topicalization Alternation - - - - - - - Conditional - Conjunction (قفاوت فطع) Disjunction (قفارت فطع) Coordination Negation - Double Negation - Universal - Existential - Temporal Intervals - Numerical Intervals - Spatial Intervals - Upward Monotone - Downward Monotone - Non-Monotone - Knowledge Knowledge & Common Sense Common Sense Pragmatic - Table 5: Hierarchy of E-CONAN Diagnostics Dataset Text Hypothesis نويروس نوثحاب ثحبلا اذه بتك ةيروس نم نوثحاب ثحبلا اذه بتك لهس ناحتملاا تننظ لهس ناحتملاا ارون ملعلا تدجو رون ملعلا اموي دوعي بابشلا تيل اموي دوعي بابشلا لهس ناحتملاا لهس ناحتملاا تيل ماعلا اذه ريفو لوصحملا ماعلا اذه ريفو لوصحملا لعل مداقلا ماعلا يفيلح نوكي حاجنلا لعل مداقلا ماعلا يفيلح نوكيس حاجنلا ةيرهزملا ديز رسك دصق ريغ نع ةيرهزملا ديز رسك دصق ريغ نع ةيرهزملا ديز رسك ةيرهزملا ديز رسك ورمع رصقلا اذه ىنب ديز رصقلا اذه ىنب ورمع رصقلا اذه م مر ديز رصقلا اذه م مر قشمد ىلإ ديز رفاس صمح ىلإ ديز رفاس - - Macro - Micro Categories Class Explanation Lexical-Lexical Semantics-Demonymy Entail Imagining an event is not the same as experiencing it. When we use "believe" verbs, we express certainty that the event has occurred. Lexical -Factivity-Probability/ Believe Verbs Neutral Entail “تيل” Particles means hoping something impossible Contradict Lexico-Syntactic-Annuller Particles-Verb-like Particles " لعل" Particles means hoping something may be possible in future, may happen or not. Moreover, we do not hope something already happened. So, we will have contradiction label if the event is in premise as it already happened. Contradict Neutral While it's ok to have more details and arguments in text, we can't make assumptions in hypothesis that are not clearly provided in text. (Restrictivity) Neutral Lexico-Syntactic-Core Args Entail Contradict “build”, “die” are unique actions that happens once for the same entity. While “rebuilt”, “travel” are not unique functions and can be repeated over time. Lexical-Unique Action Verbs Neutral Neutral صمح يف ديز تام قشمد يف ديز تام اكحاض لجرلا ءاج لجرلا ءاج لجرلا ءاج اكحاض لجرلا ءاج احابص ةهكافلا تلكأ ءاسم ةهكافلا تلكأ لزنملا يف ةهكافلا تلكأ لمعلا يف ةهكافلا تلكأ قلغم نلآا بابلا كلضف نم بابلا قلغأ حوتفم نلآا بابلا كلضف نم بابلا قلغأ ماوعأ ةدع لبق انوروك ءابو رشتنا يلاحلا نرقلا يف انوروك ءابو رشتنا ماوعأ ةدع لبق انوروك ءابو رشتنا يضاملا نرقلا يف انوروك رشتنا Table 6: Examples from E-CONAN Diagnostics Data Contradict Neutral Syntactic-Accusative Cases-State It’s the same as restrictivity in logic domain. Entail Syntactic-Accusative Cases-Circumstantial Object Contradict Different times or places. )ناكم فرظ وأ نامز فرظ( Contradict Contradict We do not request doing something already done. Pragmatic Entail Entail Pragmatic Contradict ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jallad-2026-e-conan-entailment-contradition-neutral/