Elamur Research⌕
☰
home / topics / Language models & pretraining / page 9

🧠 Language models & pretraining

1,079 papers · page 9 of 12 · 726–825 · the most cited first · all topics

Models that learn a language from raw text, and how they are pretrained, tuned and prompted.

1 … 8 9 10 11 12

  1. Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular DataMubashara Akhtar, Abhilash Shankarampeta, Vivek Gupta, et al.2023 · 38 citations
  2. DziriBERT: a Pre-trained Language Model for the Algerian DialectAmine Abdaoui, Mohamed Berrimi, Mourad Oussalah, et al.2021 · 38 citations
  3. OG-RAG: Ontology-grounded retrieval-augmented generation for large language modelsKartik Sharma, Peeyush Kumar, Yunqing Li2025 · 37 citations
  4. Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMsNicolas Boizard, Kevin El Haddad, Celine Hudelot, et al.2024 · 34 citations
  5. Language acquisition: do children and language models follow similar learning stages?Linnea Evanson, Yair Lakretz, Jean-Rémi King2023 · 33 citations
  6. RST Discourse Parsing with Second-Stage EDU-Level Pre-trainingNan Yu, Meishan Zhang, Guohong Fu, et al.2022 · 32 citations
  7. Probing Contextual Language Models for Common Ground with Visual RepresentationsGabriel Ilharco, Rowan Zellers, Ali Farhadi, et al.2021 · 31 citations
  8. Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling ApproachKoren Lazar, Benny Saret, Asaf Yehudai, et al.2021 · 30 citations
  9. Assessing Discourse Relations in Language Generation from GPT-2Wei-Jen Ko, Junyi Jessy Li2020 · 30 citations
  10. BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting EvidenceJiajie Jin, Yutao Zhu, Yujia Zhou, et al.2024 · 28 citations
  11. Observational Scaling Laws and the Predictability of Language Model PerformanceYangjun Ruan, Chris J. Maddison, Tatsunori Hashimoto2024 · 28 citations
  12. Exploring the Relationship between LLM Hallucinations and Prompt Linguistic Nuances: Readability, Formality, and ConcretenessVipula Rawte, Prachi Priya, S.M Towhidul Islam Tonmoy, et al.2023 · 27 citations
  13. Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal SamplingHritik Bansal, Arian Hosseini, Rishabh Agarwal, et al.2024 · 26 citations
  14. Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African ArabiziBenjamin Muller, Benoit Sagot, Djame Seddah2020 · 26 citations
  15. How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled BenchmarkMinglai Yang, Ethan Huang, Liang Zhang, et al.2025 · 24 citations
  16. A closer look at the performance of neural language models on reflexive anaphor licensingJennifer Hu, Sherry Yong Chen, Roger Levy2020 · 24 citations
  17. MacBERTh: Development and Evaluation of a Historically Pre-trained Language Model for English (1450-1950)Enrique Manjavacas Arevalo, Lauren Fonteyn2021 · 23 citations
  18. Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale PretrainingGrigorii Guz, Patrick Huber, Giuseppe Carenini2020 · 19 citations
  19. Do Large Language Models Understand Word Senses?Domenico Meconi, Simone Stirpe, Federico Martelli2025 · 18 citations
  20. CamelEval: Advancing Culturally Aligned Arabic Language Models and BenchmarksZhaozhi Qian, Faroq Altam, Muhammad Alqurishi, et al.2024 · 18 citations
  21. Construction Grammar Provides Unique Insight into Neural Language ModelsLeonie Weissweiler, Taiqi He, Naoki Otani, et al.2023 · 18 citations
  22. Evaluating statistical language models as pragmatic reasonersBenjamin Lipkin, Lionel Wong, Gabriel Grand, et al.2023 · 18 citations
  23. ArabianGPT: Native Arabic GPT-based Large Language ModelAnis Koubaa, Adel Ammar, Lahouari Ghouti, et al.2024 · 17 citations
  24. MultiPragEval: Multilingual Pragmatic Evaluation of Large Language ModelsDojun Park, Jiwoo Lee, Seohyun Park, et al.2024 · 17 citations
  25. Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models' Understanding of Discourse RelationsYisong Miao, Hongfu Liu, Wenqiang Lei, et al.2024 · 16 citations
  26. Is Child-Directed Speech Effective Training Data for Language Models?Steven Y. Feng, Noah D. Goodman, Michael C. Frank2024 · 16 citations
  27. Rethinking Pragmatics in Large Language Models: Towards Open-Ended Evaluation and Preference TuningShengguang Wu, Shusheng Yang, Zhenglun Chen, et al.2024 · 16 citations
  28. A Large and Diverse Arabic Corpus for Language ModelingAbbas Raza Ali, Muhammad Ajmal Siddiqui, Rema Algunaibet, et al.2022 · 16 citations
  29. Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question AnsweringRunxuan Liu, Bei Luo, Jiaqi Li, et al.2025 · 15 citations
  30. Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction CapabilitiesShaltiel Shmidman, Avi Shmidman, Amir DN Cohen, et al.2024 · 15 citations
  31. AraSTEM: A Native Arabic Multiple Choice Question Benchmark for Evaluating LLMs Knowledge In STEM SubjectsAhmad Mustapha; Hadi Al-Khansa; Hadi Al-Mubasher; Aya Mourad; Ranam Hamoud; Hasan El-Husseini; Marwah Al-Sakkaf;, Mariette Awad2024 · 15 citations
  32. Evaluating Large Language Models for Health-related Queries with PresuppositionsNavreet Kaur, Monojit Choudhury, Danish Pruthi2024 · 15 citations
  33. HeRo: RoBERTa and Longformer Hebrew Language ModelsVitaly Shalumov, Harel Haskey2023 · 15 citations
  34. Jais 2: A Family of Arabic-Centric Open Large Language ModelsMohamed Anwar, Abed Alhakim Freihat, George Ibrahim, et al.2026 · 14 citations
  35. Comparative Study of Multilingual Idioms and Similes in Large Language ModelsParia Khoshtab, Danial Namazifard, Mostafa Masoudi, et al.2025 · 14 citations
  36. Alignment at Pre-training! Towards Native Alignment for Arabic LLMsJuhao Liang, Zhenyang Cai, Jianqing Zhu, et al.2024 · 14 citations
  37. Resolving Discrepancies in Compute-Optimal Scaling of Language ModelsTomer Porian, Mitchell Wortsman, Jenia Jitsev, et al.2024 · 14 citations
  38. SaudiBERT: A Large Language Model Pretrained on Saudi Dialect CorporaFaisal Qarah2024 · 14 citations
  39. Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models — The Story Goes OnLiang Zeng, Liangjun Zhong, Liang Zhao, et al.2024 · 14 citations
  40. TigerCoder: A Novel Suite of LLMs for Code Generation in BanglaNishat Raihan, Antonios Anastasopoulos, Marcos Zampieri2026 · 13 citations
  41. SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages?Senyu Li, Jiayi Wang, Felermino D. M. A. Ali, et al.2025 · 13 citations
  42. from Benign import Toxic: Jailbreaking the Language Model via Adversarial MetaphorsYu Yan, Sheng Sun, Zenghao Duan, et al.2025 · 12 citations
  43. The Impact of Token Granularity on the Predictive Power of Language Model SurprisalByung-Doh Oh, William Schuler2025 · 12 citations
  44. When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM ReasoningNishad Singhi, Hritik Bansal, Arian Hosseini, et al.2025 · 12 citations
  45. A Construction Grammar Corpus of Varying Schematicity: A Dataset for the Evaluation of Abstractions in Language ModelsClaire Bonial, Harish Tayyar Madabushi2024 · 12 citations
  46. Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource LanguagesMichael Andersland2024 · 12 citations
  47. Towards Ontology Construction with Language ModelsMaurice Funk, Simon Hosemann, Jean Christoph Jung, et al.2023 · 12 citations
  48. How well can LLMs Grade Essays in Arabic?Rayed Ghazawi, Edwin Simpson2025 · 11 citations
  49. Memorization or Reasoning? Exploring the Idiom Understanding of LLMsJisu Kim, Youngwoo Shin, Uiji Hwang, et al.2025 · 11 citations
  50. Do Large Language Models Understand Conversational Implicature- A case study with a Chinese sitcomShisen Yue, Siyuan Song, Xinyuan Cheng, et al.2024 · 11 citations
  51. Pragmatic Competence Evaluation of Large Language Models for the Korean LanguageDojun Park, Jiwoo Lee, Hyeyun Jeong, et al.2024 · 11 citations
  52. Are Frequent Phrases Directly Retrieved like Idioms? An Investigation with Self-Paced Reading and Language ModelsGiulia Rambelli, Emmanuele Chersoni, Marco S. G. Senaldi, et al.2023 · 11 citations
  53. How Are Idioms Processed Inside Transformer Language Models?Ye Tian, Isobel James, Hye Son2023 · 11 citations
  54. LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are HighJudith Sieker, Clara Lachenmaier, Sina Zarrieß2025 · 10 citations
  55. EgyBERT: A Large Language Model Pretrained on Egyptian Dialect CorporaFaisal Qarah2024 · 10 citations
  56. Child-Directed Language Does Not Consistently Boost Syntax Learning in Language ModelsFrancesca Padovani, Jaap Jumelet, Yevgen Matusevych, et al.2025 · 9 citations
  57. When Your Language Model Cannot Even Do Determiners Right: Probing for Anti-Presuppositions and the Maximize Presupposition! PrincipleJudith Sieker, Sina Zarrieß2023 · 9 citations
  58. BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language modelBrooke Stephenson, Laurent Besacier, Laurent Girin, et al.2022 · 9 citations
  59. Can language models capture syntactic associations without surface cues? A case study of reflexive anaphor licensing in English control constructionsSoo-Hwan Lee, Sebastian Schuster2022 · 9 citations
  60. Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language UnderstandingAbbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.2022 · 9 citations
  61. From Guidelines to Practice: A New Paradigm for Arabic Language Model EvaluationSerry Sibaee, Omer Nacar, Adel Ammar, et al.2025 · 8 citations
  62. Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious QuestionsFarah Atif, Nursultan Askarbekuly, Kareem Darwish, et al.2025 · 8 citations
  63. Optimized Quran Passage Retrieval Using an Expanded QA Dataset and Fine-Tuned Language ModelsMohamed Basem; Islam Oshallah; Baraa Hikal; Ali Hamdi;, Ammar Mohamed2024 · 8 citations
  64. Bridging the Resource Gap: Exploring the Efficacy of English and Multilingual LLMs for SwedishOskar Holmström, Jenny Kunz, Marco Kuhlmann2023 · 8 citations
  65. We Understand Elliptical Sentences, and Language Models should Too: A New Dataset for Studying Ellipsis and its Interaction with Thematic FitDavide Testa, Emmanuele Chersoni, Alessandro Lenci2023 · 8 citations
  66. Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum LearningYang Zhang, Amr Mohamed, Hadi Abdine, et al.2026 · 7 citations
  67. This One or That One? A Study on Accessibility via Demonstratives with Multimodal Large Language ModelsYu Wang, Emmanuele Chersoni, Chu-Ren Huang2026 · 7 citations
  68. A Comprehensive Evaluation of Large Language Models on Mental Illnesses in Arabic ContextNoureldin Zahran, Aya E. Fouda, Radwa J. Hanafy, et al.2025 · 7 citations
  69. Bilingual Adaptation of Monolingual Foundation ModelsGurpreet Gosal, Yishi Xu, Gokul Ramakrishnan, et al.2024 · 7 citations
  70. Cutting Through the Noise: Boosting LLM Performance on Math Word ProblemsUjjwala Anantheswaran, Himanshu Gupta, Kevin Scaria, et al.2024 · 7 citations
  71. Explicit Morphological Knowledge Improves Pre-training of Language Models for HebrewEylon Gueta, Omer Goldman, Reut Tsarfaty2023 · 7 citations
  72. Towards more Human-like Language Models based on Contextualizer Pretraining StrategyChenghao Xiao, G Thomas Hudson, Noura Al Moubayed2023 · 7 citations
  73. Large-scale text pre-training helps with dialogue act recognition, but not without fine-tuningBill Noble, Vladislav Maraev2021 · 7 citations
  74. Comparing human and LLM politeness strategies in free productionHaoran Zhao, Robert D. Hawkins2025 · 6 citations
  75. Do LLMs Understand Dialogues? A Case Study on Dialogue ActsAyesha Qamar, Jonathan Tong, Ruihong Huang2025 · 6 citations
  76. IPA CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language ModelingZébulon Goriely, Paula Buttery2025 · 6 citations
  77. Pragmatic Inference Chain (PIC) Improving LLMs' Reasoning of Authentic Implicit Toxic LanguageXi Chen, Shuo Wang2025 · 6 citations
  78. Detecting Anti-Semitic Hate Speech using Transformer-based Large Language ModelsDengyi Liu, Minghao Wang, Andrew G. Catlin2024 · 6 citations
  79. Ensemble Language Models for Multilingual Sentiment AnalysisMd Arid Hasan2024 · 6 citations
  80. Large Language Models and Arabic Content: A ReviewHaneh Rhel, Dmitri Roussinov2025 · 5 citations
  81. LLM Dependency Parsing with In-Context RulesMichael Ginn, Alexis Palmer2025 · 5 citations
  82. Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language ModelsMohamed Taher Alrefaie, Nour Eldin Morsy, Nada Samir2024 · 5 citations
  83. Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses?Avi Shmidman, Cheyn Shmuel Shmidman, Dan Bareket, et al.2023 · 5 citations
  84. Assessing the Linguistic Knowledge in Arabic Pre-trained Language Models Using Minimal PairsWafa Abdullah Alrajhi, Hend Al-Khalifa, Abdulmalik AlSalman2022 · 5 citations
  85. Parameter and Data Efficient Continual Pre-training for Robustness to Dialectal Variance in ArabicSoumajyoti Sarkar, Kaixiang Lin, Sailik Sengupta, et al.2022 · 5 citations
  86. Large Language Models as Oracles for Ontology AlignmentSviatoslav Lushnei, Dmytro Shumskyi, Severyn Shykula, et al.2026 · 4 citations
  87. An Ensemble Classification Approach in A Multi-Layered Large Language Model Framework for Disease PredictionAli Hamdi, Malak Mohamed, Rokaia Emad, et al.2025 · 4 citations
  88. AraFinNews: Arabic Financial Summarisation with Domain-Adapted LLMsMo El-Haj, Paul Rayson2025 · 4 citations
  89. Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language ModelsRaha Askari, Sina Zarrieß, Özge Alacam, et al.2025 · 4 citations
  90. In the LLM era, Word Sense Induction remains unsolvedAnna Mosolova, Marie Candito, Carlos Ramisch2025 · 4 citations
  91. The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language TechnologyShahad Al-Khalifa; Nadir Durrani; Hend Al-Khalifa;, Firoj Alam2025 · 4 citations
  92. GemmAr: Enhancing LLMs Through Arabic Instruction-TuningHasna Chouikhi, Manel Aloui, Cyrine Ben Hammou, et al.2024 · 4 citations
  93. How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language ModelsMiriam Cuscito, Alfio Ferrara, Martin Ruskov2024 · 4 citations
  94. Multilingual Nonce Dependency Treebanks: Understanding how Language Models Represent and Process Syntactic StructureDavid Arps, Laura Kallmeyer, Younes Samih, et al.2024 · 4 citations
  95. Training LLMs to Recognize Hedges in Dialogues about Roadrunner CartoonsAmie Paige, Adil Soubki, John Murzaku, et al.2024 · 4 citations
  96. Chronological Thinking in Full-Duplex Spoken Dialogue Language ModelsDonghang Wu, Haoyang Zhang, Chen Chen, et al.2026 · 3 citations
  97. Do Diacritics Matter? Evaluating the Impact of Arabic Diacritics on Tokenization and LLM BenchmarksGo Inoue, Bashar Alhafni, Nizar Habash, et al.2026 · 3 citations
  98. Evaluating Multimodal Large Language Models on Vertically Written Japanese TextKeito Sasagawa, Shuhei Kurita, Daisuke Kawahara2026 · 3 citations
  99. Frame In, Frame Out: Measuring Framing Bias in LLM-Generated News SummariesValeria Pastorino, Nafise Sadat Moosavi2026 · 3 citations
  100. HalluScore: Large Language Model Hallucination Question Answering BenchmarkAisha Alansari, Hamzah Luqman2026 · 3 citations

1 … 8 9 10 11 12

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/topics/language-models/page/9/