Preslav Nakov · page 2
Papers
- Overview of the DSL Shared Task 2015 (2015 · 164 citations)
This report describes the second Discriminating between Similar Languages (DSL) shared task, held at the LT4VarDial'2015 workshop (RANLP'2015). - Automatic Stance Detection Using End-to-End Memory Networks (2018 · 157 citations)
This paper addresses stance detection as a step toward automatic fact checking. Given a claim and a news document, the system labels the document as agree, disagree, discuss, or unrelated, and it also points to the passages that justify the label. The authors build an end-to-end memory network in which each document… - SemEval-2023 Task 3: Detecting the Category, the Framing, and the Persuasion Techniques in Online News in a Multi-lingual Setup (2023 · 150 citations)
This paper presents a SemEval-2023 shared task on news analysis across languages. It goes beyond earlier propaganda-detection challenges by combining three views of an article. The first is its genre: opinion, objective reporting, or satire. The second is the framing, a multi-label choice among the 14 generic… - Fact-Checking Complex Claims with Program-Guided Reasoning (2023 · 146 citations)
Verifying real-world claims often means combining several pieces of evidence through multi-step reasoning. - Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic (2022 · 119 citations)
This paper reports on a shared task held at WANLP 2022 that aimed to bring propaganda-technique detection, previously studied almost only in English, to Arabic. - ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic (2024 · 117 citations)
Arabic-capable LLMs are usually tested for knowledge and reasoning on MMLU translated from English. Translation can introduce errors and leaves out content rooted in the Arab world. To fill this gap, the authors built ArabicMMLU, the first MMLU-style benchmark written natively in Modern Standard Arabic (MSA). They… - Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification (2024 · 96 citations)
LLMs often slip false statements into otherwise accurate text, and readers rarely notice them. The authors propose a fact-checking pipeline that needs no external knowledge base. The output is split into atomic claims (as in FactScore, via the OpenAI Chat API), and each claim is aligned with its tokens. Token-level… - Cross-Domain Label-Adaptive Stance Detection (2021 · 96 citations)
Stance detection, which identifies the position a text takes toward some target, comes in many incompatible forms. - Do-Not-Answer: Evaluating Safeguards in LLMs (2024 · 93 citations)
Open-source LLMs often ship without the safety layers that commercial systems have. Existing harmful-prompt collections tend to be small, proprietary, or unlabeled. This paper offers a cheap, public way to test whether a model refuses requests it should not fulfil. - ClaimRank: Detecting Check-Worthy Claims in Arabic and English (2018 · 91 citations)
ClaimRank addresses the first stage of fact-checking: deciding which sentences in a text deserve verification. - SemEval-2007 Task 04: Classification of Semantic Relations between Nominals (2007 · 91 citations)
This SemEval-2007 task set up a shared benchmark for deciding which semantic relation holds between two nominals (nouns or base noun phrases, not named entities) marked in an English sentence. - Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs (2023 · 78 citations)
Open-source LLMs often ship without strong safety mechanisms, and earlier harmful-prompt collections were either private, narrow in scope, or lacked labeled model responses. - Factuality of Large Language Models: A Survey (2024 · 77 citations)
Chat-oriented LLMs often give confident answers that contradict real-world facts, which limits their use in practice. - QCRI: Answer Selection for Community Question Answering - Experiments for Arabic and English (2015 · 76 citations)
The paper reports QCRI's entry in SemEval-2015 Task 3, which asks whether a comment in a Web forum thread actually answers the question that opened the thread. - SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images (2021 · 75 citations)
This shared task addresses the automatic identification of propaganda and persuasion devices in memes. Memes carry their message through both overlaid text and the picture, so studying text alone misses part of the signal. Building on SemEval-2020 task 11, which covered news text only, the organisers created a… - ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text (2023 · 67 citations)
ArAIEval, run with ArabicNLP 2023 (co-located with EMNLP 2023), is a shared task on two kinds of manipulative Arabic content. - Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness (2018 · 55 citations)
The paper reports on Task 1 of the CLEF-2018 CheckThat! Lab. Fact-checkers can only review a small share of what politicians say, so the task asks systems to rank the sentences of a debate or speech by how much they deserve verification. The organizers built the CT-CWC-18 corpus from 2016 US Presidential campaign… - SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios (2026 · 41 citations)
This shared task targets the problem of telling whether source code was written by people or produced by LLMs. - AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking (2021 · 41 citations)
Arabic fact-checking research has had few stance detection resources, and the main earlier one (Baly et al., 2018) draws its false claims from a single country and a narrow political topic. - Discourse Structure in Machine Translation Evaluation (2017 · 40 citations)
Standard metrics for machine translation evaluation, such as BLEU, mostly compare surface word overlap. They ignore how clauses within a sentence relate to one another. Joty and colleagues test whether sentence-level discourse structure can close part of this gap. They parse both system outputs and references into… - A Second Pandemic? Analysis of Fake News about COVID-19 Vaccines in Qatar (2021 · 33 citations)
The paper looks at how COVID-19 and vaccine discussion on Twitter in Qatar varies by language, and whether it carries false claims, manipulative rhetoric or particular perspectives. - AICD Bench: A Challenging Benchmark for AI-Generated Code Detection (2026 · 23 citations)
Tools for detecting machine-written source code are usually tested on narrow, custom datasets. These cover few languages and generators and mostly test human-vs-AI separation within the same distribution, which tells little about how detectors behave in real use. The authors address this with AICD Bench, a unified… - Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims (2019 · 23 citations)
The CheckThat! lab at CLEF 2019 is the second edition of a shared evaluation campaign on automated fact-checking. - Arabic Dataset for LLM Safeguard Evaluation (2025 · 22 citations)
Most safety benchmarks for large language models are built for English, with some extensions to Chinese. Arabic, with its cultural and political sensitivities, has had little coverage. The authors build an Arabic safety benchmark in two steps. First, they adapt the general-risk portion of the Chinese CDNA dataset… - Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025 · 19 citations)
The paper asks how jailbreak prompts get past an LLM's safety alignment. Earlier interpretability studies disagreed on which layers matter and on how malicious activations evolve. The authors argue those conflicts come from small samples, often around 100 prompts. They analyze last-token activations for more than…
This page in Markdown → https://md.research.elamur.ai/free/en/authors/preslav-nakov/page/2/