Elamur Research⌕
☰
home / papers / 2026 / Revisiting the Systematicity in Negation in the Era of In-Context Learning

Revisiting the Systematicity in Negation in the Era of In-Context Learning

Paper · 2026 · Hitomi Yanaka and Taisei Yamamoto

Summary

The authors ask whether in-context learning gives LLMs a systematic grasp of negation. They separate two notions. Behavioral systematicity concerns correct outputs on new inputs. Representational systematicity concerns whether the model internally encodes reusable task structure.

They use sentences from the SFU Review Corpus that contain one negation cue (not, no, n't). Two tasks are tested: finding the cue, and resolving its scope. Scope answers are requested in four output formats. Antonym prediction serves as a reference task.

  • Behavioral test: GPT-j-6B and Qwen3-4B are prompted with 10 demonstrations.
  • Representational test: function vectors and task vectors are extracted from the demonstrations and injected layer by layer into zero-shot inputs.

Both models find cues about as reliably as they predict antonyms, but neither is perfect. Scope accuracy depends heavily on the output format. Injected vectors trigger cue extraction well but scope resolution poorly.

This matters because it locates the bottleneck in negation understanding at scope. Strong benchmark scores alone may therefore hide a missing internal structure.

Key points

  • Scope formats: EXTRACT (output the scope tokens), BRACKET (mark the scope in square brackets), LAST-WORD (output the final scope token) and NER (0/1 label per token).
  • 10-shot accuracy, Qwen3-4B vs GPT-j-6B:
    • Cue extraction: 64.3% vs 62.0%.
    • EXTRACT: 76.0% vs 56.9%.
    • BRACKET: 69.8% vs 22.6%.
    • NER: 0% for both, so this format was dropped from the vector analysis.
  • Cue extraction via injected vectors: accuracy reached about 90% for GPT-j-6B and 80% for Qwen3-4B. Function vectors worked best in middle layers and task vectors in later layers.
  • Scope resolution via injected vectors: accuracy stayed between roughly 0% and 50%. Task vectors outperformed function vectors, especially in later layers. The authors suggest two possible reasons:
    • Task vectors replace the residual stream, whereas function vectors are added to it.
    • Task vectors may carry more abstract task information.
  • Evaluation method: vector effects were measured only on test items that the model answered correctly with 10 demonstrations, following Todd et al. (2024).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of TokyoTokyo · Japan96 works · 9,600 citations🔬RIKENWako · Japan30 works · 607 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Hitomi Yanaka and Taisei Yamamoto. 2026. Revisiting the Systematicity in Negation in the Era of In-Context Learning. Proceedings of the 6th Workshop on Natural Language Meets Logic and Machine Learning (NALOMA).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{yanaka2026revisiting,
  title = {{Revisiting the Systematicity in Negation in the Era of In-Context Learning}},
  author = {Hitomi Yanaka and Taisei Yamamoto},
  booktitle = {Proceedings of the 6th Workshop on Natural Language Meets Logic and Machine Learning (NALOMA)},
  year = {2026},
  url = {https://aclanthology.org/2026.naloma-1.1/}
}

References

1 references, 1 of them with a card of ours. ↗ leads to the work.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yanaka-2026-revisiting-systematicity-negation-era-context/