Elamur Research⌕
☰
home / papers / 2026 / DSBKTE at ATE-IT: From Token Classification to Zero-Shot Generation: Two Approaches to Italian ATE at EVALITA 2026

DSBKTE at ATE-IT: From Token Classification to Zero-Shot Generation: Two Approaches to Italian ATE at EVALITA 2026

Paper · 2026 · Danil Smirnov, Bita Khashechian and Giorgio Maria Di Nunzio

Summary

The paper presents one team's submission to the first subtask (Subtask A) of the ATE-IT shared task, held as part of EVALITA 2026. The task is to extract domain terms, both single words and multiword expressions, from individual Italian sentences taken from waste-management administrative documents. The data come from the ItaIst_GRU corpus and mix formal administrative acts (regulations, service charters, tenders) with looser informative texts such as public notices. The authors built two separate pipelines that share the same JSON output. The first fine-tunes a pretrained Italian BERT model to tag term spans with BIO labels. The second prompts LLMs zero-shot, with no training data or in-context examples. Under the official exact-match, term-level evaluation, the supervised BERT system achieved the best F1 because it locates term boundaries more accurately. The zero-shot LLM system had higher recall but lower precision, since it tends to over-generate and produce overly long spans. The two methods make different kinds of errors, so the authors suggest hybrid systems are a promising direction. This is especially relevant for administrative domains where annotated data are scarce.

Key points

  • Data and evaluation: the dataset is derived from ItaIst_GRU and split into train, dev and test CSV files with one sentence–term pair per row, so rows must be grouped by sentence. Scoring uses micro-averaged Precision, Recall and F1 with exact string matching after normalization.
  • Supervised system: an Italian BERT model with a linear BIO classification head (B-TERM / I-TERM / O), built with HuggingFace AutoModelForTokenClassification and the Trainer API. It uses cross-entropy loss and Adam, with subword-to-word label alignment and no domain-adaptive pretraining.
  • Zero-shot system: the main model is Gemini-2.5-Flash. LLaMA 3.1 and GPT-5.1 were also tested, but Gemini gave the most stable formatting and the best dev-set scores. Sentences were processed in batches with a single fixed prompt.
  • Errors: long sentences, complex nominal phrases and enumerations hurt both systems. BERT tends to miss terms, while the LLM over-generates and often gets span boundaries wrong. LLM outputs also vary between runs because of non-determinism.
  • Future work: proposed hybrids include using LLM output as a fallback when BERT returns nothing, or LLM candidate generation followed by filtering. Other ideas are domain-adaptive pretraining, span-based models and adding syntactic constraints. The exact numeric scores were not in the text available for this summary.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PaduaPadua · Italy18 works · 4,260 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Danil Smirnov, Bita Khashechian and Giorgio Maria Di Nunzio. 2026. DSBKTE at ATE-IT: From Token Classification to Zero-Shot Generation: Two Approaches to Italian ATE at EVALITA 2026. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{smirnov2026dsbkte,
  title = {{DSBKTE at ATE-IT: From Token Classification to Zero-Shot Generation: Two Approaches to Italian ATE at EVALITA 2026}},
  author = {Danil Smirnov, Bita Khashechian and Giorgio Maria {Di Nunzio}},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.50/}
}

References

13 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. K. Kageura, B. Umino, Methods of automatic term recognition: A review, Terminology. International Journal of Theoretical and Applied Issues in Specialized Communication 3 (1996) 259–289. doi: 10.1075/term.3.2.03kag . — [doi:10.1075/term.3.2.03kag](https://doi.org/10.1075/term.3.2.03kag) find
  2. N. Cirillo, G. M. Di Nunzio, F. Vezzani, Ate-it at evalita 2026: Overview of the automatic term extraction italian testbed task, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026. find
  3. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… find
  4. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, NAACL-HLT (2019).
  5. G. M. Di Nunzio, S. Marchesin, G. Silvello, A systematic review of Automatic Term Extraction: What happened in 2022?, Digital Scholarship in the Humanities 38 (2023) i41–i47. doi: 10.1093/ llc/fqad030 . find
  6. K. Frantzi, S. Ananiadou, H. Mima, Automatic recognition of multi-word terms:. the Cvalue/NC-value method, International Journal on Digital Libraries 3 (2000) 115–130. doi: 10. 1007/s007999900023 . find
  7. A. Holtzman, J. Buys, L. Du, M. Forbes, Y. Choi, The curious case of neural text degeneration, in: International Conference on Learning Representations, 2020.
  8. L. Ouyang, J. Wu, X. Jiang, et al., Training language models to follow instructions with human feedback, in: Advances in Neural Information Processing Systems, 2022.
  9. Google DeepMind, Gemini: A family of highly capable multimodal models, 2024. URL: https://deepmind.google/technologies/gemini/. ↗
  10. Meta AI, The llama 3.1 language model, 2024. URL: https://ai.meta.com/llama/. ↗
  11. N. Cirillo, et al., Ate-it: Automatic terminology extraction for italian, in: Proceedings of EVALITA, 2025. URL: https://nicolacirillo.github.io/ate-it/, shared Task description. ↗
  12. N. Cirillo, D. Vellutino, D. Nicoletti, E. Sabarese, B. Rubino, Itaist_gru, 2025. URL: https://doi.org/ 10.5281/zenodo.15173712. doi: 10.5281/zenodo.15173712 . — [doi:10.5281/zenodo.15173712](https://doi.org/10.5281/zenodo.15173712) find
  13. HuggingFace, Huggingface transformers, 2024. URL: https://huggingface.co/transformers/. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/smirnov-2026-dsbkte-ate-it-token-classification/