Elamur Research⌕
☰
home / papers / 2026 / LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification

LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification

Paper · 2026 · Michael Schlee, Christoph Weisser, Timo Kivimäki et al.

Summary

Commodity-focused financial NLP pipelines, such as gold or oil price forecasting from news sentiment, first need to filter out articles unrelated to the target asset. Supervised encoders like RoBERTa need many annotated examples for this filtering step, and they perform poorly when labels are scarce. The authors build a ten-topic multi-label benchmark from Reuters-21578, using the ModApte split and 9,034 documents. They compare a prompted GPT-5-nano, a fine-tuned RoBERTa, a TFIDF+LR baseline and their own LabelFusion model. LabelFusion appends the LLM's binary label vector to RoBERTa's [CLS] embedding and passes the result to a small trainable MLP that makes the final decision. Each model is trained on 5% to 100% of the training set. With all data, LabelFusion performs best. With limited data, however, prompting the LLM alone wins by a large margin, and it keeps that lead until roughly 80% of the labels are available. The practical guidance is to rely on LLM prompting when annotations are limited and to switch to fusion once enough labeled data exists. The method is released as a software package.

Key points

  • Dataset: Reuters-21578 is limited to the ten topics that occur most often. These cover corporate news (earn, acq), macro and financial themes (money-fx, interest, trade), commodities (crude, grain, wheat, corn) and shipping (ship).
    • Splits are 5,842 train, 647 validation (multilabel-stratified) and 2,545 test documents.
    • Classes are strongly imbalanced, from 3,964 earn documents down to 237 corn.
    • 9.2% of documents carry multiple labels.
  • Full-data results: LabelFusion reaches macro F1 96.0% and accuracy 92.3%. Standalone RoBERTa reaches F1 94.6% and GPT-5-nano reaches F1 93.9%.
  • Low-data results: zero-shot GPT-5-nano already achieves F1 75.9%. At 5% of the training data (292 documents), the prompted LLM scores F1 93.0, against 71.7 for Fusion and 37.2 for RoBERTa.
  • Architecture: the fused input is [h ; z] ∈ R^(768+K), built from the roberta-base embedding and the LLM's label predictions. It feeds an MLP with hidden layers [64, 32], and labels are assigned at a fixed 0.5 threshold.
  • Prompt construction: the prompt's role description is generated automatically through meta-prompting from labeled examples. GPT-5-nano is run at temperature 0.1.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of GöttingenGöttingen · Germany17 works · 158 citations🎓University of BathBath · United Kingdom5 works · 351 citations

Also on the byline: Hochschule Bielefeld · Centers for Disease Control and Prevention · Centers for Disease Control and Prevention.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksDecision Sciences · 209 worksManagement Science and Operations Research · 108 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Michael Schlee, Christoph Weisser, Timo Kivimäki, Melchizedek Mashiku and Benjamin Saefken. 2026. LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification. The 7th Financial Narrative Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5k4dk6iczjq5

@inproceedings{schlee2026labelfusion,
  title = {{LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification}},
  author = {Michael Schlee, Christoph Weisser, Timo Kivimäki, Melchizedek Mashiku and Benjamin Saefken},
  booktitle = {The 7th Financial Narrative Processing Workshop},
  year = {2026},
  doi = {10.63317/5k4dk6iczjq5},
  url = {https://doi.org/10.63317/5k4dk6iczjq5}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Harika Abburi et al. 2023. Generative ai text classification using ensemble llm approaches. arXiv preprint arXiv:2309.07755. — [arXiv:2309.07755](https://arxiv.org/abs/2309.07755) find
  2. Anonymous. 2025. Labelfusion: Learning to fuse llms and transformer classifiers for robust text classification. find
  3. Dogu Araci. 2019. FinBERT: Financial sentiment analysis with pre-trained language models. find
  4. Tom B. Brown et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901.
  5. Seung Chan Chae and Sun-Yong Choi. 2023. Forecasting the S&P 500 index using mathematical-based sentiment analysis and deep learning models. Axioms, 12(9):835. find
  6. Youngjin Chae and Thomas Davidson. 2025. Large language models for text classification. Sociological Methods & Research. find
  7. Yilin Chen, Xiaolong Li, and Yonghong Hu. 2024. Dual-attention transformer for financial news sentiment and stock price prediction. Expert Systems with Applications, 238:121134. find
  8. Franca Debole and Fabrizio Sebastiani. 2005. An analysis of the relative hardness of reuters-21578 subsets. Journal of the American Society for Information Science and Technology, 56(6):584– 596. find
  9. Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In ACL, pages 3816–3830.
  10. Jungwoo Gwak and Yoonsang Jung. 2025. Layer-aware embedding fusion for llms. arXiv preprint arXiv:2504.05764. — [arXiv:2504.05764](https://arxiv.org/abs/2504.05764) find
  11. Boshko Koloski, Senja Pollak, Roberto Navigli, and Blaž Škrlj. 2024. Automl-guided fusion of entity and llm-based representations. In ICONIP, pages 89–102. find
  12. David D. Lewis. 1997. Reuters-21578 text categorization test collection. Technical report, AT&T Labs Research. find
  13. Yilin Ma, Xin Gao, and Bowen Yang. 2024. LIAM: Label-interaction aware model for multi-label text classification. Neurocomputing, 580:127139. find
  14. Morteza Nabipour, Pejman Naeem, and Hamed Jabani. 2026. Federated learning for financial sentiment-augmented price prediction. In Proceedings of the 2026 International Conference on Machine Learning and Applications (ICMLA). IEEE. find
  15. Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification. In EACL, pages 255–269. find
  16. Konstantinos Sechidis, Grigorios Tsoumakas, and Ioannis Vlahavas. 2011. On the stratification of multi-label data. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD), pages 145–160, Berlin, Heidelberg. Springer. find
  17. Anand Sinha and Ravi Shastri. 2020. Impact of news sentiment on commodity returns. Journal of Behavioral Finance, 21(3):310–325. find
  18. Piotr Szymański and Tomasz Kajdanowicz. 2017. A network perspective on stratification of multi-label data. In Proceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications (LIDTA), pages 22–35. PMLR. find
  19. Zhiqiang Wang, Yiran Pang, and Yanbin Lin. 2023. Large language models are zero-shot text classifiers. arXiv preprint arXiv:2312.01044. find
  20. Jianfeng Yang, Yufei Wang, and Xiang Li. 2022. Prediction of stock price direction using the LASSO-LSTM model combining technical indicators and financial sentiment analysis. PeerJ Computer Science, 8:e1148. find
  21. Hao Yuan, Weiguang Han, and Yucheng Li. 2024. LACN: Label-aware co-training network for multi-label text classification. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), pages 3412–3422. ELRA and ICCL. find
  22. Dingkun Zeng, Erxue Zha, Jian Kuang, and Yong Shen. 2024. Multi-label text classification based on semantic-sensitive graph convolutional network. Knowledge-Based Systems, 282:111107. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/schlee-2026-labelfusion-fusing-large-language-models/