Elamur Research⌕
☰
home / papers / 2026 / SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection

SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection

Paper · 2026 · Huopu Zhang, Yanguang Liu, Miao Zhang et al.

Summary

The paper addresses forecasting whether a company will beat or miss earnings expectations using long, noisy financial texts. These texts often exceed 5,000 words and are full of boilerplate and jargon, which makes classifiers prone to overfitting. The authors propose SAE-FiRE. A frozen decoder-only LLM (Gemma 2) encodes each document of up to 20,000 tokens. Pretrained Sparse Autoencoders, loaded via SAELens, then map residual-stream activations into a wide, sparse feature space, and the token-level features are sum-pooled into a single document vector. Two scoring methods rank the features: an ANOVA F-test and gradient-boosted tree importance. Only the top-k features feed an ℓ2-regularized logistic regression that separates positive from negative surprises, defined by SUE thresholds. Evaluation covers three corpora (earnings call transcripts, 10-Q filings and FNSPID Nasdaq News) with time-ordered splits. SAE-FiRE beats the baselines on all three. Because SAE features can be looked up in Neuronpedia, the approach also offers some interpretability, which is relevant for finance applications.

Key points

  • Labels come from Standardized Unexpected Earnings. SUE ≥ 0.5 counts as positive and SUE ≤ −0.5 as negative, and near-zero cases are dropped. Train, validation and test sets are split chronologically.
  • On Conference Call data, switching from Gemma 2-2B (16K SAE) to Gemma 2-9B (131K SAE) improves results:
    • accuracy rises from 0.793 to 0.801;
    • AUC rises from 0.657 to 0.668;
    • weighted F1 rises from 0.743 to 0.757.
  • For Gemma 2-2B, the middle layer (12) outperforms early layer 5 and late layer 20 (accuracy 0.793, AUC 0.657, F1 0.743). The layer comparison used the top-1500 tree-selected features.
  • Neuronpedia lookups show the top features span sentiment and evaluation, monetary terms, legal and contract language, politics, urban development and healthcare.
  • The authors list several limitations:
    • the SAEs are general-purpose rather than tuned for finance;
    • the task is framed only as binary classification;
    • very high-dimensional activations raise deployment costs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations🎓Chinese University of Hong Kong, ShenzhenShenzhen · China47 works · 2,596 citations🎓New Jersey Institute of TechnologyNewark · United States15 works · 1,333 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksDecision Sciences · 209 worksManagement Science and Operations Research · 108 worksBusiness, Management and Accounting · 32 worksAccounting · 5 works

Cite

Cite as Huopu Zhang, Yanguang Liu, Miao Zhang, Zirui He and Mengnan Du. 2026. SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.265

@inproceedings{zhang2026sae,
  title = {{SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection}},
  author = {Huopu Zhang, Yanguang Liu, Miao Zhang, Zirui He and Mengnan Du},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.265},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.265}
}

References

24 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
  2. Joseph Bloom, Curt Tigges, Anthony Duong, and David Chanin. 2024. Saelens. https://github.com/ jbloomAus/SAELens. ↗
  3. Stephen Brown, Stephen A Hillegeist, and Kin Lo. 2004. Conference calls and information asymmetry. Journal of Accounting and Economics, 37(3):343–366. find
  4. Brian J Bushee, Ian D Gow, and Daniel J Taylor. 2018. Linguistic complexity in firm disclosures: Obfuscation or information? Journal of Accounting Research, 56(1):85–121. find
  5. Zihan Dong, Xinyu Fan, and Zhiyuan Peng. 2024. Fnspid: A comprehensive financial news dataset in time series. Preprint, arXiv:2402.06698. — [arXiv:2402.06698](https://arxiv.org/abs/2402.06698) find
  6. Eli Bartov. 1992. Patterns in Unexpected Earnings as an Explanation for Post-Announcement Drift. The Accounting Review, 67(3):610–622. Publisher: American Accounting Association. find
  7. Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan, and Neel Nanda. 2025. Do i know this entity? knowledge awareness and hallucinations in language models. In International Conference on Learning Representations (ICLR). find
  8. Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025. Scaling and evaluating sparse autoencoders. In International Conference on Learning Representations (ICLR). find
  9. Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, and 1 others. 2024. Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders. arXiv preprint arXiv:2410.20526. find
  10. Zirui He, Haiyan Zhao, Yiran Qiao, Fan Yang, Ali Payani, Jing Ma, and Mengnan Du. 2025. Saif: A sparse autoencoder framework for interpreting and steering instruction following of language models. arXiv preprint arXiv:2502.11356. — [arXiv:2502.11356](https://arxiv.org/abs/2502.11356) find
  11. Allen H Huang, Hui Wang, and Yi Yang. 2023. Finbert: A large language model for extracting information from financial text. Contemporary Accounting Research, 40(2):806–841. find
  12. Ross Koval, Nicholas Andrews, and Xifeng Yan. 2023. Forecasting earnings surprises from conference call transcripts. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8197–8209. find
  13. David F Larcker and Anastasia A Zakolyukina. 2012. Detecting deceptive discussions in conference calls. Journal of Accounting Research, 50(2):495–540. find
  14. Henry A Latane and Charles P Jones. 1979. Standardized unexpected earnings–1971-77. The Journal of Finance, 34(3):717–724. find
  15. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. arXiv preprint arXiv:2408.05147. find
  16. Johnny Lin. 2023. Neuronpedia: Interactive reference and tooling for analyzing neural networks. Software available from neuronpedia.org. find
  17. Tim Loughran and Bill McDonald. 2011. Liability or asset? the role of language in financial reports. Journal of Accounting Research, 49(1):135–178. find
  18. Cheng Luo, Enrichetta Ravina, Marco Sammon, and Luis M. Viceira. 2022. Retail investors’ contrarian behavior around news, attention, and the momentum effect. Technical report, Social Science Research Network. Posted: 5 Apr 2020; Last revised: 26 May 2022. find
  19. Yu Qin and Yi Yang. 2019. What you say and how you say it matters: Predicting stock volatility using verbal and vocal cues. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 390–401. Association for Computational Linguistics. find
  20. Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. 2024. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. Preprint, arXiv:2407.14435. — [arXiv:2407.14435](https://arxiv.org/abs/2407.14435) find
  21. Yunxin Sang and Yang Bao. 2022. Predicting corporate risk by jointly modeling company networks and dialogues in earnings conference calls. arXiv preprint arXiv:2206.06174. — [arXiv:2206.06174](https://arxiv.org/abs/2206.06174) find
  22. Ramit Sawhney, Puneet Mathur, Ayush Mangal, Piyush Khanna, Rajiv Ratn Shah, and Roger Zimmermann. 2020. Multimodal multi-task financial risk forecasting. In Proceedings of the 28th ACM International Conference on Multimedia, pages 456–465. ACM. find
  23. Dong Shu, Xuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, and Mengnan Du. 2025. A survey on sparse autoencoders: Interpreting the internal mechanisms of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025. Association for Computational Lingui… find
  24. Liu Yang, Mingyang Zhang, Cheng Li, Michael Bendersky, and Marc Najork. 2020. Beyond 512 tokens: Siamese multi-depth transformer-based hierarchical encoder for long-form document matching. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM), pages 172… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-sae-fire-enhancing-earnings-surprise/