Elamur Research⌕
☰
home / papers / 2026 / SteuerLLM: Local specialized large language model for German tax law analysis

SteuerLLM: Local specialized large language model for German tax law analysis

Paper · 2026 · Sebastian Wind, Jeta Sopa, Laurin Schmid et al.

Summary

The paper asks whether general-purpose LLMs can handle German tax law. This domain punishes small mistakes: answers must cite the right statutes, follow a structured legal argument, and get the numbers exactly right. The authors make two contributions.

  • SteuerEx is an open benchmark of 115 expert-validated questions taken from real tax law exams at FAU. Each reference solution is split into separately scored legal statements, so a model can earn partial credit the way a student would.
  • SteuerLLM is a 28B-parameter model specialised for German tax law. It is trained on a large synthetic corpus built from authentic exam material through a retrieval-grounded generation pipeline. Instead of full fine-tuning, it uses block expansion: new Transformer layers are inserted and trained while the original weights stay frozen.

Compared with models from 3B to over 600B parameters, SteuerLLM beats general models of similar size and several much larger ones. The authors conclude that domain data and architectural adaptation matter more than raw scale. The benchmark, training data, and weights are released.

Key points

  • SteuerEx has 115 questions worth 1,035.5 points in total. They are drawn from undergraduate and graduate exams set between 2016 and 2024 and span six domains: the taxation of corporations, of personal income, and of partnerships, plus value-added tax, procedural rules for tax administration, and the basic principles of tax law.
  • Grading works one statement at a time. A secondary LLM judges each graded reference statement against the free-form answer and awards full or partial credit. The exam score is the sum of these statement scores.
  • Most general-purpose open models scored below 25% of the total points, which shows how hard authentic exams remain for LLMs.
  • Synthetic training data was produced mainly by Mistral-Large-Instruct-2407, with retrieval through SearxNG and query generation by a local llama3:8b. Training used Axolotl and alternated instruction and completion phases.
  • There are three variants: Small-SteuerLLM, SteuerLLM, and Open-SteuerLLM (28B, public on Hugging Face). The LLM-based grader is kept behind an external service to limit benchmark leakage.

Interesting ideas

  • I1. Break each gold answer into atomic, point-weighted claims and score every claim separately, so partly right reasoning still earns credit and errors can be pinned to specific claims.
  • I2. Block expansion adds fresh trainable layers on top of a frozen pretrained network, so the model gains a specialty without forgetting general skills.
  • I3. The grading prompts and logic are hidden behind a service until it shuts down, so the benchmark cannot leak into future training data.
  • I4. Model scores are set against anonymised student results per domain, which gives raw numbers a human reference point.
  • I5. Synthetic training data is seeded from real exam items and grounded by retrieving authoritative sources, which keeps generated examples close to real tasks.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): School-course test answers could be graded claim by claim against frame-level facts, showing exactly which inference step was missed.
  • I2 (tree): gparse could be adapted to a new treebank or UD dialect by training only added layers, protecting accuracy on the original English treebanks.
  • I3 (learn): Keeping a sealed held-out evaluation set, never shown to LLM teacher tiers, would make the negative controls more trustworthy.
  • I4 (answer): Reporting reader scores alongside typical pupil performance on the same school tests would show where Mova is truly weak.
  • I5 (learn): Silver data for frames and QA could start from real fables or test items, with retrieval-backed checks before the data enters training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓RWTH Aachen UniversityAachen · Germany22 works · 4,936 citations🎓Friedrich-Alexander-Universität Erlangen-NürnbergErlangen · Germany8 works · 48 citations🏥Universitätsklinikum AachenAachen · Germany3 works · 254 citations🎓University of Technology NurembergNuremberg · Germany6 works · 30 citations

Also on the byline: DATEV.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPolitical Science and International Relations · 182 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksLegal AI · 134 works

Cite

Cite as Sebastian Wind, Jeta Sopa, Laurin Schmid, Quirin Jackl, Sebastian Kiefer, Fei Wu, Martin Mayr, Harald Kostler, Gerhard Wellein, Andreas Maier and Soroosh Tayebi Arasteh. 2026. SteuerLLM: Local specialized large language model for German tax law analysis. arXiv:2602.11081 · Scientific Reports, Volume 16, 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1038/s41598-026-58485-z · arXiv 2602.11081

@misc{wind2026steuerllm,
  title = {{SteuerLLM: Local specialized large language model for German tax law analysis}},
  author = {Sebastian Wind, Jeta Sopa, Laurin Schmid, Quirin Jackl, Sebastian Kiefer, Fei Wu, Martin Mayr, Harald Kostler, Gerhard Wellein, Andreas Maier and Soroosh Tayebi Arasteh},
  year = {2026},
  doi = {10.1038/s41598-026-58485-z},
  eprint = {2602.11081},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1038/s41598-026-58485-z}
}

References

70 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Brown, T. B. et al. Language models are few-shot learners. in Proceedings of the 34th International Conference on Neural Information Processing Systems vol. 159 1877–1901 (2020).
  2. Vaswani, A. et al. Attention Is All You Need. in NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems 6000–6010 (2017).
  3. Wind, S. et al. Multi-step retrieval and reasoning improves radiology question answering with large language models. npj Digit. Med. 8, 790 (2025). find
  4. Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824–24837 (2022).
  5. Blair-Stanek, A., Holzenberger, N. & Van Durme, B. Can gpt-3 perform statutory reasoning? in Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law 22–31 (2023).
  6. Dugac, G. & Altwicker, T. Classifying legal interpretations using large language models. Artificial Intelligence and Law 1–19 (2025). find
  7. Dietterich, T. G. Steps toward robust artificial intelligence. Ai Magazine 38, 3–24 (2017). find
  8. Wei, J. et al. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021). ↗
  9. Marcus, G. The next decade in AI: four steps towards robust artificial intelligence. arXiv preprint arXiv:2002.06177 (2020). ↗
  10. Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. in Advances in Neural Information Processing Systems (eds Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F. & Lin, H.) vol. 33 9459–9474 (Curran Associates, Inc., 2020).
  11. Chalkidis, I. et al. LexGLUE: A benchmark dataset for legal language understanding in English. in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) 4310–4330 (2022).
  12. Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N. & Androutsopoulos, I. LEGAL-BERT: The muppets straight out of law school. arXiv preprint arXiv:2010.02559 (2020). ↗
  13. Li, J., Bhambhoria, R. & Zhu, X. Parameter-efficient legal domain adaptation. arXiv preprint arXiv:2210.13712 (2022). ↗
  14. Colombo, P. et al. Saullm-54b & saullm-141b: Scaling up domain adaptation for the legal domain. Advances in Neural Information Processing Systems 37, 129672–129695 (2024). find
  15. Hendrycks, D. et al. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020). ↗
  16. Rabelo, J. et al. Overview and discussion of the competition on legal information extraction/entailment (COLIEE) 2021. The Review of Socionetwork Strategies 16, 111–133 (2022). find
  17. Bernet, H. & Berteloot, P. EUR-Lex: A multilingual on-line website for European Union law. International Review of Law Computers & Technology 20, 337–339 (2006). find
  18. Aumiller, D., Chouhan, A. & Gertz, M. EUR-lex-sum: A multi-and cross-lingual dataset for long-form summarization in the legal domain. arXiv preprint arXiv:2210.13448 (2022). ↗
  19. Kirchhof, P. Einkommensteuergesetz: Kommentar. (Verlag Dr. Otto Schmidt, 2012). find
  20. Lang, J. & Tipke, K. Steuerrecht. (Otto Schmidt, 2020). find
  21. Grechenig, K. & Gelter, M. The transatlantic divergence in legal thought: American law and economics vs. German doctrinalism. Hastings Int’l & Comp. L. Rev. 31, 295 (2008). find
  22. McGinnis, J. O. & Pearce, R. G. The great disruption: How machine intelligence will transform the role of lawyers in the delivery of legal services. Fordham L. Rev. 82, 3041 (2013). find
  23. Wang, A. et al. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. in Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP 353–355 (Association for Computational Linguistics, Brussels, Belgium, 2018). doi:10.18653/v1/W18-5446. ↗
  24. Son, G. et al. KMMLU: Measuring Massive Multitask Language Understanding in Korean. in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) 4076–4104 (Association for Computational Linguistics, Albuquerque, New Mexico, 2025). doi:10.18653/v1/2025.naacl-long.206. ↗
  25. Rajpurkar, P., Zhang, J., Lopyrev, K. & Liang, P. SQuAD: 100,000+ Questions for Machine Comprehension of Text. in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing 2383–2392 (Association for Computational Linguistics, Austin, Texas, 2016). doi:10.18653/v1/D16-1264. ↗
  26. Wolfson, T. et al. Break It Down: A Question Understanding Benchmark. Transactions of the Association for Computational Linguistics 8, 183–198 (2020).
  27. Kaplan, J. et al. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020). ↗
  28. Hoffmann, J. et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 (2022). ↗
  29. Zheng, L. et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems 36, 46595–46623 (2023).
  30. Schick, T. & Schütze, H. Generating datasets with pretrained language models. arXiv preprint arXiv:2104.07540 (2021). ↗
  31. Eldan, R. & Li, Y. Tinystories: How small can language models be and still speak coherent english? arXiv preprint arXiv:2305.07759 (2023). ↗
  32. Katz, D. M., Bommarito II, M. J. & Blackman, J. A general approach for predicting the behavior of the Supreme Court of the United States. PloS one 12, e0174698 (2017). find
  33. Katz, D. M. & Bommarito, M. J. Measuring the complexity of the law: the United States Code. Artificial intelligence and law 22, 337–374 (2014). find
  34. Surden, H. Machine learning and law. Wash. L. Rev. 89, 87 (2014). find
  35. Guu, K., Lee, K., Tung, Z., Pasupat, P. & Chang, M. Retrieval augmented language model pre-training. in International conference on machine learning 3929–3938 (PMLR, 2020).
  36. Izacard, G. & Grave, E. Leveraging passage retrieval with generative models for open domain question answering. in Proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume 874–880 (2021).
  37. Crocetti, G. Textual Spatial Cosine Similarity. Preprint at https://doi.org/10.48550/arXiv.1505.03934 (2015). ↗
  38. Shao, M., Basit, A., Karri, R. & Shafique, M. Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges. IEEE Access 12, 188664–188706 (2024). find
  39. Huang, G., Sun, Y., Liu, Z., Sedra, D. & Weinberger, K. Q. Deep Networks with Stochastic Depth. in Computer Vision – ECCV 2016 (eds Leibe, B., Matas, J., Sebe, N. & Welling, M.) vol. 9908 646–661 (Springer International Publishing, Cham, 2016). find
  40. He, K., Zhang, X., Ren, S. & Sun, J. Identity Mappings in Deep Residual Networks. in Computer Vision – ECCV 2016 (eds Leibe, B., Matas, J., Sebe, N. & Welling, M.) vol. 9908 630–645 (Springer International Publishing, Cham, 2016). find
  41. Howard, J. & Ruder, S. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146 (2018). ↗
  42. Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. U.S.A. 114, 3521–3526 (2017). find
  43. Loshchilov, I. & Hutter, F. Decoupled Weight Decay Regularization. in Proceedings of Proceedings of Seventh International Conference on Learning Representations (ICLR) 2019 (New Orleans, LA, USA, 2019). find
  44. Shoeybi, M. et al. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. Preprint at https://doi.org/10.48550/arXiv.1909.08053 (2020). ↗
  45. Narayanan, D. et al. Efficient large-scale language model training on GPU clusters using megatron-LM. in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis 1–15 (ACM, St. Louis Missouri, 2021). doi:10.1145/3458817.3476209. ↗
  46. Schnitger, A. & Fehrenbacher, O. Kommentar Körperschaftsteuer KStG. (Springer, 2012). find
  47. Lloyd, A. C. The Logical Form of Law Statements. Mind 64, 312–318 (1955). find
  48. Zhong, H. et al. How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence. in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 5218–5230 (Association for Computational Linguistics, Online, 2020). doi:10.18653/v1/2020.acl-main.466. ↗
  49. Holtzman, A., Buys, J., Du, L., Forbes, M. & Choi, Y. The Curious Case of Neural Text Degeneration. in ICLR 2020 (2020).
  50. Touvron, H. et al. LLaMA: Open and Efficient Foundation Language Models. Preprint at http://arxiv.org/abs/2302.13971 (2023). ↗
  51. Grattafiori, A. et al. The Llama 3 Herd of Models. Preprint at https://doi.org/10.48550/arXiv.2407.21783 (2024). ↗
  52. Bai, J. et al. Qwen technical report. arXiv preprint arXiv:2309.16609 (2023). ↗
  53. DeepSeek-AI et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Preprint at https://doi.org/10.48550/arXiv.2501.12948 (2025). ↗
  54. Team, G. et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295 (2024). ↗
  55. Team, G. et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786 (2025). ↗
  56. Kojima, T., Gu, S. (Shane), Reid, M., Matsuo, Y. & Iwasawa, Y. Large Language Models are Zero-Shot Reasoners. in Advances in Neural Information Processing Systems (eds Koyejo, S. et al.) vol. 35 22199–22213 (Curran Associates, Inc., 2022).
  57. OpenAI. GPT-4 Technical Report. Preprint at http://arxiv.org/abs/2303.08774 (2023). ↗
  58. Zhao, W., Strube, M. & Eger, S. DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence. in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics 3865–3883 (Association for Computational Linguistics, Dubrovnik, Croatia, 2023). doi:10.18653/v1/2023.eacl-main.278. ↗
  59. Raji, I. D. et al. Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing. in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency 33–44 (ACM, Barcelona Spain, 2020). doi:10.1145/3351095.3372873. ↗
  60. Hendrycks, D., Carlini, N., Schulman, J. & Steinhardt, J. Unsolved Problems in ML Safety. Preprint at https://doi.org/10.48550/arXiv.2109.13916 (2022). ↗
  61. Bommarito, M. & Katz, D. M. GPT Takes the Bar Exam. Preprint at https://doi.org/10.48550/arXiv.2212.14402 (2022). ↗
  62. William G., C. Sampling Techniques. (Wiley, 1991). find
  63. Shrout, P. E. & Fleiss, J. L. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 86, 420–428 (1979). find
  64. Kendall, M. G. A New Measure of Rank Correlation. Biometrika 30, 81 (1938). find
  65. Cohen, J. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin 70, 213–220 (1968). find
  66. Efron, B. & Tibshirani, R. J. An Introduction to the Bootstrap. (Chapman and Hall/CRC, 1994). doi:10.1201/9780429246593. ↗
  67. Konietschke, F. & Pauly, M. Bootstrapping and permuting paired t-test type statistics. Stat Comput 24, 283–296 (2014). find
  68. Dietterich, T. G. Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Computation 10, 1895–1923 (1998). find
  69. Benjamini, Y. & Hochberg, Y. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society Series B: Statistical Methodology 57, 289–300 (1995). find
  70. Penedo, G. et al. The fineweb datasets: Decanting the web for the finest text data at scale. Advances in Neural Information Processing Systems 37, 30811–30849 (2024).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wind-2026-steuerllm-local-specialized-large-language/