Elamur Research⌕
☰
home / papers / 2025 / German Aspect-based Sentiment Analysis in the Wild: B2B Dataset Creation and Cross-Domain Evaluation

German Aspect-based Sentiment Analysis in the Wild: B2B Dataset Creation and Cross-Domain Evaluation

Paper · 2025 · Jakob Fehle, Niklas Donhauser, Udo Kruschwitz et al.

Summary

The paper tackles aspect-based sentiment analysis (ABSA) for German in a setting that research rarely covers: Business-to-Business (B2B) software feedback. Most ABSA work is in English and focuses on consumer domains such as restaurants and hotels. The authors built a new corpus from proprietary user comments, which were submitted voluntarily inside a B2B application. They adapted annotation guidelines from GERestaurant and SemEval 2016, annotated in several stages, and recorded what made the job hard, mainly implicit aspects and domain-specific phrasing. They then compared current ABSA approaches on three subtasks (ACD, ACSA, TASD) across five German datasets: BERT-based classifiers, T5 seq-to-seq methods, fine-tuned LLMs and few-shot prompting. Lightweight classifiers stay competitive for category detection, while fine-tuned LLMs lead on the structured tasks. However, scores shift considerably from one domain to another, driven by class imbalance, linguistic variability and annotation quality. The work matters because it shows that ABSA methods do not transfer evenly across domains, and that careful annotation and evaluation sensitive to domain are needed for enterprise applications.

Key points

  • The B2B corpus comes from 1,500 sampled reviews, split with SpaCy into 3,918 sentences. Each sentence is labelled with (aspect term, aspect category, polarity) triplets, and NULL marks an implicit aspect term.
  • The aspect categories draw on three sources: earlier work on software feedback, the ISO/IEC 25,010 quality standard, and the taxonomies of review platforms G2 Crowd and Capterra. Example categories include Technical Performance, Ease of Use, Interface Design and Functional Scope.
  • The benchmark covers the B2B data plus four existing German resources, including GERestaurant, MobASA and GermEval 2017. TASD turned out to be the hardest subtask, and GermEval and the B2B set were the weakest datasets across all methods.
  • On TASD, few-shot prompting scored lowest (F1 43.78 on GermEval, 42.34 on B2B Software Reviews). MvP beat Paraphrase only slightly, at the cost of much more training time and memory.
  • The B2B data is confidential. The authors instead publish their annotation guidelines and synthetic examples written to resemble the real reviews.

Interesting ideas

  • I1. When the real data cannot be shared, the authors publish invented sentences that copy its style and structure, so readers can still see what the task looks like.
  • I2. An implicit aspect is recorded as an explicit NULL slot rather than being dropped, and a triplet only counts if every element is present.
  • I3. The label inventory merges a formal standard, practical industry taxonomies and prior research, instead of being invented from scratch.
  • I4. Error analysis showed that fine-tuned LLMs, given imbalanced and noisy inputs, fall back to placeholder outputs such as "0" for the dominant class, a sign of shortcut learning.
  • I5. The authors weigh small accuracy gains against compute cost (MvP vs Paraphrase) and note that one dataset is easy simply because its vocabulary varies little.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Restricted or licensed evaluation sets could come with synthetic look-alike samples, so failures of witness parsers can be shown and discussed openly without leaking the source text.
  • I2 (frames): Roles that are understood but never stated in the text (null-instantiated) could be stored as explicit empty fillers in frames, keeping "unstated but implied" separate from "absent".
  • I3 (world): Domain ontology layers could be built the same way, aligning WordNet/FrameNet classes with external standards and real-world category lists, so the concepts match how people in that domain actually talk.
  • I4 (learn): LLM teacher outputs could be checked for template-like collapse on frequent classes before they enter silver data, using majority-class negative controls.
  • I5 (tree): Comparisons between gparse and the perceptron parser could report cost next to LAS, and split results by lexical variability, so that easy treebanks do not inflate the scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 972 works

Cite

Cite as Jakob Fehle, Niklas Donhauser, Udo Kruschwitz, Nils Constantin Hellwig and Christian Wolff. 2025. German Aspect-based Sentiment Analysis in the Wild: B2B Dataset Creation and Cross-Domain Evaluation. Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Long and Short Papers.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{fehle2025german,
  title = {{German Aspect-based Sentiment Analysis in the Wild: B2B Dataset Creation and Cross-Domain Evaluation}},
  author = {Jakob Fehle, Niklas Donhauser, Udo Kruschwitz, Nils Constantin Hellwig and Christian Wolff},
  booktitle = {Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Long and Short Papers},
  year = {2025},
  doi = {10.5283/epub.79450},
  url = {https://doi.org/10.5283/epub.79450}
}

References

59 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asma Musabah Alkalbani, Ahmed Mohamed Ghamry, Farookh Khadeer Hussain, and Omar Khadeer Hussain. 2016. Sentiment analysis and classification for software as a service reviews. In 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), pages 53–58. IEEE. find
  2. Omar Alonso and Ricardo Baeza-Yates, editors. 2024. Information Retrieval: Advanced Topics and Techniques, 1 edition, volume 60. Association for Computing Machinery, New York, NY, USA. find
  3. Omar Alqaryouti, Nur Siyam, Azza Abdel Monem, and Khaled Shaalan. 2020. Aspect-based sentiment analysis using smart government review data. Appl. Comput. Inform., ahead-of-print(ahead-of-print). ↗
  4. Nouf Alturaief, Hamoud Aljamaan, and Malak Basly-man. 2021. AWARE: Aspect-based sentiment analysis dataset of apps reviews for requirements elicitation. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW), pages 211–218. IEEE. find
  5. Muhammad Arslan and Christophe Cruz. 2024. Business-RAG: Information extraction for business insights. In Proceedings of the 21st International Conference on Smart Business Technologies, pages 88–94. SCITEPRESS - Science and Technology Publications. find
  6. Issa Atoum and Ahmed Otoom. 2016. Mining software quality from software reviews: Research trends and open issues. Int. j. comput. trends technol., 31(2):74– 83. ↗
  7. M. Aßenmacher, A. Corvonato, and C. Heumann. 2021. Re-Evaluating GermEval17 Using German Pre-Trained Language Models. arXiv preprint. ArXiv:2102.12330 [cs]. ↗
  8. Luca Beurer-Kellner, Marc Fischer, and Martin T Vechev. 2024. Guiding LLMs the right way: Fast, non-invasive constrained generation. ICML, abs/2403.06988. ↗
  9. Hongjie Cai, Yaofeng Tu, Xiangsheng Zhou, Jianfei Yu, and Rui Xia. 2020. Aspect-category based sentiment analysis with hierarchical graph convolutional network. In Proceedings of the 28th International Conference on Computational Linguistics, pages 833– 843, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  10. Branden Chan, Stefan Schweter, and Timo Möller. 2020. German’s next language model. Int Conf Comput Linguistics, pages 6788–6796.
  11. Siva Uday Sampreeth Chebolu, Franck Dernoncourt, Nedim Lipka, and Thamar Solorio. 2023a. A review of datasets for aspect-based sentiment analysis. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Stroudsburg, PA, USA. Association for Computational Linguistics. find
  12. Siva Uday Sampreeth Chebolu, Franck Dernoncourt, Nedim Lipka, and Thamar Solorio. 2023b. A review of datasets for aspect-based sentiment analysis. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 611–628. find
  13. Simon Clematide, Stefan Gindl, Manfred Klenner, Stefanos Petrakis, Robert Remus, Josef Ruppenhofer, Ulli Waltinger, and Michael Wiegand. 2012. MLSA - a multi-layered reference corpus for german sentiment analysis. find
  14. Jin Cui, Fumiyo Fukumoto, Xinfeng Wang, Yoshimi Suzuki, Jiyi Li, Noriko Tomuro, and Wanzeng Kong. 2024. Enhanced coherence-aware network with hierarchical disentanglement for aspect-category sentiment analysis. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), page 5843–5855. ELRA and ICCL. find
  15. Laleh Davoodi, József Mezei, and Markku Heikkilä. 2025. Aspect-based sentiment classification of user reviews to understand customer satisfaction of ecommerce platforms. Electron. Commer. Res. find
  16. Mengnan Du, Fengxiang He, Na Zou, Dacheng Tao, and Xia Hu. 2024. Shortcut learning of large language models in natural language understanding. Commun. ACM, 67(1):110–120. ↗
  17. Jakob Fehle, Leonie Münster, Thomas Schmidt, and Christian Wolff. 2023. Aspect-based sentiment analysis as a multi-label classification task on the domain of german hotel reviews. In Proceedings of the 19th Conference on Natural Language Processing (KONVENS 2023), pages 202–218. Association for Computational Lingustics. find
  18. Andy Field, Jeremy Miles, and Zoë Field. 2012. Discovering Statistics Using R. SAGE. find
  19. Aleksandra Gabryszak and Philippe Thomas. 2022. MobASA: Corpus for aspect-based sentiment analysis and social inclusion in the mobility domain. In Proceedings of the First Computing Social Responsibility Workshop within the 13th Language Resources and Evaluation Conference, pages 35–39, Marseille, France. European Language Resources Association. find
  20. Zhibin Gou, Qi Guo, and Yujiu Yang. 2023. MvP: Multi-view prompting improves aspect sentiment tuple prediction. Annual Meeting of the Association for Computational Linguistics, pages 4380–4397. find
  21. Lore De Greve, Pranaydeep Singh, Cynthia Van Hee, Els Lefever, and Gunther Martens. 2021. Aspect-based Sentiment Analysis for German: Analyzing “Talk of Literature” Surrounding Literary Prizes on Social Media. Computational Linguistics in the Netherlands Journal, 11:85–104. find
  22. International Organization for Standardization. 2023. ISO/IEC 25010:2023 – Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Product quality model. Available at: https://www.iso.org/ standard/78176.html. ↗
  23. Emitza Guzman, Rana Alkadhi, and Norbert Seyff. 2017. An exploratory study of twitter messages about software applications. Requir. Eng., 22(3):387–412. ↗
  24. Jan-Christoph Klie, Richard Eckart de Castilho, and Iryna Gurevych. 2024. Analyzing dataset annotation quality management in the wild. Comput. Linguist. Assoc. Comput. Linguist., pages 1–50. ↗
  25. Rajalaxmi Hegde and Seema S. 2017. Aspect based feature extraction and sentiment classification of review data sets using incremental machine learning algorithm. In 2017 Third International Conference on Advances in Electrical, Electronics, Information, Communication and Bio-Informatics (AEEICB), pages 122–125. IEEE. find
  26. Nils Constantin Hellwig, Jakob Fehle, Markus Bink, and Christian Wolff. 2024. GERestaurant: A german dataset of annotated restaurant reviews for aspect-based sentiment analysis. volume Proceedings of the 20th Conference on Natural Language Processing (KONVENS 2024), page 123–133. Association for Computational Linguistics. find
  27. Nils Constantin Hellwig, Jakob Fehle, Udo Kruschwitz, and Christian Wolff. 2025. Do we still need human annotators? prompting large language models for aspect sentiment quad prediction. In Proceedings of the 1st Joint Workshop on Large Language Models and Structure Modeling (XLLM 2025), pages 153–172, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  28. Sophie Henning, William Beluch, Alexander Fraser, and Annemarie Friedrich. 2023. A survey of methods for addressing class imbalance in deep-learning based natural language processing. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 523–540, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  29. Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):65–70. ↗
  30. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  31. Mengting Hu, Yike Wu, Hang Gao, Yinhao Bai, and Shiwan Zhao. 2022. Improving Aspect Sentiment Quad Prediction via Template-Order Data Augmentation. arXiv preprint. ArXiv:2210.10291 [cs]. ↗
  32. Yan Cathy Hua, Paul Denny, Jörg Wicker, and Katerina Taskova. 2024. A systematic review of aspect-based sentiment analysis: domains, methods, and trends. Artif. Intell. Rev., 57(11). ↗
  33. Roman Klinger. 2014. The USAGE review corpus for fine-grained, multi-lingual opinion analysis. find
  34. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, pages 611–626, New York, NY, USA. Association for Computing Machinery.
  35. Ji-Ung Lee, Steffen Eger, Johannes Daxenberger, and Iryna Gurevych. 2017. UKP TU-DA at GermEval 2017: Deep learning for aspect based sentiment detection. Proceedings of the GSCL GermEval Shared Task on Aspect-based Sentiment in Social Media Customer Feedback, pages 22–29. find
  36. Huayang Li, Yixuan Su, Deng Cai, Yan Wang, and Lemao Liu. 2022. A survey on retrieval-augmented text generation. arXiv [cs.CL]. ↗
  37. Alexander Ligthart, Cagatay Catal, and Bedir Tekinerdogan. 2021. Systematic reviews in sentiment analysis: a tertiary study. Artificial Intelligence Review, 54(7):4997–5053. ↗
  38. Bing Liu. 2022. Sentiment analysis and opinion mining. Springer Nature. find
  39. Pruthwik Mishra, Vandan Mujadia, and Soujanya Lanka. 2017. Germeval 2017: sequence based models for customer feedback analysis. Proceedings of the GermEval, pages 36–42. find
  40. Mariana Neves and Jurica Ševa. 2021. An extensive review of tools for manual annotation of documents. Briefings in Bioinformatics, 22(1):146–163. find
  41. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammed AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, Veronique Hoste, Marianna Apidianaki, Xavier Tannier, Natalia Loukachevitch, Evgeniy Kotelnikov, Núria Bel, Salud Maria Jiménez-Zafra, and Gülşen Eryiğit. 2016. SemEval-2016 task 5 : aspect based sentiment analysis. In ProWorkshop on Semantic Evaluation (SemEval-2016), pages 19–30. Association for Computational Linguistics. ↗
  42. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35, Dublin, Ireland. Association for Computational Linguistics.
  43. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1–67. ↗
  44. Doru Rotovei and Viorel Negru. 2020. Multi-agent recommendation and aspect level sentiment analysis in B2B CRM systems. In 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pages 238–245. IEEE. find
  45. Eugen Ruppert, Abhishek Kumar, and Chris Biemann. 2017. LT-ABSA: An extensible open-source system for document-level and aspect-based sentiment analysis. Proceedings of the GSCL GermEval Shared Task on Aspect-based Sentiment in Social Media Customer Feedback, pages 55–60. find
  46. Martin Schmitt, Simon Steinheber, Konrad Schreiber, and Benjamin Roth. 2018. Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks. find
  47. Samuel Sanford Shapiro and Martin B Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika, 52(3-4):591–611. ↗
  48. Paul F Simmering and Paavo Huoviala. 2023. Large language models for aspect-based sentiment analysis. arXiv [cs.CL]. ↗
  49. Mark Swillus and Andy Zaidman. 2023. Sentiment overflow in the testing stack: Analyzing software testing posts on stack overflow. J. Syst. Softw., 205(111804):111804. ↗
  50. Mario Sänger, Ulf Leser, Steffen Kemmerer, Peter Adolphs, and Roman Klinger. 2016. SCARE - the sentiment corpus of app reviews with fine-grained annotations in German. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1114–1121, Portorož, Slovenia. European Language Resources Association (ELRA). find
  51. Ruixiang Tang, Dehan Kong, Longtao Huang, and Hui Xue. 2023. Large language models can be lazy learners: Analyze shortcuts in in-context learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4645–4657, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  52. Zengzhi Wang, Qiming Xie, and Rui Xia. 2023. A Simple yet Effective Framework for Few-Shot Aspect-Based Sentiment Analysis. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, pages 1765–1770, New York, NY, USA. Association for Computing Machinery. find
  53. Mayur Wankhade, Annavarapu Chandra Sekhara Rao, and Chaitanya Kulkarni. 2022. A survey on sentiment analysis methods, applications, and challenges. Artificial intelligence review, 55(7):5731–5780. ↗
  54. Frank Wilcoxon. 1992. Individual comparisons by ranking methods. In Samuel Kotz and Norman L Johnson, editors, Breakthroughs in Statistics: Methodology and Distribution, pages 196–202. Springer New York, New York, NY. ↗
  55. Michael Wojatzki, Eugen Ruppert, Sarah Holschneider, Torsten Zesch, and Chris Biemann. 2017. Germeval 2017: Shared task on aspect-based sentiment in social media customer feedback. Proceedings of the GermEval, pages 1–12. ↗
  56. Chengyan Wu, Bolei Ma, Yihong Liu, Zheyu Zhang, Ningyuan Deng, Yanshu Li, Baolan Chen, Yi Zhang, Barbara Plank, and Yun Xue. 2025. M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis. arXiv preprint. ArXiv:2502.11824 [cs]. ↗
  57. Julia Wunderle, Jan Pfister, and Andreas Hotho. 2023. Pointer Networks: A Unified Approach to Extracting German Opinions. find
  58. Wenxuan Zhang, Yang Deng, Xin Li, Yifei Yuan, Lidong Bing, and Wai Lam. 2021. Aspect sentiment quad prediction as paraphrase generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9209– 9219, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  59. Jakub Šmíd, Pavel Priban, and Pavel Kral. 2024. LLaMA-based models for aspect-based sentiment analysis. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 63–70, Stroudsburg, PA, USA. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fehle-2025-german-aspect-based-sentiment-analysis/