Elamur Research⌕
☰
home / papers / 2025 / SR-LLM: Rethinking the Structured Representation in Large Language Model

SR-LLM: Rethinking the Structured Representation in Large Language Model

Paper · 2025 · Jiahuan Zhang, Tianheng Wang, Hanqing Wu et al.

Summary

Structured representations such as AMR, parse syntax trees (PST) and first-order logic (FOL) have long been central to NLP. Their usefulness for LLMs is unclear, because pasting them into prompts in raw, code-like form (as in the AMRCOT baseline of Jin et al., 2024) tends to lower accuracy. The authors argue that LLMs struggle with this notation simply because it is rare in their training data. They propose SR-LLM, which has two modes:

  • Training-free mode: a structure is rewritten as plain-language statements (SR-NLD) and added to the prompt. The conversion goes through rule-based triplet mapping, and then an LLM polishes the result.
  • Training-dependent mode: the model is fine-tuned on task data enriched with structures (Gen-SR).

The evaluation spans many tasks, including paraphrase detection, entailment, classification, NER, translation and text-to-SQL. On PAWS, the raw-AMR approach loses 5.18%, while SR-LLM gains 3.17% without training and 12.38% with fine-tuning. The authors present this as the first clear evidence that structured semantics can improve LLM reasoning.

Key points

  • AMR-to-NLD pipeline: the Penman library turns AMR graphs into triplets. Variable identifiers are then replaced by their concepts, and a predefined dictionary maps each triplet to a sentence. GPT-4o Mini refines the output, with voting over several generations to reduce hallucination.
  • Benchmarks: PAWS, Logic, Pubmed45, AGNEWS, WiC, SNLI, CoNLL2003, SST-2, WMT16 (BLEU) and SPIDER (EM). The method is compared across AMR, PST and FOL.
  • Combination with reasoning methods: on Llama3.1-8b-Instruct, SR-LLM adds gains on top of CoT, ToT and Self-reflection. It also beats earlier AMR-based methods.
  • Paraphrase control: adding 1–3 GPT-4o mini paraphrases to the prompt changes scores by only about −2% to +1%. This suggests the improvement comes from the structure itself, not from extra rephrased text.
  • Limitations: the gains are inconsistent across different LLMs, and the rule-based conversion limits flexibility.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TorontoToronto · Canada96 works · 142,007 citations🏢TencentShenzhen · China134 works · 19,743 citations🎓Westlake UniversityHangzhou · China45 works · 8,028 citations🎓Tianjin UniversityTianjin · China41 works · 1,128 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSemantic parsing & meaning representation · 185 works

Cite

Cite as Jiahuan Zhang, Tianheng Wang, Hanqing Wu, Ziyi Huang, Yulong Wu, Dongbai Chen, Linfeng Song, Yue Zhang, Guozheng Rao and Kaicheng Yu. 2025. SR-LLM: Rethinking the Structured Representation in Large Language Model. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.172

@inproceedings{zhang2025sr,
  title = {{SR-LLM: Rethinking the Structured Representation in Large Language Model}},
  author = {Jiahuan Zhang, Tianheng Wang, Hanqing Wu, Ziyi Huang, Yulong Wu, Dongbai Chen, Linfeng Song, Yue Zhang, Guozheng Rao and Kaicheng Yu},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.172},
  url = {https://doi.org/10.18653/v1/2025.acl-long.172}
}

References

47 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Alessandro De Bellis. 2023. Structuring the unstructured: an llm-guided transition. In DC@ ISWC. find
  3. Bill Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Third international workshop on paraphrasing (IWP2005).
  4. Kaikai An, Shuzheng Si, Helan Hu, Haozhe Zhao, Yuchi Wang, Qingyan Guo, and Baobao Chang. 2024. Rethinking semantic parsing for large language models: Enhancing llm performance with semantic hints. arXiv preprint arXiv:2409.14469. find
  5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  6. Dzmitry Bahdanau. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
  7. Herbert B Enderton. 2001. A mathematical introduction to logic. Elsevier. find
  8. Xuefeng Bai, Yulong Chen, and Yue Zhang. 2022. Graph pre-training for amr parsing and generation. arXiv preprint arXiv:2203.07836.
  9. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th linguistic annotation workshop and interoperability with discou…
  10. Jon Barwise. 1977. An introduction to first-order logic. In Studies in Logic and the Foundations of Mathematics, volume 90, pages 5–46. Elsevier. find
  11. Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, et al. 2016. Findings of the 2016 conference on machine translation (wmt16). In First conference on machine translation, pages 131–1…
  12. Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326.
  13. Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165.
  14. Noam Chomsky. 2014. Aspects of the Theory of Syntax. 11. MIT press. find
  15. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of machine learning research, 12:2493– 2537.
  16. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177–190. Springer. find
  17. Marco Damonte, Shay B Cohen, and Giorgio Satta. 2016. An incremental parser for abstract meaning representation. arXiv preprint arXiv:1608.06111.
  18. Sahil Garg, Aram Galstyan, Ulf Hermjakob, and Daniel Marcu. 2016. Extracting biomolecular interactions using semantic parsing of biomedical text. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30. find
  19. Christopher Hahn, Frederik Schmitt, Julia J Tillman, Niklas Metzger, Julian Siber, and Bernd Finkbeiner. 2022. Formal specifications from natural language. arXiv preprint arXiv:2206.01962. — [arXiv:2206.01962](https://arxiv.org/abs/2206.01962) find
  20. Zhijing Jin, Yuen Chen, Fernando Gonzalez, Jiarui Liu, Jiayi Zhang, Julian Michael, Bernhard Schölkopf, and Mona Diab. 2024. Analyzing the role of semantic representations in the era of large language models. arXiv preprint arXiv:2405.01502. find
  21. Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea, and Bernhard Schoelkopf. 2022. Logical fallacy detection. arXiv preprint arXiv:2202.13758.
  22. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational L…
  23. Aditya Kalyanpur, Kailash Saravanakumar, Victor Barres, Jennifer Chu-Carroll, David Melville, and David Ferrucci. 2024. Llm-arc: Enhancing llms with an automated reasoning critic. arXiv preprint arXiv:2406.17663. — [arXiv:2406.17663](https://arxiv.org/abs/2406.17663) find
  24. Kevin Knight, Bianca Badarau, Laura Baranescu, Claire Bonial, Madalina Bardocz, Kira Griffitt, Ulf Hermjakob, Daniel Marcu, Martha Palmer, Tim O’Gorman, et al. 2020. Abstract meaning representation (amr) annotation release 3.0. Web Download. LDC2020T02. find
  25. Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. 2024. Datasets for large language models: A comprehensive survey. arXiv preprint arXiv:2402.18041. find
  26. Christopher D Manning. 1999. Foundations of statistical natural language processing. The MIT Press. find
  27. Rada Mihalcea, Courtney Corley, Carlo Strapparava, et al. 2006. Corpus-based and knowledge-based measures of text semantic similarity. In Aaai, 2006, pages 775–780. find
  28. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32.
  29. Mohammad Taher Pilehvar and Jose Camacho-Collados. 2018. Wic: the word-in-context dataset for evaluating context-sensitive meaning representations. arXiv preprint arXiv:1808.09121.
  30. Chuan Wang, Nianwen Xue, and Sameer Pradhan. 2015. A transition-based algorithm for amr parsing. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 366–375.
  31. José Gabriel Carrasco Ramírez. 2024. Natural language processing advancements: Breaking barriers in human-computer interaction. Journal of Artificial Intelligence General Science (JAIGS) ISSN: 3006-4023, 3(1):31–39. find
  32. Tianming Wang, Xiaojun Wan, and Hanqi Jin. 2020. Amr-to-text generation with graph transformer. Transactions of the Association for Computational Linguistics, 8:19–33.
  33. Leonardo FR Ribeiro, Yue Zhang, and Iryna Gurevych. 2021. Structural adapters in pretrained language models for amr-to-text generation. arXiv preprint arXiv:2103.09120. find
  34. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.
  35. Subhro Roy, Samuel Thomson, Tongfei Chen, Richard Shin, Adam Pauls, Jason Eisner, and Benjamin Van Durme. 2024. Benchclamp: A benchmark for evaluating language models on syntactic and semantic parsing. Advances in Neural Information Processing Systems, 36. find
  36. Devendra Singh Sachan, Yuhao Zhang, Peng Qi, and William Hamilton. 2020. Do syntax trees help pre-trained transformers extract information? arXiv preprint arXiv:2008.09084. find
  37. Erik F Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050.
  38. Kaize Shi, Xueyao Sun, Qing Li, and Guandong Xu. 2024. Compressing long context for enhancing rag with amr-based concept distillation. arXiv preprint arXiv:2405.03085. — [arXiv:2405.03085](https://arxiv.org/abs/2405.03085) find
  39. Richard Shin, Christopher H Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, and Benjamin Van Durme. 2021. Constrained language models yield few-shot semantic parsers. arXiv preprint arXiv:2104.08768. find
  40. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631…
  41. Peiran Yao, Kostyantyn Guzhva, and Denilson Barbosa. 2024. Semantic graphs for syntactic simplification: A revisit from the age of llm. arXiv preprint arXiv:2407.04067. find
  42. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023a. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. find
  43. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023b. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).
  44. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887.
  45. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28.
  46. Yuan Zhang, Jason Baldridge, and Luheng He. 2019. Paws: Paraphrase adversaries from word scrambling. arXiv preprint arXiv:1904.01130.
  47. Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. A graph-to-sequence model for amr-to-text generation. arXiv preprint arXiv:1805.02473.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2025-sr-llm-rethinking-structured-representation/