Elamur Research⌕
☰
home / papers / 2026 / SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation

SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation

Paper · 2026 · Shuo Yang, Zheyu Zhang, Bardh Prenkaj et al.

Summary

LLM-based generators of synthetic tables usually encode each row as text and let every feature attend to every other one. This has two drawbacks. Dense attention can pick up correlations that are not real. It also treats dependencies as fixed, even though they often change with the actual values, for example how a loan's purpose affects plausible age and income.

SAGE (Sparse Adaptive Guidance) addresses both issues. It splits feature values into value-aware pseudo-features and builds a sparse dependency graph from pairwise mutual information. When filling in each feature, the graph steers generation in one of two ways:

  • Feature Selector: prunes the context explicitly.
  • Logit Correction: adjusts the model's output scores implicitly.

The authors also compute the loss only on value tokens and use rejection sampling to discard invalid outputs.

Across six datasets, SAGE improved fidelity, downstream utility and realism compared with earlier LLM-based generators. The results suggest that value-sensitive, sparse dependency modeling is a useful way to control LLM generation of structured data.

Key points

  • Builds on the GReaT "feature is value" serialization and the GRaDE strategy of shuffling feature order during training. Generation completes a row from a randomly chosen prefix of real feature-value pairs.
  • Classification F1 improved by up to about 10% (+10.3% over GReaT on Adult).
  • California Housing (geographic boundary violations):
    • Logit Correction lowered the violation rate by about 1 point compared with GReaT and came close to the GPT-4o-based SPADA.
    • Feature Selector cut violations by roughly 6% relative to TabSyn.
  • On Adult Income, Feature Selector reduced education-consistency violations to 1.32%. It also tended to give lower MAPE in regression tasks.
  • Stated limitations:
    • The method relies only on pairwise mutual information, so higher-order dependencies are not modeled directly.
    • The one-time mutual information computation can be costly for very high-dimensional data.
    • On HELOC, Logit Correction sometimes suppressed useful signals.
    • Code is available at github.com/ShuoYangtum/SAGE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technical University of MunichMunich · Germany62 works · 48,970 citations🎓Munich Center for Machine LearningMunich · Germany62 works · 1,167 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Shuo Yang, Zheyu Zhang, Bardh Prenkaj and Gjergji Kasneci. 2026. SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.174

@inproceedings{yang2026sage,
  title = {{SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation}},
  author = {Shuo Yang, Zheyu Zhang, Bardh Prenkaj and Gjergji Kasneci},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.174},
  url = {https://doi.org/10.18653/v1/2026.acl-long.174}
}

References

44 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Barry Becker and Ronny Kohavi. 1996. UCI Machine Learning Repository. https://doi.org/10.24432/C5XW20. — [doi:10.24432/c5xw20](https://doi.org/10.24432/c5xw20) ↗
  2. Adult. DOI: Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. 2022. Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems, pages 1–21. find
  3. Vadim Borisov, Kathrin Sessler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. 2023. Language models are realistic tabular data generators. In The Eleventh International Conference on Learning Representations. find
  4. Stephen P Boyd and Lieven Vandenberghe. 2004. Convex optimization. Cambridge university press. find
  5. Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake Vanderplas, Arnaud Joly, Brian Holt, and Gaël Varoquaux. 2013. API design for machine learning software: experie… find
  6. Nilka Rios Burrows. 2017. Incidence of end-stage renal disease attributed to diabetes among persons with diagnosed diabetes—united states and puerto rico, 2000–2014. MMWR. Morbidity and mortality weekly report, 66. find
  7. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning, 20:273–297.
  8. Thomas M. Cover and Joy A. Thomas. 2006. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA. find
  9. Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, and Philip S. Yu. 2025. Tabgen-icl: Residual-aware in-context example selection for tabular data generation. CoRR, abs/2502.16414. find
  10. Tennison Liu, Zhaozhi Qian, Jeroen Berrevoets, and Mihaela van der Schaar. 2023b. GOGGLE: Generative modelling for tabular data by learning relational structure. In The Eleventh International Conference on Learning Representations. find
  11. Ronald A Fisher. 1936. The use of multiple measurements in taxonomic problems. Annals of eugenics, 7(2):179–188. find
  12. Tongyu Liu, Ju Fan, Guoliang Li, Nan Tang, and Xiaoyong Du. 2024. Tabular data synthesis with generative adversarial networks: design space and optimizations. The VLDB Journal, 33(2):255–280. find
  13. S.E. Golovenkin, V.A. Shulman, D.A. Rossiev, P.A. Shesternya, S.Yu. Nikulina, Yu.V. Orlova, and V.F. Voino-Yasenetsky. 2020. Myocardial infarction complications. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C53P5M. — [doi:10.24432/c53p5m](https://doi.org/10.24432/c53p5m) ↗
  14. Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. 2016. Deep learning, volume 1. MIT press Cambridge. find
  15. Guangzeng Han, Weisi Liu, and Xiaolei Huang. 2025. Attributes as textual genes: Leveraging LLMs as genetic algorithm simulators for conditional synthetic data generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 19367–19389, Suzhou, China. Association for Compu… find
  16. Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. 2025. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319–326. find
  17. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations.
  18. Sanket Kamthe, Samuel A. Assefa, and Marc Peter Deisenroth. 2021. Copula flows for synthetic data generation. ArXiv, abs/2101.00598. — [arXiv:2101.00598](https://arxiv.org/abs/2101.00598) find
  19. Jayoung Kim, Jinsung Jeon, Jaehoon Lee, Jihyeon Hyeong, and Noseong Park. 2021. Oct-gan: Neural ode-based conditional tabular gans. In Proceedings of the Web Conference 2021, pages 1506–1515. find
  20. Jinhee Kim, Taesung Kim, and Jaegul Choo. 2024. Epic: Effective prompting for imbalanced-class data synthesis in tabular data classification via large language models. In Advances in Neural Information Processing Systems, volume 37, pages 31504–31542. Curran Associates, Inc. find
  21. Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. 2023. TabDDPM: Modelling tabular data with diffusion models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 17564–17579. PMLR. find
  22. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023a. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Comput. Surv., 55(9).
  23. Yunbo Long, Liming Xu, and Alexandra Brintrup. 2025. Llm-tabflow: Synthetic tabular data generation with inter-column logical relationship preservation. arXiv preprint arXiv:2503.02161. — [arXiv:2503.02161](https://arxiv.org/abs/2503.02161) find
  24. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. Preprint, arXiv:1711.05101. find
  25. Dang Nguyen, Sunil Gupta, Kien Do, Thin Nguyen, and Svetha Venkatesh. 2024. Generating realistic tabular data with large language models. CoRR, abs/2410.21717. — [arXiv:2410.21717](https://arxiv.org/abs/2410.21717) find
  26. Cameron Nugent. 2018. California housing prices. https://www.kaggle.com/datasets/ camnugent/california-housing-prices. ↗
  27. Averkiy Oliabev. 2022. Home equity line of credit (heloc) dataset. https://www.kaggle.com/datasets/averkiyoliabev/ home-equity-line-of-creditheloc. ↗
  28. Neha Patki, Roy Wedge, and Kalyan Veeramachaneni. 2016. The synthetic data vault. In IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 399–410. find
  29. Yubin Qu, Fang Li, Long Li, Xianzhen Dou, and Hongmei Wang. 2022. Can we predict student performance based on tabular and textual data? IEEE Access, 10:86008–86019. find
  30. Timur Sattarov, Marco Schreyer, and Damian Borth. 2023. Findiff: Diffusion models for financial tabular data generation. In Proceedings of the Fourth ACM International Conference on AI in Finance, ICAIF ’23, page 64–72, New York, NY, USA. Association for Computing Machinery. find
  31. Nabeel Seedat, Nicolas Huynh, Boris van Breugel, and Mihaela van der Schaar. 2024. Curated LLM: Synergy of LLMs and data curation for tabular augmentation in ultra low-data regimes. find
  32. Ruxue Shi, Yili Wang, Mengnan Du, Xu Shen, and Xin Wang. 2025. A comprehensive survey of synthetic tabular data generation. CoRR, abs/2504.16506. — [arXiv:2504.16506](https://arxiv.org/abs/2504.16506) find
  33. Mihaela C. Stoian and Eleonora Giunchiglia. 2025. Beyond the convexity assumption: Realistic tabular data generation under quantifier-free real linear constraints. In The Thirteenth International Conference on Learning Representations. find
  34. Mihaela CÄ Stoian, Eleonora Giunchiglia, and Thomas Lukasiewicz. 2025. A survey on tabular data generation: Utility, alignment, fidelity, privacy, and beyond. arXiv preprint arXiv:2503.05954. — [arXiv:2503.05954](https://arxiv.org/abs/2503.05954) find
  35. Namjoon Suh, Xiaofeng Lin, Din-Yin Hsieh, Merhdad Honarkhah, and Guang Cheng. 2023. Autodiff: combining auto-encoder and diffusion model for tabular data synthesizing. CoRR, abs/2310.15479. — [arXiv:2310.15479](https://arxiv.org/abs/2310.15479) find
  36. Vibeke Binz Vallevik, Aleksandar Babic, Serena E. Marshall, Severin Elvatun, Helga M.B. Brøgger, Sharmini Alagaratnam, Bjørn Edwin, Narasimha R. Veeraragavan, Anne Kjersti Befring, and Jan F. Nygård. 2024. Can i trust my fake data – a comprehensive quality assessment framework for synthetic tabular… find
  37. Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. 2019. Modeling tabular data using conditional gan. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. find
  38. Shengzhe Xu, Cho-Ting Lee, Mandar Sharma, Raquib Bin Yousuf, Nikhil Muralidhar, and Naren Ramakrishnan. 2024. Are llms naturally good at synthetic tabular data generation? CoRR, abs/2406.14541. — [arXiv:2406.14541](https://arxiv.org/abs/2406.14541) find
  39. Shengzhe Xu, Cho-Ting Lee, Mandar Sharma, Raquib Bin Yousuf, Nikhil Muralidhar, and Naren Ramakrishnan. 2025. Why llms are bad at synthetic table generation (and what to do about it). Preprint, arXiv:2406.14541. — [arXiv:2406.14541](https://arxiv.org/abs/2406.14541) find
  40. Zheyu Zhang, Han Yang, Bolei Ma, David Rügamer, and Ercong Nie. 2023. Baby’s CoThought: Leveraging large language models for enhanced reasoning in compact models. In Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning, pages 158–170, Singapore. Assoc… find
  41. Zheyu Zhang, Shuo Yang, Bardh Prenkaj, and Gjergji Kasneci. 2025. Not all features deserve attention: Graph-guided dependency learning for tabular data generation with language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 6217–6242, Suzhou, China. Associati… find
  42. Zilong Zhao, Robert Birke, and Lydia Y Chen. 2025. Tabula: Harnessing language models for tabular data synthesis. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 247–259. Springer. find
  43. Zilong Zhao, Aditya Kunar, Robert Birke, and Lydia Y. Chen. 2021. Ctab-gan: Effective table data synthesizing. In Proceedings of The 13th Asian Conference on Machine Learning, volume 157 of Proceedings of Machine Learning Research, pages 97–112. PMLR. find
  44. Zilong Zhao, Aditya Kunar, Robert Birke, Hiek Van der Scheer, and Lydia Y. Chen. 2024. CTAB-GAN+: enhancing tabular data synthesis. Frontiers Big Data, 6. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-sage-sparse-adaptive-guidance-dependency/