Elamur Research⌕
☰
home / papers / 2026 / Output-Space Search: Targeting LLM Generations in a Frozen Encoder-Defined Output Space

Output-Space Search: Targeting LLM Generations in a Frozen Encoder-Defined Output Space

Paper · 2026 · Tobias Materzok

Summary

Typical ways of exploring or optimizing LLM outputs, such as repeated sampling, prompt chaining or reranking, work on token paths. They add sequential dependence and growing context, and they give an outer optimizer no compact handle to tune. The paper proposes OS-Search, which treats generation as a search over endpoints. A frozen encoder, followed by a fixed PCA+Varimax projection, defines a 3D output space Z. A 1.7B controller is then trained with sequence-level, group-based RL to produce outputs whose coordinates fall close to a requested target z∗. Each request is grounded by retrieved exemplars located near that target, and the controller also self-reports the coordinates it believes it reached. An external loop can then sweep or optimize z∗ in parallel, while decoding stays ordinary and autoregressive. On stories, sweeping a small grid in an anchored Z text produced 3.1× higher LLM-judged diversity than prompt-chaining. On a synthetic code task, Bayesian optimization over z∗ improved an objective the controller never saw, while keeping programs valid. The result is a reusable, low-dimensional control interface that sits outside the model.

Key points

  • How Z is built: a frozen encoder plus a frozen linear projection (PCA subspace with Varimax rotation), with d_z = 3 in all experiments.
  • Training reward: a validity gate is combined with a distance term (how near z(x) lands to z∗) and an honesty term (how well the self-reported ẑ matches the real coordinates). Retrieval supplies two exemplars near z∗ and one contrast example near −z∗. Removing retrieval clearly hurts targeting and, for code, validity.
  • Stories: axis z₁ of Z text is anchored with "default-style" stories. Requested z₁∗ relates monotonically to EQ-Bench Slop-Score. The grid sweep reached 3.1× the LLM-scored diversity of prompt-chaining, with no human evaluation.
  • Code: an unanchored Z code was built on the synthetic CA++ benchmark (single-function programs generating 16×16 boards). With the number of valid programs held equal across comparisons, BO over z∗ raised an executable objective that was kept hidden from the controller.
  • Release and scope: code for Z construction and code-domain training is public on GitHub (TobiasMaterzok/OS-Search). Story-domain code and data are not released. The controller is trained with QLoRA and is not compared against full fine-tuning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Tobias Materzok. 2026. Output-Space Search: Targeting LLM Generations in a Frozen Encoder-Defined Output Space. Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.surgellm-1.4

@inproceedings{materzok2026output,
  title = {{Output-Space Search: Targeting LLM Generations in a Frozen Encoder-Defined Output Space}},
  author = {Tobias Materzok},
  booktitle = {Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.surgellm-1.4},
  url = {https://doi.org/10.18653/v1/2026.surgellm-1.4}
}

References

57 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical Neural Story Generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia. Association for Computational Linguistics.
  2. A. N. M. Nafiz Abeer, Nathan Urban, M. Ryan Weil, Francis J. Alexander, and Byung-Jun Yoon. 2024. Multi-Objective Latent Space Optimization of Generative Molecular Design Models. Patterns, 5(10):101042. ArXiv:2203.00526 [cs]. — [arXiv:2203.00526](https://arxiv.org/abs/2203.00526) find
  3. Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Francisco J. R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, David Silver, Demis Hassabis, and Pushmeet Kohli. 2022. Discovering faster matrix multiplication algorithms wi… find
  4. Amal Alabdulkarim, Winston Li, Lara J. Martin, and Mark O. Riedl. 2021. Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning. arXiv preprint. ArXiv:2112.08593 [cs]. — [arXiv:2112.08593](https://arxiv.org/abs/2112.08593) find
  5. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in Language Models Is Mediated by a Single Direction. arXiv preprint. ArXiv:2406.11717 [cs].
  6. Ziyu Wan, Xidong Feng, Muning Wen, Stephen Marcus Mcaleer, Ying Wen, Weinan Zhang, and Jun Wang. 2024. AlphaZero-Like Tree-Search can Guide Large Language Model Decoding and Training. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learni… find
  7. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Ass…
  8. Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. 2024. AlphaMath Almost Zero: Process Supervision without Process. arXiv preprint. ArXiv:2405.03553.
  9. Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025. Scaling and evaluating sparse autoencoders. In International Conference on Learning Representations. find
  10. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Lingyuan Zhang, Heng Gong, Weihong Zhong, and Bing Qin. 2023. Controllable Text Generation via Probability Woo… find
  11. Density Estimation in the Latent Space. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12590–12616, Toronto, Canada. Association for Computational Linguistics. find
  12. Rafael Gómez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik. 2018. Automatic Chemical Design Using a Data-Driven Continuous Representation of… find
  13. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The Power of Scale for Parameter-Efficient Prompt Tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguist…
  14. Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. 2023. Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents. arXiv prepri…
  15. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Proc…
  16. Daniel Han, Michael Han, and the Unsloth team. 2023. Unsloth. GitHub repository. Accessed: 2025-07-29. find
  17. Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… find
  18. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-Efficient Transfer Learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Ma… find
  19. Xun Liang, Hanyu Wang, Yezhaohui Wang, Shichao Song, Jiawei Yang, Simin Niu, Jie Hu, Dan Liu, Shunyu Yao, Feiyu Xiong, and Zhiyu Li. 2024. Controllable Text Generation for Large Language Models: A Survey. arXiv preprint. ArXiv:2408.12599 [cs].
  20. Patrik O. Hoyer. 2004. Non-negative Matrix Factorization with Sparseness Constraints. Journal of Machine Learning Research, 5(Nov):1457–1469. find
  21. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  22. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations.
  23. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025. Understanding R1-Zero-Like Training: A Critical Perspective. arXiv preprint. ArXiv:2503.20783 [cs].
  24. Henry F. Kaiser. 1958. The Varimax Criterion for Analytic Rotation in Factor Analysis. Psychometrika, 23(3):187–200. find
  25. Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL: A Conditional Transformer Language Model for Controllable Generation. arXiv preprint. ArXiv:1909.05858 [cs].
  26. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In International Conference on Learning Representations. find
  27. Tobias Materzok. 2025. COS(M+O)S: Curiosity and RL-Enhanced MCTS for Exploring Story Space via Language Models. arXiv preprint. ArXiv:2501.17104 [cs]. — [arXiv:2501.17104](https://arxiv.org/abs/2501.17104) find
  28. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations.
  29. Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. GeDi: Generative Discriminator Guided Sequence Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4929–4952, Punta Cana, Do…
  30. Kolby Nottingham, Ruo-Ping Dong, Ben Kasper, and Wesley N. Kerr. 2024. Improving Branching Language via Self-Reflection. In NeurIPS 2024 Workshop on Language Gamification (LanGame). find
  31. Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. find
  32. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. 2025. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint. ArXiv:2506.13131 [cs].
  33. David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukc… find
  34. OpenAI. 2026. GPT-5.1 Model \| OpenAI API. Accessed: 2026-01-14. find
  35. OpenAI. 2024. OpenAI o1 System Card. arXiv preprint. ArXiv:2412.16720 [cs.AI].
  36. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. 2017. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Alg…
  37. Samuel J. Paech. 2023. EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models. arXiv preprint. ArXiv:2312.06281 [cs].
  38. Samuel J. Paech. 2025. slop-score. GitHub repository. Accessed: 2025-11-22. find
  39. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and Permuted Pre-training for Language Understanding. In Advances in Neural Information Processing Systems 33.
  40. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  41. Nishant Subramani, Nivedita Suresh, and Matthew Peters. 2022. Extracting Latent Steering Vectors from Pretrained Language Models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 566–581, Dublin, Ireland. Association for Computational Linguistics.
  42. Kyeongman Park, Nakyeong Yang, and Kyomin Jung. 2025. Avoidance Decoding for Diverse Multi-Branch Story Generation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7489–7505, Suzhou, China. Association for Computational Linguistics. find
  43. Trieu H. Trinh and Quoc V. Le. 2019. A Simple Method for Commonsense Reasoning. arXiv preprint. ArXiv:1806.02847v2 [cs.AI]. find
  44. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. 2024. Steering Language Models With Activation Engineering. arXiv preprint. ArXiv:2308.10248 [cs].
  45. William H. Paulsen. 2003. Creating Large Life Forms with Interactive Life. Complex Systems, 14(3):275–283. find
  46. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  47. Zhihua Wen, Zhiliang Tian, Wei Wu, Yuxin Yang, Yanqi Shi, Zhen Huang, and Dongsheng Li. 2023. GROVE: A Retrieval-augmented Complex Story Generation Framework with A Forest of Evidence. arXiv preprint. ArXiv:2310.05388 [cs]. find
  48. Stephen Wolfram. 1984. Computation theory of cellular automata. Communications in Mathematical Physics, 96(1):15–57. find
  49. Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. 2024. Mathematical discoveries from program search with large language models…
  50. Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2025. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. In International Conference on Machine Learning (ICML). find
  51. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv preprint. ArXiv:2402.03300 [cs].
  52. Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P. Lillicrap, Kenji Kawaguchi, and Michael Shieh. 2024. Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning. arXiv preprint. ArXiv:2405.00451. find
  53. Ye Shi and C. S. George Lee. 2024. Uniform Transformation: Refining Latent Representation in Variational Autoencoders. arXiv preprint. ArXiv:2407.02681 [cs] version: 1. — [arXiv:2407.02681](https://arxiv.org/abs/2407.02681) find
  54. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng corpora tended to allocate axes to variations that are irrelevant or disallowed under our make_seed constraints. find
  55. Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu,… — Qwen3 Technical Report
  56. Encoders and pooling. Stories are embedded with a sentence-transformers encoder based on all-mpnet-base-v2 (Reimers and Gurevych, 2019; Song et al., 2020). Code is embedded with jina-embeddings-v2-base-code (Günther et al., 2023). For long inputs we use automatic chunking. We mean-pool chunk embeddi… find
  57. Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, Jed McCaleb, Xiaolong Wang, Jan Kautz, Yejin Choi, James Zou, Carlos Guestrin, and Yu Sun. 2026. Learning to Discover at Test Time. arXiv preprint. ArXiv:2601.16175 [cs]. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/materzok-2026-output-space-search-targeting-llm/