Elamur Research⌕
☰
home / papers / 2026 / PseudoSeer: a Search Engine for Pseudocode

PseudoSeer: a Search Engine for Pseudocode

Paper · 2026 · Levent Toksoz, Mukund Srinath, Gang Tan et al.

Summary

Researchers have had no dedicated way to find pseudocode buried in the scientific literature. PseudoSeer fills this gap. It indexes roughly 320,000 algorithm blocks taken from the LaTeX sources of 2.2 million arXiv papers. Each block comes with metadata and "reference snippets," which are passages of surrounding text that cite the algorithm.

The authors also turn the collection into a benchmark. Algorithm captions (median five words) serve as queries against snippets of about 300 words, which are mostly prose with some LaTeX. Across embedding models from 22M to 1.5B parameters, larger size brings no advantage. The 149M GTE-ModernBERT beats 1.5B models, and BM25 holds its own against every pretrained encoder.

An attention study over 33,000 pairs explains this through two properties. The first is how much attention reaches content rather than sinks or uninformative tokens. The second is how sharply that attention focuses on discriminative terms. Contrastive fine-tuning of GTE-ModernBERT produces the system's own retriever, and fusing it with BM25 gives the best recall. The work delivers both a usable tool and practical guidance for choosing encoders for technical text.

Key points

  • Data and indexing: Blocks were found via \begin{algorithm} tags in arXiv papers from 1991 to June 2023, building on the Toksoz et al. (2024) dataset. They are indexed in Elasticsearch with BM25 and HNSW-based k-NN over 768-dimensional embeddings.
  • Pretrained models: GTE-ModernBERT (149M) outperforms 1.5B models such as Qwen2 and Stella. Snowflake (568M) sends 77% of its attention to natural language versus ModernBERT's 47%, yet recalls less because its attention is diffuse. Qwen2 is fairly concentrated (Gini 0.74) but loses much attention to sinks.
  • Fine-tuned retriever: Training used in-batch negatives for 10 epochs, with batch size 32 and learning rate 1e-5. The resulting model reaches 64.9% Recall@10, which is 8.7 points above the best pretrained model and 5.5 points above BM25.
  • Hybrid retrieval: Reciprocal Rank Fusion (k = 60) of BM25 and the learned embeddings achieves the top score of 66.5% R@10.
  • Latency and deployment: Queries take about 47 ms with BM25, about 106 ms with embeddings, and about 1 s with hybrid search on a 64-core AMD EPYC CPU. The system is live at pseudoseer.ist.psu.edu.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢AmazonSeattle · United States112 works · 40,565 citations🎓Pennsylvania State UniversityState College · United States56 works · 5,360 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 worksComputational Theory and Mathematics · 252 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Inverted index · 26 works

Cite

Cite as Levent Toksoz, Mukund Srinath, Gang Tan and C. Lee Giles. 2026. PseudoSeer: a Search Engine for Pseudocode. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1978

@inproceedings{toksoz2026pseudoseer,
  title = {{PseudoSeer: a Search Engine for Pseudocode}},
  author = {Levent Toksoz, Mukund Srinath, Gang Tan and C. Lee Giles},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1978},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1978}
}

References

29 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. 2023. Nougat: Neural optical understanding for academic documents. Preprint, arXiv:2308.13418. find
  2. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. BGE M3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Preprint, arXiv:2402.03216.
  3. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? An analysis of BERT’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276–286. Association for Computational Linguistics. find
  4. Yusuke Oda, Hiroyuki Fudaba, Graham Neubig, Hideaki Hata, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2015. Learning to generate pseudo-code from source code using statistical machine translation (t). 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 5… find
  5. Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. SPECTER: Document-level representation learning using citation-informed transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2270–2282, Online. Association for Comp…
  6. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  7. Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, page 758–759, New York,…
  8. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and permuted pre-training for language understanding. Preprint, arXiv:2004.09297.
  9. Bernard J Jansen, Amanda Spink, and Tefko Saracevic. 2000. Real life, real users, and real needs: A study and analysis of user queries on the web. Information Processing & Management, 36(2):207–227. find
  10. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. find
  11. Michael Kohlhase, Bogdan A. Matican, and Corneliu C. Prodescu. 2012. Mathwebsearch 0.5 – scaling an open formula search engine. In Intelligent Computer Mathematics, LNAI, pages 342–357. Springer. find
  12. Levent Toksoz, Gang Tan, and C. Lee Giles. 2024. Automatic pseudocode extraction at scale. In 2024 IEEE International Conference on Information Reuse and Integration for Data Science (IRI), pages 264–269. find
  13. Michael Kohlhase and Ioan Şucan. 2006. A search engine for mathematical formulae. In Proceedings of Artificial Intelligence and Symbolic Computation, AISC’2006, LNAI, pages 241–253. Springer. find
  14. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4365–437…
  15. Sumith Kulal, Panupong Pasupat, Kartik Chandra, Mina Lee, Oded Padon, Alex Aiken, and Percy Liang. 2019. Spoc: Search-based pseudocode to code. CoRR, abs/1906.04908. find
  16. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281.
  17. Patrice Lopez. 2008–2023. github.com/kermitt2/grobid. find
  18. Ioana Normann and Michael Kohlhase. 2007. Extended formula normalization for ϵ-retrieval and sharing of mathematical knowledge. In MKM/Calculemus - Towards Mechanized Mathematical Assistants, LNAI, pages 266–279. Springer. find
  19. Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5797–5808. Association…
  20. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533.
  21. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Preprint, arXiv:2002.10957. find
  22. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient streaming language models with attention sinks. In International Conference on Learning Representations (ICLR). find
  23. Haoyi Xiong, Jiang Bian, Yuchen Li, Xuhong Li, Mengnan Du, Shuaiqiang Wang, Dawei Yin, and Sumi Helal. 2024. When search engine services meet large language models: Visions and challenges. Preprint, arXiv:2407.00128. — [arXiv:2407.00128](https://arxiv.org/abs/2407.00128) find
  24. Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang, Jimmy Lin, and Vivek Srikumar. 2025. A survey of model architectures in information retrieval. Preprint, arXiv:2502.14822. — [arXiv:2502.14822](https://arxiv.org/abs/2502.14822) find
  25. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… find
  26. Puxuan Yu, Luke Merrick, Gaurav Nuti, and Daniel Campos. 2024. Arctic-embed 2.0: Multilingual retrieval without compromise. Preprint, arXiv:2412.04506. — [arXiv:2412.04506](https://arxiv.org/abs/2412.04506) find
  27. Maksym Zavershynskyi, Alex Skidanov, and Illia Polosukhin. 2018. Naps: Natural program synthesis dataset. Preprint, arXiv:1807.03168. find
  28. Dun Zhang, Jiacheng Li, Ziyang Zeng, and Fulong Wang. 2025. Jasper and stella: distillation of sota embedding models. Preprint, arXiv:2412.19048. — [arXiv:2412.19048](https://arxiv.org/abs/2412.19048) find
  29. Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, et al. 2024. mGTE: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024 Conference on Empirical Methods in…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/toksoz-2026-pseudoseer-search-engine-pseudocode/