🔗 Who cites «Orca: A Distributed Serving System for Transformer-Based Generative Models»

478 papers of our library cite it (of the 149,439 whose bibliographies we read) · 38 with a page here, the most cited first · ← back to «Orca: A Distributed Serving System for Transformer-Based Generative Models»

1. SGLang: Efficient Execution of Structured Language Model ProgramsConference on Neural Information Processing Systems (NeurIPS)Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, et al.2023 · 729 citations
2. HybridFlow: A Flexible and Efficient RLHF FrameworkarXiv:2409.19256Guangming Sheng, Chi Zhang, Zilingfeng Ye, et al.2024 · 624 citations
3. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model ServingOSDI 2024Yinmin Zhong, Shengyu Liu, Junda Chen, et al.2024 · 409 citations
4. Yi: Open Foundation Models by 01.AIarXiv:2403.04652Alex Young, Bei Chen, Chao Li, et al.2024 · 408 citations
5. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUarXiv:2303.06865Ying Sheng, Lianmin Zheng, Binhang Yuan, et al.2023 · 402 citations
6. Splitwise: Efficient generative LLM inference using phase splittingISCA 2024Pratyush Patel, Esha Choukse, Chaojie Zhang, et al.2023 · 379 citations
7. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-ServeOSDI 2024Amey Agrawal, Nitin Kedia, Ashish Panwar, et al.2024 · 369 citations
8. PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUACM SIGOPS Symposium on Operating Systems Principles (SOSP)Yixin Song, Zeyu Mi, Haotong Xie, et al.2023 · 272 citations
9. DynamoLLM: Designing LLM Inference Clusters for Performance and Energy EfficiencyProceedings - International Symposium on High-Performance Computer Architecture/ProceedingsJovan Stojkovic, Chaojie Zhang, Inigo Goiri, et al.2024 · 218 citations
10. Deja Vu: Contextual Sparsity for Efficient LLMs at Inference TimeInternational Conference on Machine Learning (ICML)Zichang Liu, Jue Wang, Tri Dao, et al.2023 · 185 citations
11. FlashInfer: Efficient and Customizable Attention Engine for LLM Inference ServingMLSys 2025Zihao Ye, Lequn Chen, Ruihang Lai, et al.2025 · 177 citations
12. AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning ServingOSDI 2023Zhuohan Li, Lianmin Zheng, Yinmin Zhong, et al.2023 · 170 citations
13. Mooncake: A KVCache-centric Disaggregated Architecture for LLM ServingFAST 2025Ruoyu Qin, Zheming Li, Weiran He, et al.2024 · 161 citations
14. Atom: Low-bit Quantization for Efficient and Accurate LLM ServingMLSys 2024Yilong Zhao, Chien-Yu Lin, Kan Zhu, et al.2023 · 156 citations
15. Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative DecodingAnnual Meeting of the Association for Computational Linguistics (ACL)Jun Zhang, Jue Wang, Huan Li, et al.2024 · 148 citations
16. Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative DecodingFindings of the Association for Computational LinguisticsHeming Xia, Zhe Yang, Qingxiu Dong, et al.2024 · 146 citations
17. SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked PrefillsarXiv 2023Amey Agrawal, Ashish Panwar, Jayashree Mohan, et al.2023 · 144 citations
18. Fast Distributed Inference Serving for Large Language ModelsarXiv 2023Bingyang Wu, Yinmin Zhong, Zili Zhang, et al.2023 · 123 citations
19. QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM ServingMLSys 2025Yujun Lin, Haotian Tang, Shang Yang, et al.2024 · 120 citations
20. Llumnix: Dynamic Scheduling for Large Language Model ServingOSDI 2024Biao Sun, Ziming Huang, Hanyu Zhao, et al.2024 · 119 citations
21. A Survey on Efficient Inference for Large Language ModelsarXiv:2404.14294Zixuan Zhou, Xuefei Ning, Ke Hong, et al.2024 · 113 citations
22. Efficient Large Language Models: A SurveyarXiv:2312.03863Zhongwei Wan, Xin Wang, Che Liu, et al.2023 · 110 citations
23. Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to SystemsACM Computing SurveysXupeng Miao, Gabriele Oliaro, Zhihao Zhang, et al.2023 · 110 citations
24. Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentionUSENIX ATC 2024Bin Gao, Zhuomin He, Puru Sharma, et al.2024 · 105 citations
25. ServerlessLLM: Low-Latency Serverless Inference for Large Language ModelsarXiv:2401.14351Yao Fu, Leyang Xue, Yeqi Huang, et al.2024 · 105 citations
26. Hybrid LLM: Cost-Efficient and Quality-Aware Query RoutingarXiv:2404.14618Dujian Ding, Ankur Mallick, Chi Wang, et al.2024 · 98 citations
27. LLM Inference Unveiled: Survey and Roofline Model InsightsarXiv 2024Zhihang Yuan, Yuzhang Shang, Yang Zhou, et al.2024 · 91 citations
28. Vidur: A Large-Scale Simulation Framework For LLM InferencearXiv:2405.05465Amey Agrawal, Nitin Kedia, Jayashree Mohan, et al.2024 · 90 citations
29. Punica: Multi-Tenant LoRA ServingMLSys 2024Lequn Chen, Zihao Ye, Yongji Wu, et al.2023 · 81 citations
30. S-LoRA: Serving Thousands of Concurrent LoRA AdaptersMLSys 2024Ying Sheng, Shiyi Cao, Dacheng Li, et al.2023 · 80 citations
31. DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-InferencearXiv 2024 (Microsoft DeepSpeed)Connor Holmes, Masahiro Tanaka, Michael Wyatt, et al.2024 · 78 citations
32. Skeleton-of-Thought: Prompting LLMs for Efficient Parallel GenerationarXiv:2307.15337Xuefei Ning, Zinan Lin, Zixuan Zhou, et al.2023 · 72 citations
33. From Decoding to Meta-Generation: Inference-time Algorithms for Large Language ModelsarXiv:2406.16838Sean Welleck, Amanda Bertsch, Matthew Finlayson, et al.2024 · 51 citations
34. Faster MoE LLM Inference for Extremely Large ModelsFindings of the Association for Computational LinguisticsHaoqi Yang, Luohe Shi, Qiwei Li, et al.2026 · 8 citations
35. MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context ReasoningAnnual Meeting of the Association for Computational Linguistics (ACL)Tao Zhang, Ziqian Zeng, Hao Peng, et al.2026 · 5 citations
36. SSSD: Simply-Scalable Speculative DecodingAnnual Meeting of the Association for Computational Linguistics (ACL)Michele Marzollo, Jiawei Zhuang, Niklas Roemer, et al.2026 · 3 citations
37. Rectified Sparse Attention for Efficient Long-Sequence GenerationFindings of the Association for Computational LinguisticsYutao Sun, Tianzhu Ye, Li Dong, et al.2026 · 1 citation
38. QUARTZ: Quantile-Aware Routing and Queueing for TTFT SLOs in LLM ServingFindings of the Association for Computational LinguisticsZhipeng Liu, Yifan Zheng, Fanqi Kong, et al.2026

440 more papers of our library cite it; they have no page here yet.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/cited-by/