Elamur Research⌕
☰
home / papers / 2026 / Are Large Language Models Economically Viable for Industry Deployment?

Are Large Language Models Economically Viable for Industry Deployment?

Paper · 2026 · Abdullah Mohammad, Sushant Kumar Ray, Pushkar Arora et al.

Summary

Industry teams choosing an LLM must weigh energy, latency, hardware use and cost, but common benchmarks report mostly accuracy. The authors call this missing operational view the Deployment–Evaluation Gap. To address it, they built EDGE-EVAL, a benchmarking framework that follows each model through adaptation, optional quantization and serving on older NVIDIA Tesla T4 GPUs. They test LLaMA and Qwen-2.5 models of three sizes on summarization (XSum), retrieval-augmented generation (SQuAD v1.1) and conversational agents (UltraChat). Each model is tuned with four LoRA/QLoRA precision setups and served through vLLM at batch size 1, giving 72 configurations. Measurements of energy, throughput, latency and VRAM feed five deployment metrics: N_break, IPW, ρ_sys, C_tax and Q_ret. Models under 2B parameters with INT4 weights beat the 7B–8B models on cost recovery, energy-normalized quality and throughput per GB, while losing almost no task quality. QLoRA, despite saving memory, can make fine-tuning of small models far more energy-hungry. The study suggests that compact models are often the more economical choice when hardware is limited.

Key points

  • Setup: Two 16GB Tesla T4 GPUs were used with LLaMA (3.2-1B, 3.2-3B, 3.1-8B) and Qwen2.5 (1.5B, 3B, 7B). Models were adapted with LoRA rank 16 and α = 32 under LoRA-FP16, LoRA-INT8, LoRA-INT4 and QLoRA-INT4, then served with vLLM v0.6.3. Power was logged with pynvml every 100 ms.
  • Cost recovery and density: LLaMA-3.2-1B (INT4) pays back its local adaptation cost after a median of 14 requests, versus 43 for the 7B tier. It reaches about 6,930 tokens/s/GB, compared with roughly 387 for 7B, and about three times the IPW of 7B models.
  • QLoRA anomaly: QLoRA cuts memory use by about 60%, yet it raises fine-tuning energy for small models by up to 7.2× compared with standard LoRA.
  • Quantization fidelity: INT4 retains more than 99% of quality in most configurations. The clearest exception is Qwen-3B on Chat, which drops 9.5% in GPT-4o-judged helpfulness.
  • Serving behavior:
    • Small INT4 models achieve inter-token latency under 10 ms and time-to-first-token under 50 ms.
    • LLaMA-3.2-1B draws a steady ~35W during inference.
    • Cold-start overheads of 179–237× make scale-to-zero serving uneconomical for low-traffic workloads.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States506 works · 330,052 citations🎓Macquarie UniversitySydney · Australia44 works · 5,778 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Abdullah Mohammad, Sushant Kumar Ray, Pushkar Arora, Rafiq Ali, Ebad Shabbir, Gautam Siddharth Kashyap, Jiechao Gao and Usman Naseem. 2026. Are Large Language Models Economically Viable for Industry Deployment?. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.106

@inproceedings{mohammad2026are,
  title = {{Are Large Language Models Economically Viable for Industry Deployment?}},
  author = {Abdullah Mohammad, Sushant Kumar Ray, Pushkar Arora, Rafiq Ali, Ebad Shabbir, Gautam Siddharth Kashyap, Jiechao Gao and Usman Naseem},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.106},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.106}
}

References

29 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Basem Almadani, Hunain Kaisar, Irfan Rashid Thoker, and Farouq Aliyu. 2025. A systematic survey of distributed decision support systems in healthcare. Systems, 13(3):157. find
  2. Colby Banbury, Vijay Janapa Reddi, Peter Torelli, Jeremy Holleman, Nat Jeffries, Csaba Kiraly, Pietro Montino, David Kanter, Sebastian Ahmed, Danilo Pau, and 1 others. 2021. Mlperf tiny benchmark. arXiv preprint arXiv:2106.07597. — [arXiv:2106.07597](https://arxiv.org/abs/2106.07597) find
  3. Christian Bauer, Samira Afzal, Sandro Linder, Radu Prodan, and Christian Timmerer. 2024. Greem: An open-source energy measurement tool for video processing. In Proceedings of the 15th ACM Multimedia Systems Conference, pages 264–270. find
  4. Bogdan-Iulian Ciubotaru. 2025. Generative ai and large language models: A comprehensive scientific review. find
  5. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088–10115.
  6. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. Enhancing chat language models by scaling high-quality instructional conversations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3029–3051.
  7. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323. find
  8. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  9. Rakibul Hasan and Samia Akter. 2022. Information system-based decision support tools: A systematic review of strategic applications in service-oriented enterprises. Review of Applied Science and Technology, 1(04):26–65. find
  10. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  11. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. True: Re-evaluating factual consistency evaluation. arXiv preprint arXiv:2204.04991.
  12. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  13. Zhe Jia, Marco Maggioni, Jeffrey Smith, and Daniele Paolo Scarpazza. 2019. Dissecting the nvidia turing t4 gpu via microbenchmarking. Preprint, arXiv:1903.07486. find
  14. Harsh Joshi, Gautam Siddharth Kashyap, Rafiq Ali, Ebad Shabbir, Niharika Jain, Sarthak Jain, Jiechao Gao, and Usman Naseem. 2025. Can argus judge them all? comparing vlms across domains. arXiv preprint arXiv:2507.01042. — [arXiv:2507.01042](https://arxiv.org/abs/2507.01042) find
  15. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pages 611–626.
  16. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  17. Vladislav Lialin, Vijeta Deshpande, and Anna Rumshisky. 2023. Scaling down to scale up: A guide to parameter-efficient fine-tuning. arXiv preprint arXiv:2303.15647.
  18. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  19. Vasile Denis Manolescu, Hamzah AlZu’bi, and Emanuele Lindo Secco. 2025. Interactive conversational ai with iot devices for enhanced human-robot interaction. Journal of Intelligent Communication. find
  20. Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745.
  21. C Nvidia. 2018. Nvidia turing gpu architecture. NVIDIA Whitepaper, 1. find
  22. David Patterson, Joseph Gonzalez, Urs Hölzle, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R So, Maud Texier, and Jeff Dean. 2022. The carbon footprint of machine learning training will plateau, then shrink. Computer, 55(7):18– 28. find
  23. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  24. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
  25. Nikolaos Schizas, Aristeidis Karras, Christos Karras, and Spyros Sioutas. 2022. Tinyml for ultra-low power ai and large scale iot deployments: A systematic review. Future Internet, 14(12):363. find
  26. Zohaib Hasan Siddiqui, Jiechao Gao, Ebad Shabbir, Mohammad Anas Azeez, Rafiq Ali, Gautam Siddharth Kashyap, and Usman Naseem. 2025. Llms on a budget? say hola. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1035–1043. find
  27. Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. 2022. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in neural information processing systems, 35:27168– 27183.
  28. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623.
  29. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of machine learning and systems, 6:87–100. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mohammad-2026-are-large-language-models-economically/