Elamur Research⌕
☰
home / papers / 2026 / Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems

Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems

Paper · 2026 · Xingchen Zou, Yuhao Yang, Zheng Chen et al.

Summary

Traffic signal control (TSC) systems based on fixed heuristics or reinforcement learning often fail when moved to new road networks. They also make decisions that are hard to inspect and struggle with unusual events such as accidents or emergency vehicles. Earlier LLM add-ons either describe an RL policy after the fact or are consulted only during incidents, which adds latency. The authors present Traffic-R1, a 3B-parameter controller built on Qwen2.5-3B and fine-tuned with two RL stages. In the offline stage, the model is rewarded for agreeing with decisions in human expert TSC recordings. In the online stage, it explores a traffic simulator and is rewarded by the simulated traffic results. Both stages also reward correct output format. The model generates its own Chain-of-Thought instead of imitating a larger teacher, which the authors argue limits catastrophic forgetting. An asynchronous message-passing network lets agents at neighbouring intersections coordinate. On benchmarks and incident tasks, Traffic-R1 is reported to match or beat RL controllers and larger LLMs such as GPT-4o without retraining. It is already in real-world use, showing that a compact, explainable signal controller can be deployed.

Key points

  • The model is released on Hugging Face as Season998/Traffic-R1. The authors say it is small enough to be quantized for low-power edge hardware, though the current deployment runs on a centralized server.
  • The ablation variants are (-) Expert, (-) Open-world and (-) Communicate. Results show that:
    • the expert-guided stage provides the base knowledge needed for stable exploration;
    • the simulator stage drives the top performance;
    • communication matters mainly for multi-intersection coordination.
  • The field deployment covers 10 linked intersections in an unnamed Chinese Tier-1 city, with over 55,000 vehicles per day. The setup works as follows:
    • Inputs come from fine-tuned Grounding DINO camera detection plus millimetre-wave radar.
    • A LightGBM model trained on 30 days of history forecasts next-day traffic flow.
    • Humans approve the resulting timing plans, and real-time control is limited to expert-supervised incident response.
  • A six-week A/B test against manual control (weekday peaks 16:00–19:30, systems alternating weekly) gave these results:
    • average queue fell from 34.5 to 31.3 (−9.3%);
    • maximum queue fell from 50.3 to 48.1 (−4.4%);
    • operator working hours dropped by about 75%.
    • The abstract cites more conservative figures: a queue reduction of over 5% and halved workload.
  • The authors list two limitations. The model needs observations converted into structured text, which can lose information. They also could not analyse hallucination-related instability at scale or collect driver feedback.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Hong KongPok Fu Lam · China89 works · 11,774 citations🎓The Hong Kong University of Science and Technology (Guangzhou)Guangzhou · China53 works · 1,367 citations

Also on the byline: PCI Biotech.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksEngineering · 133 worksControl and Systems Engineering · 45 worksBuilding and Construction · 5 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Xingchen Zou, Yuhao Yang, Zheng Chen, Xixuan Hao, Yiqi Chen, Chao Huang and Yuxuan Liang. 2026. Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.995

@inproceedings{zou2026traffic,
  title = {{Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems}},
  author = {Xingchen Zou, Yuhao Yang, Zheng Chen, Xixuan Hao, Yiqi Chen, Chao Huang and Yuxuan Liang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.995},
  url = {https://doi.org/10.18653/v1/2026.acl-long.995}
}

References

59 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baher Abdulhai, Rob Pringle, and Grigoris J Karak-oulas. 2003. Reinforcement learning for true adaptive traffic signal control. Journal of Transportation Engineering, 129(3):278–285. find
  2. Chirag Agarwal, Sree Harsha Tanneru, and Himabindu Lakkaraju. 2024. Faithfulness vs. plausibility: On the (un) reliability of explanations from large language models. arXiv preprint arXiv:2402.04614. — [arXiv:2402.04614](https://arxiv.org/abs/2402.04614) find
  3. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  4. Yuji Cao, Huan Zhao, Yuheng Cheng, Ting Shu, Yue Chen, Guolong Liu, Gaoqi Liang, Junhua Zhao, Jinyue Yan, and Yun Li. 2024. Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods. IEEE Transactions on Neural Networks and Learning Systems. find
  5. Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. 2020. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3414–342… find
  6. Rex Chen, Fei Fang, and Norman Sadeh. 2022. The real deal: A review of challenges and opportunities in moving reinforcement learning-based traffic signal control systems towards reality. arXiv preprint arXiv:2206.11996. — [arXiv:2206.11996](https://arxiv.org/abs/2206.11996) find
  7. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, and 1 others. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. find
  8. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088–10115.
  9. Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, and 1 others. 2025. Understanding world or predicting future? a comprehensive survey of world models. ACM Computing Surveys, 58(3):1–38. find
  10. Jie Feng, Jun Zhang, Tianhui Liu, Xin Zhang, Tianjian Ouyang, Junbo Yan, Yuwei Du, Siqi Guo, and Yong Li. 2025. Citybench: Evaluating the capabilities of large language models for urban tasks. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 5413–54… find
  11. Leo Gao, John Schulman, and Jacob Hilton. 2023. Scaling laws for reward model overoptimization. In International Conference on Machine Learning, pages 10835–10866. PMLR.
  12. Claire Glanois, Paul Weng, Matthieu Zimmer, Dong Li, Tianpei Yang, Jianye Hao, and Wulong Liu. 2024. A survey on interpretable reinforcement learning. Machine Learning, 113(8):5847–5890. find
  13. Jiani Huang, Xingchen Zou, Lianghao Xia, and Qing Li. 2025. Mr. rec: Synergizing memory and reasoning for personalized recommendation assistant with llms. arXiv preprint arXiv:2510.14629. — [arXiv:2510.14629](https://arxiv.org/abs/2510.14629) find
  14. Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, and 1 others. 2025. A survey of frontiers in llm reasoning: Inference scaling, learning to reason, and agentic systems. arXiv preprint arXiv:2504.09037. find
  15. Peter Koonce and 1 others. 2008. Traffic signal timing manual. Technical report, United States. Federal Highway Administration. find
  16. Siqi Lai, Zhao Xu, Weijia Zhang, Hao Liu, and Hui Xiong. 2023. Llmlight: Large language models as traffic signal control agents. arXiv preprint arXiv:2312.16044. — [arXiv:2312.16044](https://arxiv.org/abs/2312.16044) find
  17. Zhonghang Li, Lianghao Xia, Xubin Ren, Jiabin Tang, Tianyi Chen, Yong Xu, and Chao Huang. 2025. Urban computing in the era of large language models. ACM Transactions on Intelligent Systems and Technology. find
  18. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  19. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024a. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  20. Syed Shah Sultan Mohiuddin Qadri, Mahmut Ali Gökçe, and Erdinç Öner. 2020. State-of-art review of traffic signal control methods: challenges and opportunities. European transport research review, 12(1):55. find
  21. Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, and 1 others. 2024b. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pages 38–55. Springer.
  22. Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kianté Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi, and Yejin Choi. 2022. Is reinforcement learning (not) for natural language processing: Benchmarks, baselines, and building blocks for natural language policy optimization.…
  23. Vaishali Mahavar and Jayesh Juremalani. 2018. Literature review on traffic control systems used worldwide. Journal of Emerging Technologies and Innovative Research, 5(5):77–79. find
  24. Ben Malin, Tatiana Kalganova, and Nikolaos Boulgo-uris. 2025. A review of faithfulness metrics for hallucination assessment in large language models. IEEE Journal of Selected Topics in Signal Processing. find
  25. Pallavi A Mandhare, Vilas Kharat, and CY Patil. 2018. Intelligent road traffic control system for traffic congestion: a perspective. International Journal of Computer Sciences and Engineering, 6(07):2018. find
  26. Hao Mei, Xiaoliang Lei, Longchao Da, Bin Shi, and Hua Wei. 2024. Libsignal: An open library for traffic signal control. Machine Learning, 113(8):5235– 5271. find
  27. Pedro Mercader, Wasim Uwayid, and Jack Haddad. 2020. Max-pressure traffic controller based on travel times: An experimental analysis. Transportation Research Part C: Emerging Technologies, 110:275–290. find
  28. Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large language models: A survey. arXiv preprint arXiv:2402.06196.
  29. Arthur Müller, Vishal Rangras, Tobias Ferfers, Florian Hufen, Lukas Schreckenberg, Jürgen Jasperneite, Georg Schnittker, Michael Waldmann, Maxim Friesen, and Marco Wiering. 2021. Towards real-world deployment of reinforcement learning for traffic signal control. In 2021 20th IEEE International Confe… find
  30. Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, and Jorge Silva. 2020. Attendlight: Universal attention-based reinforcement learning model for traffic signal control. Advances in Neural Information Processing Systems, 33:4079–4090. find
  31. Faizan Rasheed, Kok-Lim Alvin Yau, Rafidah Md Noor, Celimuge Wu, and Yeh-Ching Low. 2020. Deep reinforcement learning for traffic signal control: A review. IEEE Access, 8:208016–208044. find
  32. Jingqing Ruan, Ziyue Li, Hua Wei, Haoyuan Jiang, Jiaming Lu, Xuantang Xiong, Hangyu Mao, and Rui Zhao. 2024. Coslight: Co-optimizing collaborator selection and decision-making to enhance traffic signal control. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,… find
  33. Paolo Serafini and Walter Ukovich. 1989. A mathematical model for the fixed-time traffic control problem. European Journal of Operational Research, 42(2):152–165. find
  34. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256.
  35. Dipti Srinivasan, Min Chee Choy, and Ruey Long Cheu. 2006. Neural networks for real-time traffic signal control. IEEE Transactions on intelligent transportation systems, 7(3):261–272. find
  36. ModelScope Team. 2024. EvalScope: Evaluation framework for large models. find
  37. Theresa Thunig, Robert Scheffler, Martin Strehler, and Kai Nagel. 2019. Optimization and simulation of fixed-time traffic signal control in real-world applications. Procedia Computer Science, 151:826–833. find
  38. Pravin Varaiya. 2013. Max pressure control of a network of signalized intersections. Transportation Research Part C: Emerging Technologies, 36:177–195. find
  39. George A Vouros. 2022. Explainable deep reinforcement learning: state of the art and challenges. ACM Computing Surveys, 55(5):1–39. find
  40. Maonan Wang, Yirong Chen, Aoyu Pang, Yuxin Cai, Chung Shue Chen, Yuheng Kan, and Man-On Pun. 2025a. Vlmlight: Traffic signal control via vision-language meta-control and dual-branch reasoning. arXiv preprint arXiv:2505.19486. — [arXiv:2505.19486](https://arxiv.org/abs/2505.19486) find
  41. Kok-Lim Alvin Yau, Junaid Qadir, Hooi Ling Khoo, Mee Hong Ling, and Peter Komisarczuk. 2017. A survey on reinforcement learning models and algorithms for traffic signal control. ACM Computing Surveys (CSUR), 50(3):1–38. find
  42. Maonan Wang, Aoyu Pang, Yuheng Kan, Man-On Pun, Chung Shue Chen, and Bo Huang. 2024. Llm-assisted light: Leveraging large language model capabilities for human-mimetic traffic signal control in complex urban environments. arXiv preprint arXiv:2403.08337. — [arXiv:2403.08337](https://arxiv.org/abs/2403.08337) find
  43. Bao-Lin Ye, Weimin Wu, Keyu Ruan, Lingxi Li, Tehuan Chen, Huimin Gao, and Yaobin Chen. 2019. A survey of model predictive control methods for traffic signal control. IEEE/CAA Journal of Automatica Sinica, 6(3):623–640. find
  44. Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. 2025b. Ragen: Understanding self-evolution in llm agents via mul… find
  45. Hua Wei, Nan Xu, Huichu Zhang, Guanjie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Li. 2019. Colight: Learning network-level cooperation for traffic signal control. In Proceedings of the 28th ACM international conference on information and knowledge management, pag… find
  46. Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. 2021. Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation. ACM SIGKDD explorations newsletter, 22(2):12–18. find
  47. Hua Wei, Guanjie Zheng, Huaxiu Yao, and Zhenhui Li. 2018. Intellilight: A reinforcement learning approach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2496–2505. find
  48. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.
  49. Zirui Yuan, Siqi Lai, and Hao Liu. 2025. Collmlight: Cooperative large language model agents for network-wide traffic signal control. arXiv preprint arXiv:2503.11739. — [arXiv:2503.11739](https://arxiv.org/abs/2503.11739) find
  50. Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. 2019. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The world wide web conference, pages 3620–3624. find
  51. Liang Zhang, Qiang Wu, Jun Shen, Linyuan Lü, Bo Du, and Jianqing Wu. 2022. Expression might be enough: Representing pressure and demand for reinforcement learning based traffic signal control. In International Conference on Machine Learning, pages 26645–26654. PMLR. find
  52. Siyao Zhang, Daocheng Fu, Wenzhe Liang, Zhao Zhang, Bin Yu, Pinlong Cai, and Baozhen Yao. 2024. Trafficgpt: Viewing, processing and interacting with traffic foundation models. Transport Policy, 150:95– 105. find
  53. Licheng Wen, Daocheng Fu, Xin Li, Xinyu Cai, Tao Ma, Pinlong Cai, Min Dou, Botian Shi, Liang He, and Yu Qiao. 2023. Dilu: A knowledge-driven approach to autonomous driving with large language models. arXiv preprint arXiv:2309.16292. — [arXiv:2309.16292](https://arxiv.org/abs/2309.16292) find
  54. Dongbin Zhao, Yujie Dai, and Zhen Zhang. 2011. Computational intelligence in urban traffic signal control: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4):485–494. find
  55. Qiang Wu, Liang Zhang, Jun Shen, Linyuan Lü, Bo Du, and Jianqing Wu. 2021. Efficient pressure: Improving efficiency for signalized intersections. arXiv preprint arXiv:2112.02336. — [arXiv:2112.02336](https://arxiv.org/abs/2112.02336) find
  56. Guoyang Zhao, Fulong Ma, Weiqing Qi, Chenguang Zhang, Yuxuan Liu, Ming Liu, and Jun Ma. 2024. Tsclip: Robust clip fine-tuning for worldwide cross-regional traffic sign recognition. arXiv preprint arXiv:2409.15077. — [arXiv:2409.15077](https://arxiv.org/abs/2409.15077) find
  57. Xiao-Kun Wu, Min Chen, Wanyi Li, Rui Wang, Limeng Lu, Jia Liu, Kai Hwang, Yixue Hao, Yanru Pan, Qingguo Meng, and 1 others. 2025. Llm fine-tuning: Concepts, opportunities, and challenges. Big Data and Cognitive Computing, 9(4):87. find
  58. X Zhou, M Liu, BL Zagar, E Yurtsever, and AC Knoll. 2023. Vision language models in autonomous driving and intelligent transportation systems. arxiv 2023. arXiv preprint arXiv:2310.14414. — [arXiv:2310.14414](https://arxiv.org/abs/2310.14414) find
  59. Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, and 1 others. 2025. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zou-2026-traffic-r1-reinforced-llms-bring/