Elamur Research⌕
☰
home / papers / 2023 / Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency

Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency

Paper · 2023 · Zhihan Liu, Hao Hu, Shenao Zhang et al.

Summary

LLM agents can reason well, but they still struggle to turn that reasoning into good actions with few environment interactions. They also lack theoretical guarantees for doing so. Liu et al. address this with RAFA ("reason for future, act for now"), which frames LLM reasoning as learning and planning in a Bayesian adaptive MDP whose unknown variable is the environment itself. The agent keeps a memory buffer of past states, actions, rewards and feedback. In the learning step, the LLM is prompted with this buffer to estimate transitions, rewards or values. In the planning step, it builds a multi-step trajectory that maximizes value. Only the first action is executed. The new feedback is added to memory, and the agent replans from the resulting state. Improvement comes entirely through in-context data rather than weight updates. The authors prove a √T regret bound, where T is the number of online interactions, and report that RAFA beats existing agent frameworks on several interactive benchmarks. This makes RAFA one of the first LLM-agent designs with provable sample efficiency.

Key points

  • Evaluation tasks: The authors test on Game of 24, ALFWorld, BlocksWorld, and a Tic-Tac-Toe benchmark they designed themselves. RAFA is reported to outperform prior prompting and agent frameworks on all of them.
  • Planning subroutine: Depending on the setting, it mimics value iteration, random shooting, or Monte-Carlo tree search. Replanning after each executed action works like model predictive control.
  • Role of the LLM: It is treated as a functional of the environment posterior, for example Bayesian model averaging. Its uncertainty therefore shrinks as more feedback enters the context.
  • Exploration variants: Two variants, one with an optimistic bonus and one with posterior sampling (approximable via bootstrap), remove the dependence on the concentrability coefficient κ.
  • Linear-kernel bound: For d-dimensional Bayesian linear kernel MDPs, the regret is roughly √(d³T)/(1−γ), up to log factors, with high probability.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China439 works · 94,783 citations🎓Chinese University of Hong KongHong Kong · China176 works · 17,536 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksDecision Sciences · 306 worksManagement Science and Operations Research · 145 works

Cite

Cite as Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu and Zhaoran Wang. 2023. Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency. arXiv:2309.17382.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.17382 · arXiv 2309.17382

@misc{liu2023reason,
  title = {{Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency}},
  author = {Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu and Zhaoran Wang},
  year = {2023},
  doi = {10.48550/arxiv.2309.17382},
  eprint = {2309.17382},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.17382}
}

References

105 references, 37 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbasi-Yadkori, Y., Pál, D. and Szepesvári, C. (2011). Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems. find
  2. Abbasi-Yadkori, Y. and Szepesvári, C. (2015). Bayesian optimal control of smoothly parameterized systems. In Uncertainty in Artificial Intelligence. find
  3. Abernethy, J., Agarwal, A., Marinov, T. V. and Warmuth, M. K. (2023). A mechanism for sample-efficient in-context learning for sparse retrieval tasks. arXiv preprint arXiv:2305.17040. — [arXiv:2305.17040](https://arxiv.org/abs/2305.17040) find
  4. Agarwal, A., Kakade, S., Krishnamurthy, A. and Sun, W. (2020). Flambe: Structural complexity and representation learning of low rank mdps. Advances in neural information processing systems, 33 20095–20107. find
  5. Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K. et al. (2022). Do as I can, not as I say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691.
  6. Akyürek, E., Schuurmans, D., Andreas, J., Ma, T. and Zhou, D. (2022). What learning algorithm is in-context learning? Investigations with linear models. arXiv preprint arXiv:2211.15661.
  7. Beck, J. (2008). Combinatorial games: Tic-Tac-Toe theory. find
  8. Betts, J. T. (1998). Survey of numerical methods for trajectory optimization. Journal of Guidance, Control, and Dynamics. find
  9. Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. The annals of statistics, 9 1196–1217. find
  10. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A. et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33 1877–1901.
  11. Cai, Q., Yang, Z., Jin, C. and Wang, Z. (2020). Provably efficient exploration in policy optimization. In International Conference on Machine Learning. find
  12. Cai, T., Wang, X., Ma, T., Chen, X. and Zhou, D. (2023). Large language models as tool makers. arXiv preprint arXiv:2305.17126.
  13. Chen, Y., He, J. and Gu, Q. (2022). On the sample complexity of learning infinite-horizon discounted linear kernel MDPs. In International Conference on Machine Learning. find
  14. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S. et al. (2022). PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  15. Chua, K., Calandra, R., McAllister, R. and Levine, S. (2018). Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems, 31. find
  16. Creswell, A. and Shanahan, M. (2022). Faithful reasoning using large language models. arXiv preprint arXiv:2208.14271.
  17. Creswell, A., Shanahan, M. and Higgins, I. (2022). Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712. find
  18. Davison, A. C. and Hinkley, D. V. (1997). Bootstrap methods and their application. 1, Cambridge university press. find
  19. Dong, K., Wang, Y., Chen, X. and Wang, L. (2019). Q-learning with UCB exploration is sample efficient for infinite-horizon MDP. arXiv preprint arXiv:1901.09311. — [arXiv:1901.09311](https://arxiv.org/abs/1901.09311) find
  20. Efron, B. (1982). The jackknife, the bootstrap and other resampling plans. SIAM. find
  21. Fan, J., Wang, Z., Xie, Y. and Yang, Z. (2020). A theoretical analysis of deep q-learning. In Learning for dynamics and control. PMLR. find
  22. Garg, S., Tsipras, D., Liang, P. S. and Valiant, G. (2022). What can transformers learn in-context? A case study of simple function classes. In Advances in Neural Information Processing Systems.
  23. Ghavamzadeh, M., Mannor, S., Pineau, J., Tamar, A. et al. (2015). Bayesian reinforcement learning: A survey. Foundations and Trends® in Machine Learning, 8 359–483. find
  24. Ghosh, M. (2021). Exponential tail bounds for chisquared random variables. Journal of Statistical Theory and Practice, 15 35. find
  25. Gruver, N., Finzi, M., Qiu, S. and Wilson, A. G. (2023). Large language models are zero-shot time series forecasters. arXiv preprint arXiv:2310.07820.
  26. Guo, H., Liu, Z., Zhang, Y. and Wang, Z. (2024). Can large language models play games? a case study of a self-play approach. arXiv preprint arXiv:2403.05632. — [arXiv:2403.05632](https://arxiv.org/abs/2403.05632) find
  27. Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H. and Davidson, J. (2019). Learning latent dynamics for planning from pixels. In International conference on machine learning. PMLR.
  28. Hao, B., Abbasi Yadkori, Y., Wen, Z. and Cheng, G. (2019). Bootstrapping upper confidence bound. Advances in neural information processing systems, 32. find
  29. Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z. and Hu, Z. (2023). Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992. find
  30. Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A., Welbl, J., Clark, A. et al. (2022). Training compute-optimal large language models. arXiv preprint arXiv:2203.15556.
  31. Huang, W., Abbeel, P., Pathak, D. and Mordatch, I. (2022a). Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning.
  32. Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y. et al. (2022b). Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608. find
  33. Janner, M., Fu, J., Zhang, M. and Levine, S. (2019). When to trust your model: Model-based policy optimization. Advances in neural information processing systems, 32. find
  34. Jiang, H. (2023). A latent space theory for emergent abilities in large language models. arXiv preprint arXiv:2304.09960. — [arXiv:2304.09960](https://arxiv.org/abs/2304.09960) find
  35. Jin, C., Yang, Z., Wang, Z. and Jordan, M. I. (2020). Provably efficient reinforcement learning with linear function approximation. In Conference on Learning Theory. PMLR. find
  36. Kim, G., Baldi, P. and McAleer, S. (2023). Language models can solve computer tasks. arXiv preprint arXiv:2303.17491.
  37. Kirsch, L., Harrison, J., Sohl-Dickstein, J. and Metz, L. (2022). General-purpose in-context learning by meta-learning transformers. arXiv preprint arXiv:2212.04458. — [arXiv:2212.04458](https://arxiv.org/abs/2212.04458) find
  38. Lazaric, A., Ghavamzadeh, M. and Munos, R. (2010). Analysis of a classification-based policy iteration algorithm. In ICML-27th International Conference on Machine Learning. Omnipress. find
  39. Lee, J. N., Xie, A., Pacchiano, A., Chandak, Y., Finn, C., Nachum, O. and Brunskill, E. (2023). Supervised pretraining can learn in-context reinforcement learning. arXiv preprint arXiv:2306.14892. — [arXiv:2306.14892](https://arxiv.org/abs/2306.14892) find
  40. Li, B. Z., Nye, M. and Andreas, J. (2022). Language modeling with latent situations. arXiv preprint arXiv:2212.10012. find
  41. Li, Y., Ildiz, M. E., Papailiopoulos, D. and Oymak, S. (2023). Transformers as algorithms: Generalization and implicit model selection in in-context learning. arXiv preprint arXiv:2301.07067. — [arXiv:2301.07067](https://arxiv.org/abs/2301.07067) find
  42. Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A. et al. (2022). Holistic evaluation of language models. arXiv preprint arXiv:2211.09110. find
  43. Liu, B., Jiang, Y., Zhang, X., Liu, Q., Zhang, S., Biswas, J. and Stone, P. (2023a). LLM+P: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477.
  44. Liu, Z., Lu, M., Wang, Z., Jordan, M. and Yang, Z. (2022a). Welfare maximization in competitive equilibrium: Reinforcement learning for markov exchange economy. In International Conference on Machine Learning. PMLR. find
  45. Liu, Z., Lu, M., Xiong, W., Zhong, H., Hu, H., Zhang, S., Zheng, S., Yang, Z. and Wang, Z. (2023b). Maximize to explore: One objective function fusing estimation, planning, and exploration. In Advances in Neural Information Processing Systems, vol. 36. find
  46. Liu, Z., Lu, M., Xiong, W., Zhong, H., Hu, H., Zhang, S., Zheng, S., Yang, Z. and Wang, Z. (2024). Maximize to explore: One objective function fusing estimation, planning, and exploration. Advances in Neural Information Processing Systems, 36. find
  47. Liu, Z., Zhang, Y., Fu, Z., Yang, Z. and Wang, Z. (2022b). Learning from demonstration: Provably efficient adversarial policy imitation with linear function approximation. In International conference on machine learning. PMLR. find
  48. Lu, P., Peng, B., Cheng, H., Galley, M., Chang, K.-W., Wu, Y. N., Zhu, S.-C. and Gao, J. (2023). Chameleon: Plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304.09842.
  49. Lu, X. and Van Roy, B. (2019). Information-theoretic confidence bounds for reinforcement learning. Advances in Neural Information Processing Systems. find
  50. Morari, M. and Lee, J. H. (1999). Model predictive control: past, present and future. Computers & chemical engineering, 23 667–682. find
  51. Newton, M. A. and Raftery, A. E. (1994). Approximate bayesian inference with the weighted likelihood bootstrap. Journal of the Royal Statistical Society Series B: Statistical Methodology, 56 3–26. find
  52. Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A. et al. (2022). In-context learning and induction heads. arXiv preprint arXiv:2209.11895.
  53. OpenAI (2023). GPT-4 technical report.
  54. Osband, I., Blundell, C., Pritzel, A. and Van Roy, B. (2016). Deep exploration via bootstrapped dqn. Advances in neural information processing systems, 29.
  55. Osband, I., Russo, D. and Van Roy, B. (2013). (More) efficient reinforcement learning via posterior sampling. In Advances in Neural Information Processing Systems. find
  56. Paul, D., Ismayilzada, M., Peyrard, M., Borges, B., Bosselut, A., West, R. and Faltings, B. (2023). REFINER: Reasoning feedback on intermediate representations. arXiv preprint arXiv:2304.01904.
  57. Pouplin, T., Sun, H., Holt, S. and Van der Schaar, M. (2024). Retrieval-augmented thought process as sequential decision making. arXiv preprint arXiv:2402.07812. — [arXiv:2402.07812](https://arxiv.org/abs/2402.07812) find
  58. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D. and Sutskever, I. (2019). Language models are unsupervised multitask learners. find
  59. Rawlings, J. B. (2000). Tutorial overview of model predictive control. IEEE Control Systems Magazine. find
  60. Razeghi, Y., Logan IV, R. L., Gardner, M. and Singh, S. (2022). Impact of pretraining term frequencies on few-shot reasoning. arXiv preprint arXiv:2202.07206. — [arXiv:2202.07206](https://arxiv.org/abs/2202.07206) find
  61. Russo, D. and Van Roy, B. (2014a). Learning to optimize via information-directed sampling. In Advances in Neural Information Processing Systems. find
  62. Russo, D. and Van Roy, B. (2014b). Learning to optimize via posterior sampling. Mathematics of Operations Research. find
  63. Russo, D. and Van Roy, B. (2016). An information-theoretic analysis of Thompson sampling. Journal of Machine Learning Research. find
  64. Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D. and Pathak, D. (2020). Planning to explore via self-supervised world models. In International Conference on Machine Learning. PMLR. find
  65. Sel, B., Al-Tawaha, A., Khattar, V., Wang, L., Jia, R. and Jin, M. (2023). Algorithm of thoughts: Enhancing exploration of ideas in large language models. arXiv preprint arXiv:2308.10379. find
  66. Shen, Y., Song, K., Tan, X., Li, D., Lu, W. and Zhuang, Y. (2023). HuggingGPT: Solving AI tasks with ChatGPT and its friends in HuggingFace. arXiv preprint arXiv:2303.17580.
  67. Shin, S., Lee, S.-W., Ahn, H., Kim, S., Kim, H., Kim, B., Cho, K., Lee, G., Park, W., Ha, J.-W. et al. (2022). On the effect of pretraining corpora on in-context learning by a large-scale language model. arXiv preprint arXiv:2204.13509. find
  68. Shinn, N., Cassano, F., Labash, B., Gopinath, A., Narasimhan, K. and Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366.
  69. Shridhar, M., Yuan, X., Côté, M.-A., Bisk, Y., Trischler, A. and Hausknecht, M. (2020). Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768.
  70. Singh, K. (1981). On the asymptotic accuracy of efron’s bootstrap. The Annals of Statistics 1187–1195. find
  71. Strens, M. (2000). A Bayesian framework for reinforcement learning. In International Conference on Machine Learning. find
  72. Sun, H. (2023). Reinforcement learning in the era of llms: What is essential? what is needed? an rl perspective on rlhf, prompting, and beyond. find
  73. Sun, H., Hüyük, A. and van der Schaar, M. (2023a). Query-dependent prompt evaluation and optimization with offline inverse rl. In The Twelfth International Conference on Learning Representations. find
  74. Sun, H., Zhuang, Y., Kong, L., Dai, B. and Zhang, C. (2023b). AdaPlanner: Adaptive planning from feedback with language models. arXiv preprint arXiv:2305.16653.
  75. Sutton, R. S. and Barto, A. G. (2018). Reinforcement learning: An introduction. find
  76. Todd, E., Li, M. L., Sharma, A. S., Mueller, A., Wallace, B. C. and Bau, D. (2023). Function vectors in large language models. arXiv preprint arXiv:2310.15213.
  77. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F. et al. (2023). LLaMa: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  78. Von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A. and Vladymyrov, M. (2023). Transformers learn in-context by gradient descent. In International Conference on Machine Learning.
  79. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A. and Zhou, D. (2022a). Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  80. Wang, X., Zhu, W. and Wang, W. Y. (2023a). Large language models are implicitly topic models: Explaining and finding good demonstrations for in-context learning. arXiv preprint arXiv:2301.11916.
  81. Wang, Z., Cai, S., Liu, A., Ma, X. and Liang, Y. (2023b). Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560.
  82. Wang, Z., Pan, T., Zhou, Q. and Wang, J. (2023c). Efficient exploration in resource-restricted reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37 10279–10287. find
  83. Wang, Z., Wang, J., Zhou, Q., Li, B. and Li, H. (2022b). Sample-efficient reinforcement learning via conservative model-based actor-critic. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36. find
  84. Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of mathematical psychology, 44 92–107. find
  85. Wei, C.-Y., Jahromi, M. J., Luo, H., Sharma, H. and Jain, R. (2020). Model-free reinforcement learning in infinite-horizon average-reward Markov decision processes. In International Conference on Machine Learning. find
  86. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V. and Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
  87. Wies, N., Levine, Y. and Shashua, A. (2023). The learnability of in-context learning. arXiv preprint arXiv:2303.07895.
  88. Wies, N., Levine, Y. and Shashua, A. (2024). The learnability of in-context learning. Advances in Neural Information Processing Systems, 36.
  89. Xie, S. M., Raghunathan, A., Liang, P. and Ma, T. (2021). An explanation of in-context learning as implicit Bayesian inference. arXiv preprint arXiv:2111.02080.
  90. Yang, L. and Wang, M. (2019). Sample-optimal parametric q-learning using linearly additive features. In International Conference on Machine Learning. find
  91. Yang, L. and Wang, M. (2020). Reinforcement learning in feature space: Matrix bandit, kernels, and regret bound. In International Conference on Machine Learning. find
  92. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y. and Narasimhan, K. (2023a). Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601. find
  93. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. and Cao, Y. (2022). ReAct: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.
  94. Yao, Y., Li, Z. and Zhao, H. (2023b). Beyond chain-of-thought, effective graph-of-thought reasoning in large language models. arXiv preprint arXiv:2305.16582.
  95. Zelikman, E., Wu, Y., Mu, J. and Goodman, N. (2022). STaR: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems.
  96. Zhang, S., Zheng, S., Ke, S., Liu, Z., Jin, W., Yuan, J., Yang, Y., Yang, H. and Wang, Z. (2024). How can llm guide rl? a value-based approach. arXiv preprint arXiv:2402.16181. — [arXiv:2402.16181](https://arxiv.org/abs/2402.16181) find
  97. Zhang, Y., Cai, Q., Yang, Z. and Wang, Z. (2020). Generative adversarial imitation learning with neural network parameterization: Global optimality and convergence rate. In International Conference on Machine Learning. PMLR. find
  98. Zhang, Y., Liu, B., Cai, Q., Wang, L. and Wang, Z. (2022). An analysis of attention via the lens of exchangeability and latent variable models. arXiv preprint arXiv:2212.14852. — [arXiv:2212.14852](https://arxiv.org/abs/2212.14852) find
  99. Zhang, Y., Yang, J., Yuan, Y. and Yao, A. C.-C. (2023a). Cumulative reasoning with large language models. arXiv preprint arXiv:2308.04371.
  100. Zhang, Y., Zhang, F., Yang, Z. and Wang, Z. (2023b). What and how does in-context learning learn? Bayesian model averaging, parameterization, and generalization. arXiv preprint arXiv:2305.19420. — [arXiv:2305.19420](https://arxiv.org/abs/2305.19420) find
  101. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. et al. (2023). Judging LLM-as-a-judge with MT-bench and chatbot arena. arXiv preprint arXiv:2306.05685.
  102. Zheng, S., Wang, L., Qiu, S., Fu, Z., Yang, Z., Szepesvari, C. and Wang, Z. (2022). Optimistic exploration with learned features provably solves markov decision processes with neural dynamics. In The Eleventh International Conference on Learning Representations. find
  103. Zhong, H., Xiong, W., Zheng, S., Wang, L., Wang, Z., Yang, Z. and Zhang, T. (2022). Gec: A unified framework for interactive decision making in mdp, pomdp, and beyond. arXiv preprint arXiv:2211.01962. — [arXiv:2211.01962](https://arxiv.org/abs/2211.01962) find
  104. Zhou, D., Gu, Q. and Szepesvári, C. (2021a). Nearly minimax optimal reinforcement learning for linear mixture Markov decision processes. In Conference on Learning Theory. find
  105. Zhou, D., He, J. and Gu, Q. (2021b). Provably efficient reinforcement learning for discounted MDPs with feature mapping. In International Conference on Machine Learning. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2023-reason-future-act-now-principled/