LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2212.04088

Summary

Language-driven embodied agents usually need many annotated instruction–trajectory pairs for every task, which makes them expensive to build and slow to extend. LLM-Planner addresses this by using a large language model (GPT-3) as the high-level planner in a hierarchical agent. Given an instruction and a few retrieved example pairs in the prompt, the LLM writes a sequence of subgoals such as navigating to or picking up an object. A low-level planner then turns each subgoal into primitive actions. When the agent stalls, taking too many steps or failing repeatedly, the LLM is prompted again with the subgoals already completed and the objects seen so far. It then produces a revised continuation that fits the actual environment. The planner is plugged into the perception and low-level modules of HLSM and tested on ALFRED. With only a tiny fraction of the paired data, it performs comparably to fully trained baselines, while other methods nearly fail in the same few-shot regime. This suggests a cheap route to versatile, sample-efficient agents.

Key points

• Data efficiency: The planner uses about 100 paired examples, under 0.5% of ALFRED's training data, and no parameter updates (in-context learning only).
• Prompting techniques: The method adds a kNN retriever for dynamic in-context example selection and logit biases. In the ablation, LOOCV HLP accuracy is 40.59 for the full model. It falls to 17.48 without the retriever, 38.10 without logit biases, and 13.43 without both.
• Comparison with SayCan: Unlike SayCan, which ranks a pre-enumerated list of admissible skills, LLM-Planner generates plans directly. This removes the need for prior environment knowledge and reduces the number of LLM calls.
• Hyperparameter sensitivity: Around 9 in-context examples works well. Gains from more training examples level off near 250.
• Case studies: Grounded re-planning helps locate objects, for example searching a cabinet for a missing cup. It also resolves ambiguous references, for example mapping "lamp" to the desklamp present in the scene.