← projects

LLMs · Scene graphs · PDDL In progress

LLM-guided common-sense planning

An LLM supplies the common sense; a symbolic planner keeps the plan correct.

Pipeline from an underspecified instruction and scene graph, through LLM goal inference and verification, to a symbolic plan

about

Instructions like “set the dining table” name an activity but never state the goal. Given the scene as a 3D scene graph, an LLM infers grounded PDDL goal predicates, which are checked against the planning domain and repaired through a feedback loop before a symbolic planner produces the plan.

The LLM supplies the common sense; the planner keeps the plan correct. Evaluation is planned in a simulated kitchen–dining domain with BEHAVIOR-1K tasks, followed by a real-robot demo.