This project aims to sensibly improve the safety and success rate for long-horizon tasks of autonomous robots leveraging GenAI “robotic brains” (e.g. LLMs, VLMs) in unstructured environments.

We plan to achieve this by combining several approaches including: limiting the risk of collisions with humans and objects by building on top of solid, pre-engineered robotic primitives for navigation and manipulation; using human supervision and guidance to validate and correct plans and actions at runtime; continuously learning from successful executions and specific context/environment by leveraging long-term memory (e.g., embodied Retrieval Augmented Generation, RAG); incorporating world models to predict the effects of planned actions on the environment; leveraging self-critique approaches.