Supervised Human-Interactive Embodied Learning and Development (SHIELD)

This project aims to sensibly improve the safety and success rate for long-horizon tasks of autonomous robots leveraging GenAI “robotic brains” (e.g. LLMs, VLMs) in unstructured environments.

Eckdaten

Kontakt

Beschreibung

This project aims to sensibly improve the safety and success rate for long-horizon tasks of autonomous robots leveraging GenAI “robotic brains” (e.g. LLMs, VLMs) in unstructured environments.

We plan to achieve this by combining several approaches including: limiting the risk of collisions with humans and objects by building on top of solid, pre-engineered robotic primitives for navigation and manipulation; using human supervision and guidance to validate and correct plans and actions at runtime; continuously learning from successful executions and specific context/environment by leveraging long-term memory (e.g., embodied Retrieval Augmented Generation, RAG); incorporating world models to predict the effects of planned actions on the environment; leveraging self-critique approaches.