Supervised Human-Interactive Embodied Learning and Development (SHIELD)
This project aims to sensibly improve the safety and success rate for long-horizon tasks of autonomous robots leveraging GenAI “robotic brains” (e.g. LLMs, VLMs) in unstructured environments.
Description
This project aims to sensibly improve the safety and success rate for long-horizon tasks of autonomous robots leveraging GenAI “robotic brains” (e.g. LLMs, VLMs) in unstructured environments.
We plan to achieve this by combining several approaches including: limiting the risk of collisions with humans and objects by building on top of solid, pre-engineered robotic primitives for navigation and manipulation; using human supervision and guidance to validate and correct plans and actions at runtime; continuously learning from successful executions and specific context/environment by leveraging long-term memory (e.g., embodied Retrieval Augmented Generation, RAG); incorporating world models to predict the effects of planned actions on the environment; leveraging self-critique approaches.
Key Data
Projectlead
Project status
Start imminent, 06/2025
Institute/Centre
Institute of Computer Science (InIT)
Funding partner
Foundation
Project budget
49'770 CHF
Steinberggasse 13
8400 Winterthur