A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models
A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models
自动化技术、自动化技术设备计算技术、计算机技术自动化基础理论
Michael C. Welle,Nils Ingelhag,Anastasia Varava,Danica Kragic,Jesper Munkeby,Jonne van Haastregt.A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models[EB/OL].(2024-03-25)[2025-09-24].https://arxiv.org/abs/2403.16730.点此复制
In this paper, we build upon two major recent developments in the field,
Diffusion Policies for visuomotor manipulation and large pre-trained multimodal
foundational models to obtain a robotic skill learning system. The system can
obtain new skills via the behavioral cloning approach of visuomotor diffusion
policies given teleoperated demonstrations. Foundational models are being used
to perform skill selection given the user's prompt in natural language. Before
executing a skill the foundational model performs a precondition check given an
observation of the workspace. We compare the performance of different
foundational models to this end as well as give a detailed experimental
evaluation of the skills taught by the user in simulation and the real world.
Finally, we showcase the combined system on a challenging food serving scenario
in the real world. Videos of all experimental executions, as well as the
process of teaching new skills in simulation and the real world, are available
on the project's website.
展开英文信息
评论