Background: Traditionally, robots perform desired tasks with the aid of end-users analytically
decomposing and manually programming. Robots controlled the program can only follow program
instructions to move, which bring great difficulty to non-professional users. So it is meaningful to
study new robot control paradigm for robots to accomplish tasks actively without the professional program.
Vision-based robot learning by demonstration (vision-based LbD) is an effective approach that a
robot can autonomously accomplish a task with help of the combination of the learning by demonstration
(LbD) and vision sensing technology. Vision-based LbD allows robots to learn skills through ‘seeing’
demonstrations of users with vision sensors. Vision-based LbD reduces the operation difficulty of
robots and provides an intuitive manner for human interact with robot, especially for those users who
have no professional program experience.
Objective: Providing the references for researchers who work in related fields by reviewing recent advances
of vision-based LbD.
Methods: This paper reviews the latest patents and current representative articles related to visionbased
LbD. The key methods of these references are introduced in the aspects of algorithms, innovations
Results: The researches related to vision-based LbD in the last 5 years are classified, the advantages of
different algorithms in these patents and articles are introduced and analyzed, the future developments
and potential problems in this field are discussed.
Conclusion: The main advantage of vision-based LbD is to allow users training robots for new tasks by
the demonstration under the vision sensor without programming control. So, vision-based LbD provides
an intuitive manner of robot learning by demonstration to solve the problem of human-robot interaction.
Further improvement is required in the following aspects: Algorithm innovation, multiple demonstrations,
many definitions of human action and so on. More patents on vision-based LbD should be invented.