Journal of Bionic Engineering ›› 2020, Vol. 17 ›› Issue (6): 1126-1138.doi: 10.1007/s42235-020-0102-8
Distance-directed Target Searching for a Deep Visual Servo SMA Driven Soft Robot Using Reinforcement Learning
Wuji Liu1, Zhongliang Jing1*, Han Pan1, Lingfeng Qiao1, Henry Leung2, Wujun Chen3
- 1. School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China
2. Department of Electrical and Computer Engineering, University of Calgary, Calgary AB T2N 1N4, Canada
3. School of Naval Architecture, Ocean & Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
Distance-directed Target Searching for a Deep Visual Servo SMA Driven Soft Robot Using Reinforcement Learning
Wuji Liu1, Zhongliang Jing1*, Han Pan1, Lingfeng Qiao1, Henry Leung2, Wujun Chen3
- 1. School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China
2. Department of Electrical and Computer Engineering, University of Calgary, Calgary AB T2N 1N4, Canada
3. School of Naval Architecture, Ocean & Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
摘要： Performing complex tasks by soft robots in constrained environment remains an enormous challenge owing to the limitations of flexible mechanisms and control methods. In this paper, a novel biomimetic soft robot driven by Shape Memory Alloy (SMA) with light weight and multi-motion abilities is introduced. We adapt deep learning to perceive irregular targets in an unstructured environment. Aiming at the target searching task, an intelligent visual servo control algorithm based on Q-learning is proposed to generate
distance-directed end effector locomotion. In particular, a threshold reward system for the target searching task is proposed to enable a certain degree of tolerance for pointing errors. In addition, the angular velocity and working space of the end effector with load and without load based on the established coupling kinematic model are presented. Our framework enables the trained soft robot to take actions and perform target searching. Realistic experiments under different conditions demonstrate the convergence of the learning process and effectiveness of the proposed algorithm.