关键词: Bipedal robot , · Reinforcement learning , · Sim-to-real , · Mechanics properties

Abstract: Reinforcement learning (RL) provides much potential for locomotion of legged robot. Due to the gap between simulation and the real world, achieving sim-to-real for legged robots is challenging. However, the support polygon of legged robots can help to overcome some of these challenges. Quadruped robot has a considerable support polygon, followed by bipedal robot with actuated feet, and point-footed bipedal robot has the smallest support polygon. Therefore, despite the existing sim-to-real gap, most of the recent RL approaches are deployed to the real quadruped robots that are inherently more stable, while the RL-based locomotion of bipedal robot is challenged by zero-shot sim-to-real task. Especially for the point-footed one that gets better dynamic performance, the inevitable tumble brings extra barriers to sim-to-real task. Actually, the crux of this type of problem is the difference of mechanics properties between the physical robot and the simulated one, making it difficult to play the learned skills well on the physical bipedal robot. In this paper, we introduce the embedded mechanics properties (EMP) based on the optimization with Gaussian processes to RL training, making it possible to perform sim-to-real transfer on the BRS1-P robot used in this work, hence the trained policy can be deployed on the BRS1-P without any struggle. We validate the performance of the learning-based BRS1-P on the condition of disturbances and terrains not ever learned, demonstrating the bipedal locomotion and resistant performance.

Key words: Bipedal robot , · Reinforcement learning , · Sim-to-real , · Mechanics properties