We propose a highly robust, standing and walking controller for humanoids. Our RL method is free of clocks, reference trajectories, footstep planning or demonstrations, and can seamlessly transition between standing and walking.
Full video
Summary video
A necessary capability for humanoid robots is the ability to stand and walk while rejecting natural disturbances. Recent progress has been made using sim-to-real reinforcement learning (RL) to train such locomotion controllers, with approaches differing mainly in their reward functions. However, prior works lack a clear method to systematically test new reward functions and compare controller performance through repeatable experiments. This limits our understanding of the trade-offs between approaches and hinders progress.
To address this, we propose a low-cost, quantitative benchmarking method to evaluate and compare the real-world performance of standing and walking (SaW) controllers on metrics like command following, disturbance recovery, and energy efficiency. We also revisit reward function design and construct a minimally constraining reward function to train SaW controllers.
We experimentally verify that our benchmarking framework can identify areas for improvement, which can be systematically addressed to enhance the policies. We also compare our new controller to state-of-the-art controllers on the Digit humanoid robot. The results provide clear quantitative trade-offs among the controllers and suggest directions for future improvements to the reward functions and expansion of the benchmarks.
Transitioning between commands
Shaking the robot
Pulling the robot
Random pushes
Strong push backwards
Lateral disturbance rejection
@article{bart2024robustSaW,
author = {Bart van Marum, Aayam Shrestha, Helei Duan, Pranay Dugar, Jeremy Dao, Alan Fern},
title = {Revisiting Reward Design and Evaluation for Robust Human Standing and walking},
journal = {(Under Submission)},
year = {2024},
}