Overview

Tobias is a 3D-printed four-legged robot that walks with a policy trained by reinforcement learning. The policy learns in a physics simulator, and a gait that works there doesn’t automatically work on real servos on a real floor.

Hardware

A Raspberry Pi 4 drives eight LX-16A serial bus servos, one at each hip and knee, with an IMU for orientation. I designed the body in Fusion 360 and exported the same model to build the simulation’s URDF, so the simulated robot has the real one’s geometry. A Flask API on the Pi exposes stand, sit, walk and stop, and a React app on my phone calls it.

Why SAC instead of DQN?

Training ran in PyBullet with Stable-Baselines3 and PyTorch, with a reward for forward velocity and a penalty for falling. I tried both algorithms. DQN needs discrete actions, so the joint commands had to be cut into bins. Soft Actor-Critic works directly in the continuous 8-dimensional action space, one angle per joint. SAC outperformed DQN and learned smoother gaits, which is about what you’d expect from joints that don’t move in steps.

Getting it onto the robot

There are two routes to hardware. Replay saves the best training runs as joint angles per frame and plays them back on the servos. The live route runs the trained policy on the Pi, reading the IMU and sending joint commands straight to the servos.

Replay produced the stable gait in real conditions. The live policy was less consistent, though it showed the network could drive the hardware with no recorded path behind it.