Overview

Tobias is a quadrupedal robot project that integrates CAD design, physics simulation, and reinforcement learning to achieve stable walking gaits and straight-line locomotion. By combining RL techniques with detailed physical modeling, it became a platform for exploring autonomous robot movement and control.

Key features

• End-to-end integration pipeline from CAD design to physical deployment using Fusion 360 and PyBullet
• Comparison of DQN and SAC reinforcement learning algorithms for optimal gait learning
• Custom controller class for precise joint movement and physical constraint management
• Successful sim-to-real transfer through both closed-loop IMU-based control and action replay

Technical details

Learning system

• SAC with entropy-based exploration and automatic temperature tuning
• 28-dimensional state space capturing position, orientation, velocities, and joint angles
• Continuous 8-dimensional action space controlling joint angles (two per leg)
• Multi-component reward shaping for stability and efficiency

Architecture & implementation

• Dual actor-critic networks with 256-neuron hidden layers and ReLU activation
• Custom controller class handling joint constraints and gear ratios
• PyBullet physics engine for real-time simulation
• URDF robot model generated from a Fusion 360 CAD design

Results & impact

The SAC implementation significantly outperformed both DQN and traditional approaches. The action-replay method proved particularly effective for sim-to-real transfer, producing stable walking gaits in real-world conditions. The framework provides a foundation for future work in quadrupedal robotics and embodied reinforcement learning.