Optimal Intelligent Control of Robotic Manipulators with Optimal Reliance on Physics Information

ABSTRACT Physics-Informed Reinforcement Learning (PI-RL) is a promising research area, aiming at addressing the shortcomings of both traditional Deep Reinforcement Learning (DRL) and model-based control methods. This study presents an optimal control algorithm for the trajectory tracking of serial robotic manipulators. The proposed approach incorporates physics information directly into the policy of an online model-free DRL algorithm to control highly uncertain manipulators. To enable more accurate parameter identification, the robot parameters are learned through continual prediction and control of its motion. This algorithm utilizes a novel reparameterization for inertia tensors to enforce magnitude constraints, which further confines the solution space. A feedforward network is used to identify the robot’s physical parameters. Furthermore, the policy update rule is modified to ensure an optimal reliance on the physics informed component. Approximate ranges for certain trainable parameters are utilized to constrain them to physically plausible values. Finally, the proposed algorithm is evaluated on robotic arms simulated in the MuJoCo environment, with its tracking performance compared to the Independent Joint Control (IJC) and the Computed Torque Control (CTC) methods. The results show that this new approach outperforms these methods, while significantly enhancing DRL’s interpretability, stability, and sample efficiency, potentially making its application more viable for real-world robotic control problems.

[Download]