Industrial Reinforcement Learning with Stabilizing Gradients

Existing automation solutions are often designed for large-scale production. Due to increasing customer demands for more and more individual products at competitive prices, new automation solutions are required that no longer follow the paradigm of mass production. These systems must be able to be put into operation with a minimum of time while having the ability to react to changing production conditions, caused for example by new products. An approach for this problem is reinforcement learning.

Reinforcement learning is the machine learning approach for learning control strategies from interaction with the environment. In industrial automation, reinforcement learning has the potential to increase process efficiency and to adapt processes on changing situations without human intervention. However, in industrial application, the reinforcement learning algorithm has to deal with uncertain processes, limited training data and high performance requirements. Current algorithms typically handle only a subset of these requirements. Therefore, this thesis proposes a novel approach combining methods from stabilizing gradients and variational inference with guided policy search. The so-called “industrial reinforcement learning with stabilizing gradients” is evaluated within the well-known FetchReach-v1 benchmark scenario and is exemplified on a vacuum bulk conveyer as real-world case study. In the FetchReach-v1 benchmark scenario, the proposed algorithm has reached a 50 % accuracy improvement in untrained situations. In the real-world case study, the algorithm outperformed prior approaches in terms of robustness to new products and data-efficiency. The results show that reinforcement learning is now applicable to industrial automation systems with an added-value.