Efektivní metody pro učení modelů a řízení v robotice

Abstract

Cílem této disertační práce je navrhnout řešení aktuálních problém ů v oblasti učení modelů z dat v robotice. Práce představuje několik variant a rozšíření symbolické regrese. Tato technika, založená na genetickém programování, je vhodná pro automatické vytváření kompaktních a přesných modelů v podobě analytických rovnic i z malých souborů dat. Jedním z problémů v robotice je velké množství dat, které jsou roboty během provozu shromažd’ovány, což vyžaduje výběr podmnožiny trénovacích vzorků. Tato práce představuje novou metodu výběru vzorku založenou na predikční chybě modelu a porovnává ji se čtyřmi alternativními metodami. Experimentální vyhodnocení na mobilním robotu ukazuje, že model naučený jen z několika desítek vzorků vybraných navrženou metodou může být využit pro úspěšné vykonání úlohy založené na řízení metodou posilovaného učení.

Constructing mathematical models of dynamic systems is central to many engineering and science disciplines. Models facilitate simulations, analysis of the system’s behavior, decision making, and design of automatic control algorithms. Even inherently model-free control tech niques such as reinforcement learning have been shown to benefit from the use of models. However, applying model learning methods to robotics is not straightforward. Obtaining in formative data for constructing dynamic models can be difficult, especially when the models are to be learned during task execution. Despite their increasing popularity, commonly used model learning methods such as deep neural networks come with drawbacks. They are data hungry and require a lot of computational power to learn a large number of parameters in their complex structure. Their black-box nature does not offer any insight into or interpretation of the model. Also, configuring these methods to achieve good results is often a difficult task. The objective of this thesis is to address the present challenges in data-driven model learn ing in robotics. Several variants and extensions of symbolic regression are introduced. This technique, based on genetic programming, is suitable to automatically build compact and ac curate models in the form of analytic equations even from small data sets. One of the chal lenges is posed by the large amount of data the robots collect during their operation, demand ing techniques to select a smaller subset of training samples. To that end, this thesis presents a novel sample-selection method based on model prediction error and compares it to four al ternative approaches. A real-world experimental evaluation on a mobile robot shows that a model learned from only a few tens of samples selected by the proposed method can be used to accomplish a motion control task within a reinforcement learning scheme. Standard data-driven model learning techniques in many cases yield models that violate the physical constraints of the robot. However, a partial theoretical or empirical model of the robot is often known. It is shown in this work how symbolic regression can be naturally ex tended to include the prior information into the model construction process. An experimental evaluation on two real-world robotic platforms demonstrates that symbolic regression is able to automatically build models that are both accurate and physically valid and compensate for theoretical or empirical model deficiencies. Efficient methods are needed not only to learn robot models but also to learn models of the robot’s environment. The thesis is concluded by presenting a novel method for reliable robot localization in dynamic environments. The proposed approach introduces an environ ment representation based on weighted local visual features and a change detection algorithm that updates the weights as the robot moves around the environment. The core idea of the method consists in using the weights to distinguish the useful information in stable regions of the scene from the unreliable information in the regions that are changing. An extensive eval uation and comparison to state-of-the-art alternatives show that using the proposed change detection algorithm improves the localization accuracy