Hierarchical probabilistic model of language acquisition

Abstract

In this thesis, I propose an unsupervised computational model of language acquisition through visual grounding. I especially focus on a case where the language input is in a form of variable length sentences. The state-of-the-art cognitive architectures with the focus on grounding language in vision are explored. I take an advantage of probabilistic Bayesian models which are besides neural networks one of the main tools used in a computational cognitive modeling. The probabilistic (Bayesian) models have been used in the tasks such as language processing, decision making or causality learning. In the first part of the thesis newly proposed method for estimating a number of clusters in data is described. In the second part of the thesis I focus on the description of the cognitive architecture itself. The developed hierarchical cognitive architecture processes separately visual (static) and language (time-sequence) data and combines them in a multimodal layer. The important feature is a compositionality of the system - ability to derive meaning of previously unheard sentences and unseen objects and its ability to learn all features describing the object from sentences of variable length. The proposed architecture was implemented into the humanoid robot iCub and tested on both artificially generated data and on the realworld data.