Discriminative learning from partially annotated examples

Abstract

A number of algorithms and its applications for automatic classi ers learning from examples is ever growing. Most of existing algorithms require a training set of completely annotated examples, which are often hard to obtain. In this thesis, we tackle the problem of learning from partially annotated examples, which means that each training input comes with a set of admissible labels only one of which is correct. We contributed to two di erent cases of this scenario. In the rst case, we studied the problem of learning the ordinal classi ers from examples with interval annotation of labels. We designed a convex learning algorithm for this case and demonstrated its advantage on real data empirically. At the same time, we made several contributions to the supervised learning of the ordinal classi ers, namely, we proposed new parametrization of the ordinal classi er, we introduced more exible piece wise version of the ordinal classi er, and we proposed a generic cutting plane solver with convergence guarantees. In the second case, we studied the problem of learning the structured output classi ers from examples with missing annotation of a subset of labels. We have de ned the concept of a surrogate classi cation calibrated partial loss, the minimization of which guarantees that learning is statistical consistent under fairly general conditions on the data generating process. We proved the existence of a convex classi cation calibrated surrogate loss for learning from partially annotated examples. We showed which existing surrogate losses are classi cation calibrated and which are not. Our work thus provides a missing theoretical justi cation for so far heuristic methods which have been successfully used in practice.