Novelty detection based on learning entropy

Abstract

The Approximate Individual Sample Learning Entropy is based on incremental learning of a predictor x(k+h)=f(x(k),w), where x(k)is an input vector of a given size at time k, w is a vector of weights (adaptive parameters), and his a prediction horizon. The basic assumption is that, after the underlying process x changes its behavior, the incrementally learning system will adapt the weights w to improve the predictor̃ x. Our goal is to detect a change in the behavior of the weight increment process. The main idea of this paper is based on the fact that weight increments△w(k), where△w(k)=w(k+1)−w(k), create a weakly stationary process until a change occurs. Once a novelty behavior of the under-lying process x(k)occurs, the process △w(k) changes its characteristics (eg, the mean or variation). We suggest using convenient characteristics of△w(k) in a multivariate detection scheme (eg, the Hotelling's T2 control chart).