%\chapter{Introduction}
\chapter*{Introduction}
\addcontentsline{toc}{chapter}{Introduction}

\subsubsection{Motivation and Related Work}

We consider the problem of minimizing a convex piecewise-affine function, given either as the point-wise maximum of affine function (which we abbreviate as MAF) or as the sum of point-wise maxima of affine functions (SMAF). This problem can be formulated as a linear program (LP), therefore it can be solved in polynomial time. However, for very large sparse instances, on which we focus, solving this LP is in practice impossible because the space and time complexity of general LP solvers is highly superlinear and thus prohibitive. It does not help that the problem is sparse because these methods do not maintain sparsity during their run, therefore large enough instances simply do not fit in the computer memory. Alternatively, minimizing MAF or SMAF can be seen as an instance of convex non-differentiable minimization and thus one could apply subgradient methods which have linear space complexity -- however, these methods have been experimentally observed to be very slow.

A special example of sparse large-scale convex piecewise-affine minimization is the minimization of an upper bound on the max-sum labelling problem. In the binary form of this problem, we aim to minimize a sum of unary and binary functions of discrete variables. One powerful approach to this NP-hard combinatorial optimization problem is LP relaxation. The dual of this LP relaxation leads to minimizing an upper bound on the true optimum by equivalent transformations of the problem, which in fact means minimizing a special form of MAF or SMAF.

A number of algorithms to minimize the above mentioned upper bound have been proposed. These algorithms could be divided into two groups -- exact (resp. global) algorithms and approximate (resp. local) algorithms. The exact algorithms always find the true optimum. Those are for example the LP solving methods, subgradient methods or smoothing methods. The approximate algorithms do not find a global minimum of the upper bound, but only a local minimum, where "locality" is not meant as usually with respect to the Euclidean metric, but has a different meaning. Max-sum diffusion, which is described in \cite{Kovalevsky-diffusion} and listed in \cite{4204160}, is an example of a simple approximate algorithm. There are multiple algorithms based on it, e.g. TRW-S from \cite{kolmogorov2006convergent}. The approximate minimization is also performed in the Augmenting DAG algorithm that was introduced in \cite{schl76} or in Virtual Arc Consistency algorithm from \cite{cooper2010soft}.

These approximate algorithms were observed by \cite{Kappes-study-2013} to be significantly faster than the global methods and be able to find local minima not far from the global ones for sparse instances. This is not surprising because it has been proved in \cite{Prusa-PAMI-2015} and \cite{pruuvsa2017lp} that finding a global minimum of the upper bound for a binary max-sum problems with 3 or more labels is not easier than solving a general linear program.

The local minima found by the local methods are of the same nature -- they are local with respect to separately changing the individual variables or blocks of variables. The fact that these algorithms may not find a global minimum corresponds with the well-known fact that coordinate-wise minimization may not find a global minimum of a convex non-differentiable function. However, naively applying coordinate-wise minimization to the upper bound leads to very poor local minima, whereas the algorithms above are its clever modifications that usually yield good local minima.

The local minima found by the approximate algorithms are closely related to local consistencies or constraint propagation in CSP, in particular to arc consistency. These terms are explained for example in \cite{bessiere2006constraint}.

One can naturally ask whether the above local methods can be generalized from minimizing the upper bound (i.e. a special form of MAF or SMAF) to minimizing arbitrary MAF or SMAF. This has been outlined in \cite{2017arXiv170904989W} for max-sum diffusion. Here, the concept of arc consistency in the CSP was modified to a suitable local consistency in MAF. This in fact corresponds to the sign relaxation of the global optimality condition for MAF, which says that the subdifferential at the considered point must contain the zero vector. Then, max-sum diffusion was generalized to a simple algorithm that finds a local minimum of an arbitrary MAF.

\subsubsection{Contribution}

Our aim in this thesis is to generalize the Augmenting DAG algorithm for arbitrary convex-piecewise affine function in the form of MAF or SMAF. To do that, we first focus on the MAF form -- we show that if a point is not locally consistent in the sense of \cite{2017arXiv170904989W}, we can efficiently (faster than by solving a linear program) find an improving direction in which the function decreases. Then, we can perform a step in this direction and repeat this process iteratively until a locally consistent point is reached.

In more detail, we focus on the more general form, SMAF. We use a similar notion of local consistency and propose an algorithm that finds a decreasing direction of the function at any point that is not locally consistent and also provide a fast method of line search.

To provide better results in terms of both the final function value and runtime, we introduce a relaxed notion of local consistency, called local $\epsilon$-consistency. This relaxation improves the results and also allows us to prove the correctness of the algorithm.

Additionally, we introduce a version of the algorithm that uses only integer arithmetic and present theorems concerning the optimality of its results. The implementation of this version in C++ is enclosed and tested on instances generated using two-dimensional grammars, experimentally showing the capability to reach near-optimal results. Furthermore, the runtime of our algorithm was significantly lower than the runtime of the chosen LP solver on each tested instance.

\subsubsection{Structure}

In Chapter \ref{chapter1}, we define the binary max-sum problem and the related theory, namely what is the upper bound on quality of such problem, how it can be minimized and in what way is the upper bound connected to arc consistency. We also survey existing algorithms that minimize the upper bound of such problem.

Chapter \ref{chapter2} presents the convex piecewise-affine functions and the forms with which we will deal. We show that the upper bound of a binary max-sum problem can be viewed as a convex piecewise-affine function and we also show multiple transformation algorithms. This chapter also introduces the notion of local consistency and in what way it can be used to minimize convex piecewise-affine functions given as either maximum of affine functions or sum of maxima.

Then, Chapter \ref{chapter3} presents our minimization algorithm -- we use the previously defined local consistency, which is its core idea and follow with a detailed description of the algorithm. After the algorithm is presented, we also introduce the generalised version with local $\epsilon$-consistency and prove its correctness for real numbers and also show how to deal with issues caused by limited precision of numbers in our computers. We also present theorems concerning the optimality of the results.

In Chapter \ref{chapter4}, we discuss how the algorithm can be implemented efficiently along with simple speed-ups, mainly based on pre-calculation and aggregation of values. After the implementation is described, we calculate the asymptotic space complexity of the whole algorithm and asymptotic time complexity of one iteration. We also present the implementation details of the interface -- namely the format of the input and output file and the usage of the enclosed implementation.

Chapter \ref{chapter5} introduces two-dimensional grammars and describes how the max-sum problem of finding the nearest image from the grammar can be used to produce large sparse instances of convex piecewise-affine functions. We created such instances and run our algorithm on them. The results of the algorithm are compared to the results of an LP solver in the sense of both optimality and runtime. It is also shown how the algorithm can recalculate its result after a small change in the input data and we analyse the runtime of the algorithm in detail.

The thesis is concluded in Chapter \ref{chapter6}, where we also summarise the possible areas of further research that were encountered during the work.