Isotonic regression

From Wikipedia, the free encyclopedia

In numerical analysis, isotonic regression (IR) involves finding a weighted least-squares fit $x\in \Bbb{R}^n$ to a vector $a\in \Bbb{R}^n$ with weights vector $w\in \Bbb{R}^n$ subject to a set of monotonicity constraints giving a simple or partial order over the variables. The monotonicity constraints define a directed acyclic graph $G = (N, E)$ over the nodes $N={1,2,\ldots,n}$ corresponding to the variables $x={x_1,x_2,\ldots,x_n}$ . Thus, the IR problem where a simple order is defined corresponds to the following quadratic program (QP):

$\min \sum_{i=1}^n w_i (x_i - a_i)^2$ $\mathrm{subject~to~}x_i\ge x_j~\forall (i,j)\in E.$

In the case when $G = (N, E)$ is a total order, a simple iterative algorithm for solving this QP is called the pool adjacent violators algorithm (PAVA). Best and Chakravarti (1990) have studied the problem as an active set identification problem, and have proposed a primal algorithm in O(n), the same complexity as the PAVA, which can be seen as a dual algorithm.

IR has applications in statistical inference, for example, computing the cost at the minimum of the above goal function, gives the "stress" of the fit of an isotonic curve to mean experimental results when an order is expected.

Another application is nonmetric multidimensional scaling (Kruskal, 1964), where a low-dimensional embedding for data points is sought such that order of distances between points in the embedding matches order of dissimilarity between points. Isotonic regression is used iteratively to fit ideal distances to preserve relative dissimilarity order.

Isotonic regression is also sometimes referred to as monotonic regression. Correctly speaking, isotonic is used when the direction of the trend is strictly increasing, while monotonic could imply a trend that is either strictly increasing or strictly decreasing.

Isotonic Regression under the $L p$ for $p > 0$ is defined as follows:

$\min \sum_{i=1}^n w_i |x_i - a_i|^p$ $\mathrm{subject~to~}x_i\ge x_j~\forall (i,j)\in E.$

[edit] References

Please expand this article using the suggested source(s) below.
More information might be found in a section of the talk page.

Best, M.J.; & Chakravarti N. (1990) "Active set algorithms for isotonic regression; a unifying framework". Mathematical Programming 47, 425–439.

Robertson, T.; Wright, F.T.; & Dykstra, R.L. (1988). Order restricted statistical inference. New York: Wiley, 1988. ISBN 0-471-91787-7

Barlow, R. E.; Bartholomew, D.J.; Bremner, J. M.; & Brunk, H. D. (1972). Statistical inference under order restrictions; the theory and application of isotonic regression. New York: Wiley, 1972. ISBN 0-471-04970-0.