Projections onto convex sets

In mathematics, projections onto convex sets (POCS), sometimes known as the alternating projection method, is a method to find a point in the intersection of two closed convex sets. It is a very simple algorithm and has been rediscovered many times.^[1] The simplest case, when the sets are affine spaces, was analyzed by John von Neumann.^[2] ^[3] The case when the sets are affine spaces is special, since the iterates not only converge to a point in the intersection (assuming the intersection is non-empty) but to the orthogonal projection of the point onto the intersection. For general closed convex sets, the limit point need not be the projection. Classical work on the case of two closed convex sets shows that the rate of convergence of the iterates is linear. ^[4] ^[5] There are now extensions that consider cases when there are more than one set, or when the sets are not convex,^[6] or that give faster convergence rates. Analysis of POCS and related methods attempt to show that the algorithm converges (and if so, find the rate of convergence), and whether it converges to the projection of the original point. These questions are largely known for simple cases, but a topic of active research for the extensions. There are also variants of the algorithm, such as Dykstra's projection algorithm. See the references in the further reading section for an overview of the variants, extensions and applications of the POCS method; a good historical background can be found in section III of.^[7]

Algorithm

Example on two circles

The POCS algorithm solves the following problem:

{\text{find}}\;x\in {\mathcal {R}}^{n}\quad {\text{such that}}\;x\in C\cap D

where C and D are closed convex sets.

To use the POCS algorithm, one must know how to project onto the sets C and D separately. The algorithm starts with an arbitrary value for $x_{0}$ and then generates the sequence

x_{{k+1}}={\mathcal {P}}_{C}\left({\mathcal {P}}_{D}(x_{k})\right).

The simplicity of the algorithm explains some of its popularity. If the intersection of C and D is non-empty, then the sequence generated by the algorithm will converge to some point in this intersection.

Unlike Dykstra's projection algorithm, the solution need not be a projection onto the intersection C and D.

Related algorithms

Example of averaged projections variant

The method of averaged projections is quite similar. For the case of two closed convex sets C and D, it proceeds by

x_{{k+1}}={\frac {1}{2}}({\mathcal {P}}_{C}(x_{k})+{\mathcal {P}}_{D}(x_{k}))

It has long been known to converge globally.^[8] Furthermore, the method is easy to generalize to more than two sets; some convergence results for this case are in.^[9]

The averaged projections method can be reformulated as alternating projections method using a standard trick. Consider the set

E=\{(x,y):x\in C,\;y\in D\}

which is defined in the product space ${\mathcal {R}}^{n}\times {\mathcal {R}}^{n}$ . Then define another set, also in the product space:

F=\{(x,y):x\in {\mathcal {R}}^{b},\,y\in {\mathcal {R}}^{n},\;x=y\}.

Thus finding $C\cap D$ is equivalent to finding $E\cap F$ .

To find a point in $E\cap F$ , use the alternating projection method. The projection of a vector $(x,y)$ onto the set F is given by $(x+y,x+y)/2$ . Hence

(x_{{k+1}},y_{{k+1}})={\mathcal {P}}_{{F}}({\mathcal {P}}_{{E}}((x_{{k}},y_{{k}})))={\mathcal {P}}_{{F}}(({\mathcal {P}}_{{C}}x_{{k}},{\mathcal {P}}_{{D}}y_{{k}}))={\frac {1}{2}}({\mathcal {P}}_{C}(x_{k})+{\mathcal {P}}_{D}(y_{k}),({\mathcal {P}}_{C}(x_{k})+{\mathcal {P}}_{D}(y_{k})).

Since $x_{{k+1}}=y_{{k+1}}$ and assuming $x_{0}=y_{0}$ , then $x_{j}=y_{j}$ for all $j\geq 0$ , and hence we can simplify the iteration to $x_{{k+1}}={\frac {1}{2}}({\mathcal {P}}_{C}(x_{k})+{\mathcal {P}}_{D}(x_{k}))$ .

References

1 2 Bauschke, H.H.; Borwein, J.M. (1996). "On projection algorithms for solving convex feasibility problems". SIAM Review. 38 (3): 367–426. doi:10.1137/S0036144593251710.
↑ "On rings of operators. Reduction theory". Ann. of Math. 50 (2): 401–485. 1949. doi:10.2307/1969463. (a reprint of lecture notes first distributed in 1933)
↑ J. von Neumann. Functional Operators, volume II. Princeton University Press, Princeton, NJ, 1950. Reprint of mimeographed lecture notes first distributed in 1933.
↑ Gubin, L.G.; Polyak, B.T.; Raik, E.V. (1967). "The method of projections for finding the common point of convex sets". U.S.S.R. Computational Mathematics and Mathematical Physics. 7: 1–24. doi:10.1016/0041-5553(67)90113-9.
↑ Bauschke, H.H.; Borwein, J.M. (1993). "On the convergence of von Neumann's alternating projection algorithm for two sets". Set-Valued Analysis. 1: 185–212. doi:10.1007/bf01027691.
↑ Lewis, A. S.; Malick, J. (2008). "Alternating Projections on Manifolds". Mathematics of Operations Research. 33: 216–234. doi:10.1287/moor.1070.0291.
↑ Combettes, P. L. (1993). "The foundations of set theoretic estimation" (PDF). Proceedings of the IEEE. 81 (2): 182–208. doi:10.1109/5.214546.
↑ A. Auslender. Methodes Numeriques pour la Resolution des Problems d’Optimisation avec Constraintes. PhD thesis, Faculte des Sciences, Grenoble, 1969
↑ Local convergence for alternating and averaged nonconvex projections. A Lewis, R Luke, J Malick, 2007. arXiv

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

Projections onto convex sets

Algorithm

Related algorithms

Further reading

References