Matrix decomposition

In the mathematical discipline of linear algebra, a matrix decomposition or matrix factorization is a factorization of a matrix into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems.

Example

In numerical analysis, different decompositions are used to implement efficient matrix algorithms.

For instance, when solving a system of linear equations $Ax=b$ , the matrix A can be decomposed via the LU decomposition. The LU decomposition factorizes a matrix into a lower triangular matrix L and an upper triangular matrix U. The systems $L(Ux)=b$ and $Ux=L^{{-1}}b$ require fewer additions and multiplications to solve, compared with the original system $Ax=b$ , though one might require significantly more digits in inexact arithmetic such as floating point.

Similarly, the QR decomposition expresses A as QR with Q an orthogonal matrix and R an upper triangular matrix. The system Q(Rx) = b is solved by Rx = Q^Tb = c, and the system Rx = c is solved by 'back substitution'. The number of additions and multiplications required is about twice that of using the LU solver, but no more digits are required in inexact arithmetic because the QR decomposition is numerically stable.

Decompositions related to solving systems of linear equations

LU decomposition

Applicable to: square matrix A
Decomposition: $A=LU$ , where L is lower triangular and U is upper triangular
Related: the LDU decomposition is $A=LDU$ , where L is lower triangular with ones on the diagonal, U is upper triangular with ones on the diagonal, and D is a diagonal matrix.
Related: the LUP decomposition is $A=LUP$ , where L is lower triangular, U is upper triangular, and P is a permutation matrix.
Existence: An LUP decomposition exists for any square matrix A. When P is an identity matrix, the LUP decomposition reduces to the LU decomposition. If the LU decomposition exists, then the LDU decomposition exists.^[1]
Comments: The LUP and LU decompositions are useful in solving an n-by-n system of linear equations $Ax=b$ . These decompositions summarize the process of Gaussian elimination in matrix form. Matrix P represents any row interchanges carried out in the process of Gaussian elimination. If Gaussian elimination produces the row echelon form without requiring any row interchanges, then P = I, so an LU decomposition exists.

LU reduction

Block LU decomposition

Rank factorization

Applicable to: m-by-n matrix A of rank r
Decomposition: $A=CF$ where C is an m-by-r full column rank matrix and F is an r-by-n full row rank matrix
Comment: The rank factorization can be used to compute the Moore–Penrose pseudoinverse of A,^[2] which one can apply to obtain all solutions of the linear system $Ax=b$ .

Cholesky decomposition

Applicable to: square, hermitian, positive definite matrix A
Decomposition: $A=U^{*}U$ , where U is upper triangular with real positive diagonal entries
Comment: if the matrix A is Hermitian and positive semi-definite, then it has a decomposition of the form $A=U^{*}U$ if the diagonal entries of $U$ are allowed to be zero
Uniqueness: for positive definite matrices Cholesky decomposition is unique. However, it is not unique in the positive semi-definite case.
Comment: if A is real and symmetric, $U$ has all real elements
Comment: An alternative is the LDL decomposition, which can avoid extracting square roots.

QR decomposition

Applicable to: m-by-n matrix A
Decomposition: $A=QR$ where Q is an unitary matrix of size m-by-m, and R is an upper triangular matrix of size m-by-n
Uniqueness: In general it is not unique, but if $A$ is of full rank, then there exists a single $R$ that has all positive diagonal elements. If $A$ is square, also $Q$ is unique.
Comment: The QR decomposition provides an alternative way of solving the system of equations $Ax=b$ without inverting the matrix A. The fact that Q is orthogonal means that $Q^{T}Q=I$ , so that $Ax=b$ is equivalent to $Rx=Q^{T}b$ , which is easier to solve since R is triangular.

RRQR factorization

Interpolative decomposition

Decompositions based on eigenvalues and related concepts

Eigendecomposition

Also called spectral decomposition
Applicable to: square matrix A with linearly independent eigenvectors (not necessarily distinct eigenvalues).
Decomposition: $A=VDV^{{-1}}$ , where D is a diagonal matrix formed from the eigenvalues of A, and the columns of V are the corresponding eigenvectors of A.
Existence: An n-by-n matrix A always has n (complex) eigenvalues, which can be ordered (in more than one way) to form an n-by-n diagonal matrix D and a corresponding matrix of nonzero columns V that satisfies the eigenvalue equation $AV=VD$ . $V$ is invertible if and only if the n eigenvectors are linearly independent (i.e., each eigenvalue has geometric multiplicity equal to its algebraic multiplicity). A sufficient (but not necessary) condition for this to happen is that all the eigenvalues are different (in this case geometric and algebraic multiplicity are equal to 1)
Comment: One can always normalize the eigenvectors to have length one (see definition of the eigenvalue equation)
Comment: Every normal matrix A (i.e., matrix for which $AA^{*}=A^{*}A$ , where $A^{*}$ is a conjugate transpose) can be eigendecomposed. For normal matrix A (and only for normal matrix), the eigenvectors can also be made orthonormal ( $VV^{*}=I$ ) and eigendecomposition reads as $A=VDV^{*}$ . In particular all unitary, Hermitian, and skew-Hermitian (in real-valued case, all orthogonal, symmetric, and skew-symmetric, respectively) matrices are normal and therefore possess this property.
Comment: For real symmetric matrix A the eigendecomposition always exists and can be written as $A=VDV^{T}$ , where both D and V are real-valued.
Comment: The eigendecomposition is useful for understanding the solution of a system of linear ordinary differential equations or linear difference equations. For example, the difference equation $x_{{t+1}}=Ax_{t}$ starting from the initial condition $x_{0}=c$ is solved by $x_{t}=A^{t}c$ , which is equivalent to $x_{t}=VD^{t}V^{{-1}}c$ , where V and D are the matrices formed from the eigenvectors and eigenvalues of A. Since D is diagonal, raising it to power $D^{t}$ , just involves raising each element on the diagonal to the power t. This is much easier to do and to understand than raising A to power t, since A is usually not diagonal.

Jordan decomposition

The Jordan normal form and the Jordan–Chevalley decomposition

Applicable to: square matrix A
Comment: the Jordan normal form generalizes the eigendecomposition to cases where there are repeated eigenvalues and cannot be diagonalized, the Jordan–Chevalley decomposition does this without choosing a basis.

Schur decomposition

Applicable to: square matrix A
Decomposition (complex version): $A=UTU^{*}$ , where U is a unitary matrix, $U^{*}$ is the conjugate transpose of U, and T is an upper triangular matrix called the complex Schur form which has the eigenvalues of A along its diagonal.
Comment: if A is a normal matrix, then T is diagonal and the Schur decomposition coincides with the spectral decomposition.

Real Schur decomposition

Applicable to: square matrix A
Decomposition: This is a version of Shur decomposition where $V$ and $S$ only contain real numbers. One can always write $A=VSV^{T}$ where V is a real orthogonal matrix, $V^{T}$ is the transpose of V, and S is a block upper triangular matrix called the real Schur form. The blocks on the diagonal of S are of size 1×1 (in which case they represent real eigenvalues) or 2×2 (in which case they are derived from complex conjugate eigenvalue pairs).

QZ decomposition

Also called: generalized Schur decomposition
Applicable to: square matrices A and B
Comment: there are two versions of this decomposition: complex and real.
Decomposition (complex version): $A=QSZ^{H}$ and $B=QTZ^{H}$ where Q and Z are unitary matrices, the H superscript represents conjugate transpose, and S and T are upper triangular matrices.
Comment: in the complex QZ decomposition, the ratios of the diagonal elements of S to the corresponding diagonal elements of T, $\lambda _{i}=S_{{ii}}/T_{{ii}}$ , are the generalized eigenvalues that solve the generalized eigenvalue problem $Av=\lambda Bv$ (where $\lambda$ is an unknown scalar and v is an unknown nonzero vector).
Decomposition (real version): $A=QSZ^{T}$ and $B=QTZ^{T}$ where A, B, Q, Z, S, and T are matrices containing real numbers only. In this case Q and Z are orthogonal matrices, the T superscript represents transposition, and S and T are block upper triangular matrices. The blocks on the diagonal of S and T are of size 1×1 or 2×2.

Takagi's factorization

Applicable to: square, complex, symmetric matrix A.
Decomposition: $A=VDV^{T}$ , where D is a real nonnegative diagonal matrix, and V is unitary. $V^{T}$ denotes the matrix transpose of V.
Comment: The diagonal elements of D are the nonnegative square roots of the eigenvalues of $AA^{H}$ .
Comment: V may be complex even if A is real.
Comment: This is not a special case of the eigendecomposition (see above).

Singular value decomposition

Applicable to: m-by-n matrix A.
Decomposition: $A=UDV^{*}$ , where D is a nonnegative diagonal matrix, and U and V are unitary matrices, and $V^{*}$ is the conjugate transpose of V (or simply the transpose, if V contains real numbers only).
Comment: The diagonal elements of D are called the singular values of A.
Comment: Like the eigendecomposition above, the singular value decomposition involves finding basis directions along which matrix multiplication is equivalent to scalar multiplication, but it has greater generality since the matrix under consideration need not be square.
Uniqueness: the singular values of $A$ are always uniquely determined. $U$ and $V$ need not to be unique in general.

Other decompositions

Polar decomposition

Applicable to: any square complex matrix A.
Decomposition: $A=UP$ (right polar decomposition) or $A=P'U$ (left polar decomposition), where U is a unitary matrix and P and P' are positive semidefinite Hermitian matrices.
Uniqueness: $P$ is always unique and equal to $\sqrt{A^*A}$ (which is always hermitian and positive semidefinite). If $A$ is invertible, then $U$ is unique.
Comment: Since any Hermitian matrix admits a spectral decomposition with a unitary matrix, $P$ can be written as $P=VDV^{*}$ . Since $P$ is positive semidefinite, all elements in $D$ are non-negative. Since the product of two unitary matrices is unitary, taking $W=UV$ one can write $A=U(VDV^{*})=WDV^{*}$ which is the singular value decomposition. Hence, the existence of the polar decomposition is equivalent to the existence of the singular value decomposition.

Algebraic polar decomposition

Applicable to: square, complex, non-singular matrix A.^[3]
Decomposition: $A=QS$ , where Q is a complex orthogonal matrix and S is complex symmetric matrix.
Uniqueness: If $A^{{T}}A$ has no negative real eigenvalues, then the decomposition is unique.^[4]
Comment: The existence of this decomposition is equivalent to $AA^{{T}}$ being similar to $A^{{T}}A$ .^[5]
Comment: A variant of this decomposition is $A=RC$ , where R is a real matrix and C is a circular matrix.^[4]

Mostow's decomposition

Applicable to: square, complex, non-singular matrix A.^[6]^[7]
Decomposition: $A=Ue^{iM}e^{S}$ , where U is unitary, M is real anti-symmetric and S is real symmetric.
Comment: The matrix A can also be decomposed as $A=U_{2}e^{S_{2}}e^{iM_{2}}$ , where U₂ is unitary, M₂ is real anti-symmetric and S₂ is real symmetric.^[4]

Sinkhorn normal form

Applicable to: square real matrix A with strictly positive elements.
Decomposition: $A=D_{{1}}SD_{{2}}$ , where S is doubly stochastic and D₁ and D₂ are real diagonal matrices with strictly positive elements.

Sectoral decomposition^[8]

Applicable to: square, complex matrix A with numerical range contained in the sector $S_{\alpha }=\{re^{i\theta }\in \mathbb {C} \mid r>0,|\theta |\leq \alpha <{\frac {\pi }{2}}\}$ .
Decomposition: $A=CZC^{{H}}$ , where C is an invertible complex matrix and $Z=diag(e^{{i\theta _{1}}},\ldots ,e^{{i\theta _{n}}})$ with all $|\theta _{{j}}|\leq \alpha$ .^[8]^[9]

Williamson's normal form^[10]

Applicable to: square, positive-definite real matrix A with order 2n-by-2n.
Decomposition: $A=S^{T}{\textrm {diag}}(D,D)S$ , where S is a symplectic matrix and D is a nonnegative diagonal matrix.

Generalizations

There exist analogues of the SVD, QR, LU and Cholesky factorizations for quasimatrices and cmatrices or continuous matrices.^[11] A ‘quasimatrix’ is, like a matrix, a rectangular scheme whose elements are indexed, but one discrete index is replaced by a continuous index. Likewise, a ‘cmatrix’, is continuous in both indices. As an example of a cmatrix, one can think of the kernel of an integral operator.

These factorizations are based on early work by Fredholm (1903), Hilbert (1904) and Schmidt (1907). For an account, and a translation to English of the seminal papers, see Stewart (2011).

Notes

↑ Simon & Blume 1994 Chapter 7.
↑ Piziak, R.; Odell, P. L. (1 June 1999). "Full Rank Factorization of Matrices". Mathematics Magazine. 72 (3): 193. doi:10.2307/2690882.
↑ Choudhury & Horn 1987, pp. 219–225
1 2 3 Bhatia, Rajendra (2013-11-15). "The bipolar decomposition". Linear Algebra and its Applications. 439 (10): 3031–3037. doi:10.1016/j.laa.2013.09.006.
↑ Horn & merino 1995, pp. 43–92
↑ Mostow, G. D. (1955), Some new decomposition theorems for semi-simple groups, Mem. Amer. Math. Soc., 14, pp. 31–54
↑ Nielsen, Frank; Bhatia, Rajendra (2012). Matrix Information Geometry. Springer. p. 224. ISBN 9783642302329. doi:10.1007/978-3-642-30232-9.
1 2 Zhang, Fuzhen (30 June 2014). "A matrix decomposition and its applications". Linear and Multilinear Algebra: 1–10. doi:10.1080/03081087.2014.933219.
↑ Drury, S.W. (November 2013). "Fischer determinantal inequalities and Highamʼs Conjecture". Linear Algebra and its Applications. 439 (10): 3129–3133. doi:10.1016/j.laa.2013.08.031.
↑ Idel, Martin; Soto Gaona, Sebastián; Wolf, Michael M. (2017-07-15). "Perturbation bounds for Williamson's symplectic normal form". Linear Algebra and its Applications. 525: 45–58. doi:10.1016/j.laa.2017.03.013.
↑ Townsend & Trefethen 2015

References

Choudhury, Dipa; Horn, Roger A. (April 1987). "A Complex Orthogonal-Symmetric Analog of the Polar Decomposition". SIAM Journal on Algebraic Discrete Methods. 8 (2). doi:10.1137/0608019.
Fredholm, I. (1903), "Sur une classe d’´equations fonctionnelles", Acta Mathematica (in French), 27: 365–390, doi:10.1007/bf02421317
Hilbert, D. (1904), "Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen", Nachr. Königl. Ges. Gött (in German), 1904: 49–91
Horn, Roger A.; Merino, Dennis I. (January 1995). "Contragredient equivalence: A canonical form and some applications". Linear Algebra and its Applications. 214. doi:10.1016/0024-3795(93)00056-6.
Meyer, C. D. (2000), Matrix Analysis and Applied Linear Algebra, SIAM, ISBN 978-0-89871-454-8
Schmidt, E. (1907), "Zur Theorie der linearen und nichtlinearen Integralgleichungen. I Teil. Entwicklung willkürlichen Funktionen nach System vorgeschriebener", Mathematische Annalen (in German), 63: 433–476, doi:10.1007/bf01449770
Simon, C.; Blume, L. (1994). Mathematics for Economists. Norton. ISBN 0-393-95733-0.
Stewart, G. W. (2011), Fredholm, Hilbert, Schmidt: three fundamental papers on integral equations (PDF), retrieved 2015-01-06
Townsend, A.; Trefethen, L. N. (2015), "Continuous analogues of matrix factorizations", Proc. R. Soc. A, 471 (2173), doi:10.1098/rspa.2014.0585

External links

Online Matrix Calculator
Wolfram Alpha Matrix Decomposition Computation » LU and QR Decomposition
Springer Encyclopaedia of Mathematics » Matrix factorization
GraphLab GraphLab collaborative filtering library, large scale parallel implementation of matrix decomposition methods (in C++) for multicore.

Linear algebra
Basic concepts	Scalar Vector Vector space Scalar multiplication Vector projection Linear span Linear map Linear projection Linear independence Linear combination Basis Column space Row space Dual space Orthogonality Kernel Eigenvalues and eigenvectors Outer product Inner product space Dot product Transpose Gram–Schmidt process Linear equations
Vector algebra	Cross product Triple product Seven-dimensional cross product
Multilinear algebra	Geometric algebra Exterior algebra Bivector Multivector
Matrices	Block Decomposition Invertible Minor Multiplication Rank Transformation Cramer's rule Gaussian elimination
Numerical	Floating point Matrix Laboratory Numerical stability Basic Linear Algebra Subprograms (BLAS) Sparse matrix Comparison of linear algebra libraries Comparison of numerical analysis software

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.