Singular value decomposition

In <a href="/facts/Linear_algebra/8VaB4VWk">linear algebra</a>, the singular value decomposition (SVD) is a <a href="/facts/Matrix_decomposition/xVBnAg0x">factorization</a> of a <a href="/facts/Real_number/R02gw5Pb">real</a> or <a href="/facts/Complex_number/2w7mqI7e">complex</a> <a href="/facts/Matrix_(mathematics)/qa8FI4ko">matrix</a> into a rotation, followed by a rescaling followed by another rotation. It generalizes the <a href="/facts/Eigendecomposition/a2nfF7hJ">eigendecomposition</a> of a square <a href="/facts/Normal_matrix/WVKxrEK9">normal matrix</a> with an orthonormal eigenbasis to any ⁠
 
 
 
 m
 ×
 n
 
 
 {\displaystyle m\times n}
 
⁠ matrix. It is related to the <a href="/facts/Polar_decomposition/XWQLdp9B">polar decomposition</a>.
Specifically, the singular value decomposition of an 
 
 
 
 m
 ×
 n
 
 
 {\displaystyle m\times n}
 
 complex matrix ⁠
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
⁠ is a factorization of the form 
 
 
 
 
 M
 
 =
 
 U
 Σ
 
 V
 
 ∗
 
 
 
 ,
 
 
 {\displaystyle \mathbf {M} =\mathbf {U\Sigma V^{*}} ,}
 
 where ⁠
 
 
 
 
 U
 
 
 
 {\displaystyle \mathbf {U} }
 
⁠ is an ⁠
 
 
 
 m
 ×
 m
 
 
 {\displaystyle m\times m}
 
⁠ complex <a href="/facts/Unitary_matrix/8fsxv3KD">unitary matrix</a>, 
 
 
 
 
 Σ
 
 
 
 {\displaystyle \mathbf {\Sigma } }
 
 is an 
 
 
 
 m
 ×
 n
 
 
 {\displaystyle m\times n}
 
 <a href="/facts/Rectangular_diagonal_matrix/tjIAHrE6">rectangular diagonal matrix</a> with non-negative real numbers on the diagonal, ⁠
 
 
 
 
 V
 
 
 
 {\displaystyle \mathbf {V} }
 
⁠ is an 
 
 
 
 n
 ×
 n
 
 
 {\displaystyle n\times n}
 
 complex unitary matrix, and 
 
 
 
 
 
 V
 
 
 ∗
 
 
 
 
 {\displaystyle \mathbf {V} ^{*}}
 
 is the <a href="/facts/Conjugate_transpose/7eBWtqVG">conjugate transpose</a> of ⁠
 
 
 
 
 V
 
 
 
 {\displaystyle \mathbf {V} }
 
⁠. Such decomposition always exists for any complex matrix. If ⁠
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
⁠ is real, then ⁠
 
 
 
 
 U
 
 
 
 {\displaystyle \mathbf {U} }
 
⁠ and ⁠
 
 
 
 
 V
 
 
 
 {\displaystyle \mathbf {V} }
 
⁠ can be guaranteed to be real <a href="/facts/Orthogonal_matrix/WDXBXYB5">orthogonal</a> matrices; in such contexts, the SVD is often denoted 
 
 
 
 
 U
 
 
 Σ
 
 
 
 V
 
 
 
 T
 
 
 
 .
 
 
 {\displaystyle \mathbf {U} \mathbf {\Sigma } \mathbf {V} ^{\mathrm {T} }.}

The diagonal entries 
 
 
 
 
 σ
 
 i
 
 
 =
 
 Σ
 
 i
 i
 
 
 
 
 {\displaystyle \sigma _{i}=\Sigma _{ii}}
 
 of 
 
 
 
 
 Σ
 
 
 
 {\displaystyle \mathbf {\Sigma } }
 
 are uniquely determined by ⁠
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
⁠ and are known as the <a href="/facts/Singular_value/0QWHnqGG">singular values</a> of ⁠
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
⁠. The number of non-zero singular values is equal to the <a href="/facts/Rank_of_a_matrix/Yp9b5LK3">rank</a> of ⁠
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
⁠. The columns of ⁠
 
 
 
 
 U
 
 
 
 {\displaystyle \mathbf {U} }
 
⁠ and the columns of ⁠
 
 
 
 
 V
 
 
 
 {\displaystyle \mathbf {V} }
 
⁠ are called left-singular vectors and right-singular vectors of ⁠
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
⁠, respectively. They form two sets of <a href="/facts/Orthonormal_basis/pzfKHlJG">orthonormal bases</a> ⁠
 
 
 
 
 
 u
 
 
 1
 
 
 ,
 …
 ,
 
 
 u
 
 
 m
 
 
 
 
 {\displaystyle \mathbf {u} _{1},\ldots ,\mathbf {u} _{m}}
 
⁠ and ⁠
 
 
 
 
 
 v
 
 
 1
 
 
 ,
 …
 ,
 
 
 v
 
 
 n
 
 
 ,
 
 
 {\displaystyle \mathbf {v} _{1},\ldots ,\mathbf {v} _{n},}
 
⁠ and if they are sorted so that the singular values 
 
 
 
 
 σ
 
 i
 
 
 
 
 {\displaystyle \sigma _{i}}
 
 with value zero are all in the highest-numbered columns (or rows), the singular value decomposition can be written as

 
 
 
 
 M
 
 =
 
 ∑
 
 i
 =
 1
 
 
 r
 
 
 
 σ
 
 i
 
 
 
 
 u
 
 
 i
 
 
 
 
 v
 
 
 i
 
 
 ∗
 
 
 ,
 
 
 {\displaystyle \mathbf {M} =\sum _{i=1}^{r}\sigma _{i}\mathbf {u} _{i}\mathbf {v} _{i}^{*},}

where 
 
 
 
 r
 ≤
 min
 {
 m
 ,
 n
 }
 
 
 {\displaystyle r\leq \min\{m,n\}}
 
 is the rank of ⁠
 
 
 
 
 M
 
 .
 
 
 {\displaystyle \mathbf {M} .}
 
⁠
The SVD is not unique. However, it is always possible to choose the decomposition such that the singular values 
 
 
 
 
 Σ
 
 i
 i
 
 
 
 
 {\displaystyle \Sigma _{ii}}
 
 are in descending order. In this case, 
 
 
 
 
 Σ
 
 
 
 {\displaystyle \mathbf {\Sigma } }
 
 (but not ⁠
 
 
 
 
 U
 
 
 
 {\displaystyle \mathbf {U} }
 
⁠ and ⁠
 
 
 
 
 V
 
 
 
 {\displaystyle \mathbf {V} }
 
⁠) is uniquely determined by ⁠
 
 
 
 
 M
 
 .
 
 
 {\displaystyle \mathbf {M} .}
 
⁠
The term sometimes refers to the compact SVD, a similar decomposition ⁠
 
 
 
 
 M
 
 =
 
 
 U
 Σ
 V
 
 
 ∗
 
 
 
 
 {\displaystyle \mathbf {M} =\mathbf {U\Sigma V} ^{*}}
 
⁠ in which ⁠
 
 
 
 
 Σ
 
 
 
 {\displaystyle \mathbf {\Sigma } }
 
⁠ is square diagonal of size ⁠
 
 
 
 r
 ×
 r
 ,
 
 
 {\displaystyle r\times r,}
 
⁠ where ⁠
 
 
 
 r
 ≤
 min
 {
 m
 ,
 n
 }
 
 
 {\displaystyle r\leq \min\{m,n\}}
 
⁠ is the rank of ⁠
 
 
 
 
 M
 
 ,
 
 
 {\displaystyle \mathbf {M} ,}
 
⁠ and has only the non-zero singular values. In this variant, ⁠
 
 
 
 
 U
 
 
 
 {\displaystyle \mathbf {U} }
 
⁠ is an ⁠
 
 
 
 m
 ×
 r
 
 
 {\displaystyle m\times r}
 
⁠ <a href="/facts/Semi-orthogonal_matrix/XSQd0jH6">semi-unitary matrix</a> and 
 
 
 
 
 V
 
 
 
 {\displaystyle \mathbf {V} }
 
 is an ⁠
 
 
 
 n
 ×
 r
 
 
 {\displaystyle n\times r}
 
⁠ <a href="/facts/Semi-orthogonal_matrix/XSQd0jH6">semi-unitary matrix</a>, such that 
 
 
 
 
 
 U
 
 
 ∗
 
 
 
 U
 
 =
 
 
 V
 
 
 ∗
 
 
 
 V
 
 =
 
 
 I
 
 
 r
 
 
 .
 
 
 {\displaystyle \mathbf {U} ^{*}\mathbf {U} =\mathbf {V} ^{*}\mathbf {V} =\mathbf {I} _{r}.}

Mathematical applications of the SVD include computing the <a href="/facts/Moore%25E2%2580%2593Penrose_pseudoinverse/CU73tntq">pseudoinverse</a>, matrix approximation, and determining the rank, <a href="/facts/Range_of_a_matrix/INXbQVNI">range</a>, and <a href="/facts/Kernel_(matrix)/adhGUYa0">null space</a> of a matrix. The SVD is also extremely useful in many areas of science, <a href="/facts/Engineering/RFVg8H3I">engineering</a>, and <a href="/facts/Statistics/DNPsKGYU">statistics</a>, such as <a href="/facts/Signal_processing/Npaja6zb">signal processing</a>, <a href="/facts/Least_squares/50jIwbxC">least squares</a> fitting of data, and <a href="/facts/Process_control/tmjQhchm">process control</a>.

Singular value decomposition open-in-new

Singular value decomposition