A diagram of the Singular Value Decomposition (SVD) formula showing matrix A being decomposed into matrices U, Sigma, and V-star. The image displays the dimensions of each matrix and uses color-coding to show how specific columns of U, diagonal values of Sigma, and rows of V-star are linked together as sets of triplets.
Linear Algebra

Deriving the Singular Value Decomposition (SVD) from First Principles

The Singular Value Decomposition (SVD) is “a highlight of linear algebra” to quote Prof. Strang ( [1] p. 371). However, I must confess that when I studied it I had a difficult time understanding it and this was due to how it was presented. The SVD is often introduced as a given formula which is then shown to just work. But it always felt very unsatisfying to me not knowing why. So – here is the SVD explained the way I wish I had been taught, which is deriving it from first principles.