This article discusses the derivative of the softmax function, providing an introductory example suitable for readers with a basic understanding of single-variable calculus. It outlines the mathematical principles necessary for computing this derivative and presents a concise expression for the softmax function’s Jacobian matrix, showcasing its application in probability distributions.
The Singular Value Decomposition (SVD) is “a highlight of linear algebra” to quote Prof. Strang ( [1] p. 371). However, I must confess that when I studied it I had a difficult time understanding it and this was due to how it was presented. The SVD is often introduced as a given formula which is then shown to just work. But it always felt very unsatisfying to me not knowing why. So – here is the SVD explained the way I wish I had been taught, which is deriving it from first principles.
