矩阵微积分-没你想象的那么简单！.pdf

文档介绍：Matrix Calculus and
Algebra
Yi Zhang
Outline
 Matrix calculus and algebra
 DimensioMatrix Calculus and
Algebra
Yi ZhangOutline
 Matrix calculus and algebra
 Dimensions of derivatives
 Basic calculations of matrix derivatives
 Rules for product, chain, trace, determinant and
normsMatrix Derivatives:
Dimensions
 The basic case: y scalar, x scalar
 dy/dx: scalar
 Generalize to vectors/matricesMatrix Derivatives:
Dimensions [T. Minka]Basic Calculations
 Step 1: know the dimensions
 Step 2: element-wise calculations
 Step 3: put into the vector/matrix formBasic Calculations
 Example:
 Dimensions? Element-wise? Vector/matrix form?Basic Calculations
 Example:Basic Calculations
 Too simple? Try this … [J. Rennie]
 We want:Basic Calculations
… …Basic CalculationsSome ExamplesGradient and HessianSome Basic Rules
 Take derivatives . z (omitted)Product Rules
 Take derivatives . z (omitted)Derivatives of Determinants
 Take derivatives . z (omitted)The Chain Rule
 Suppose
 The chain rule:Derivatives of Traces [. Petersen]
 Assume F(X) is an element-wise
differentiable function
 f() is the scalar derivative of F()Derivatives of Traces [. Petersen]
 Given:Derivatives of Frobenius Norm
 Frobenius norm
 DerivativesReferences
 T. Minka. Old and New Matrix Algebra Useful
for Statistics
 K. B. Petersen. The Matrix Cookbook
 JDMRJ. D. M. Renni e. A ASi Simpl e E xerci se on MtiMatrix
Derivatives.