Calculus

A deep net is trained by optimizing a loss function, and that means taking derivatives through matrices, not single numbers. This page is the starting point for the matrix calculus used when differentiating through neural nets: one explained.ai note to read first, and an open invitation to grow the page.

The note to read first is Start here!, "The Matrix Calculus You Need For Deep Learning". It starts from the fact that most of us last saw calculus in school, even though derivatives are a critical part of training deep neural networks, and tries to explain all the matrix calculus needed to understand that training while assuming nothing beyond calculus 1.

That one note is all the page holds for now. Help needed to make this page great, please contact me.

Deprecated links

These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.

• Start here! This address no longer opens: http://parrt.cs.usfca.edu/doc/matrix-calculus/