### Topmoumoute Online Natural Gradient Algorithm

par/by *Nicolas Le Roux*

Département d'Informatique et de Recherche Opérationnelle

Université de Montréal

Many machine learning algorithms, such as neural networks, are trained
using gradient descent. Natural gradient descent is a variant of the
usual gradient descent which has been popularized by Amari. At the time,
its justification was to be the direction of steepest descent in an
underlying Riemannian space.

We first present a statistical approach which justifies the use of
natural gradient descent for limiting overfitting. Then, we will develop
an efficient approach of this gradient, making it practical for online
learning.