Evaluating and Learning Invariant Features

par/by Ian Goodfellow
Stanford

One approach to unsupervised learning is to attempt to discover a function that decomposes the input into variables representing distinct factors of variation. A weaker version of this is to learn a representation where each variable is invariant to a small number of confounding aspects of the input. Experimental results show that the current degree of invariance achieved by some deep learning algorithms is small compared to that achieved by systems hand-designed to be invariant to specific transformations. I discuss some ideas for future invariance evaluation experiments and for methods of automatically learning to separate factors of variation.