We introduce a new neural architecture and an unsupervised algorithm for learning invariant representations from temporal sequence of images. The system uses two groups of complex cells whose outputs are combined multiplicatively: one that represents the content of the image, constrained to be constant over several consecutive frames, and one that represents the precise location of features, which is allowed to vary over time. The architecture uses an encoder to extract features, and a decoder to reconstruct the input from the features. The method was applied to patches extracted from consecutive movie frames and produces orientation and frequency selective units analogous to the complex cells in V1. An extension of the method was used to train a network composed of units with local receptive field spread over a large image of arbitrary size. Pooling over local neighborhoods was also used which produces orientation-selective cells that are organized in pinwheel patterns similar to those observed in the mammalian visual cortex. By adding logistic regression classifier applied to complex cell layer, the complete system becomes a very fast, biologically inspired object recognition system - it contains locally-connected simple and complex like cells, it can be applied to full sized images, exhibits pinwheel patterns and has a feed-forward encoder for efficient feature computation.