IFT 6085 Winter 2013: some final project ideas ---------------------------------------------- - Explore the local statistics of a subset of all natural images, such as images of faces. Can what you have learned help perform recognition tasks in that domain? - Explore a spatio-temporal K-means variant that takes videos or image pairs as the input and learns motion features, by multiplicatively combining the K-means filter responses of individual frames. - Do a systematic study of the effect of introducing complex cells or multiplicative interactions in a recognition pipeline (for example, the CIFAR pipeline). For example, compare classification performance between simple cells and (a) complex cells, (b) simple cells responses concatenated with complex cell responses, (c) simple cell decisions averaged with complex cell decisions. - Explore the statistics of pairs of patches as a function of their spatial relationship. You will need to devise a way to introduce spatial information ("eye motion") into a feature learning model. One way to do is by using three-way connections that connect patch_1, patch_2 and a code for motion (see, eg., Larochelle & Hinton for a model that does something similar, though in a very specialized architecture). - Try to get a better understanding of the whitening requirement for feature learning: We know that for linear models, whitening is important, because encoder weights tend to be orthonormal. For non-linear models, the situation is less clear, but there is evidence that most of these models crucially rely on whitening as well. Study the whitening requirement theoretically and/or empirically, for example, by investigating the relationship between sparsity, tied weights and the empirical independence of features across a variety of models. - Implement Hoyer and Hyvarinen's contour coding (*) by training a pooling layer on top of quadrature complex cells. Investigate the usefulness of the contour representation on some recognition task (eg. the CIFAR pipeline). (*) "A multi-layer sparse coding network learns contour coding from natural images" Hoyer, Hyvarinen, 2002. - Implement one of the models in the papers we discussed in class. See if you can reproduce the results and implement a modification or improvement on that work. - Explore feature learning approaches in the context of unconventional image representations, such as depth-maps or RGB-D images, hyperspectral images, etc. You will need to get your hands on such data for this. - Using a simple recognition pipeline (eg. the CIFAR pipeline) compare the potential benefits of foveated features and keypoints over fixed resolution features extracted from a regular grid. See me if you want a simple keypoint detector in Python for this. - Study the universality of learned features. In contrast to SIFT, SURF and other handcrafted features, learned features can depend on the dataset used for training. In practice, this can be an advantage (domain specificity) or disadvantage (lack of universality). Study the effects of various parameters, like choice of non-linearities, choice of training set(s), and others, on the universality of features. To this end, evaluate classification performance using features trained from other datasets than the one you evaluate on. - We know that the responses of features learned with ICA or sparse coding are not actually independent. Especially even functions (like squares) applied to the responses, for example, tend to be correlated. A possible explanation for this is that certain filters like to come on together while others do not. This may imply that we need logical "AND" gates, which have been uncommon in feature learning, to learn better representations. Explore this hypothesis, for example, (a) by studying the correlations of products of filter responses, (b) learning a layer on top of products of ICA (or RBM/autoencoder/kmeans/etc.) features and evaluating their effect in a simple recognition task (eg. CIFAR).