# Statistical Learning of Complex Data with Complex Distributions

Pr. Yoshua Bengio

Canada Research Chair in Statistical Learning Algorithms

Adaptative Computer
Science Laboratory (LISA)

*Departement of
Computer Science and Operations Research
Université de Montréal
Yoshua.Bengio@umontreal.ca*

## Summary

As we move ever more into a digital age, organizations and individuals are acquiring increasing quantities of data, and advances in computer technology make it possible to efficiently process gigabytes or even terabytes of data. But algorithms able to understand that deluge of data are lagging. This is making it all the more important to invest in research to develop better automated tools for interpretation and analysis of data, so that it can be used to take decisions.

The field of *statistical learning* can be viewed as a set of theories and techniques whose
purpose is precisely the discovery of useful regularities from a data
source, for improved predictions or understanding. These algorithms
are often rather general in nature, able to work on data of different
types, from the array of pixels measured by a camera to
the values in a retail database. In most cases, the raw input to these algorithms
is a flow of "examples", each of which being characterized by
many observed variables, related by unknown intricate statistical relationships.

Statistical machine learning is a core component of *data-mining*,
a corpus of techniques applied in industrial settings generally
involving large quantities of data, with the objective of extracting something
useful from the data. Mathematically, one can think of a *learning algorithm*
as an object that maps a data set into a function. The data set is a collection
of examples (possibly ordered). The learned function is something we can use
to answer questions, typically about new examples coming from the same underlying
and unknown process that generated our data set.

Recent years have seen impressive advances in statistical learning
methodology and data-mining, with a plethora of applications in many areas
of science, industry, and business. However, despite these advances, the
progress offered by machine learning to many long-standing problems remains
relatively modest. Typically, these harder problems involve learning from
data with very rich statistical structure. In practice, and especially with
most data-mining applications, traditional machine learning and statistics
research has focused on relatively simple problems. The data distribution
was explicitly *or implicitly* assumed to have a rather simple form,
either because it is known ahead of time up to a few parameters, because it
is assumed to be rather smooth, or because it only involves a few
interactions between variables.
Another challenge arises because data is almost always *incomplete*;
that is, relatively few training instances
have been fully analyzed to reveal their target structure (e.g. a complete
sentence analysis, a complete image parse, objective robot position, or
true cell conditions), even though raw input observations are readily
available (unanalyzed sentences, raw images, sensor readings, microrarray data).
The need to infer missing components while
training a predictive model remains a source of conceptual and computational difficulty.

One long-term goal of machine learning research is to produce methods that are applicable to complex tasks, such as perception, reasoning, intelligent control, and other artificially intelligent behaviors. However, despite impressive progress on both the academic and technological sides, the aforementioned challenges have yet to be adequately met and these long-term goals remain elusive.