|
|
Machine Learning Laboratory
Canada Research Chair in Statistical Learning Algorithms
Machine learning in adaptive computer systems is an area of research
at the intersection of artificial intelligence, statistical inference and
optimization. Such systems must learn to perform a task from a set of
examples and a priori knowledge about the task. These systems are often
also adaptive, i.e. can adapt to changes in their environment. Several of
the algorithms used in our laboratory are from research in artificial
neural networks, inspired from the way in which the brain processes
information. Many algorithms are expressed in the framework of
probabilistic modeling, where uncertainty is expressed explicitly, and
relations between random variables are captured by formulas called factors,
that can be visualized graphically.
Machine learning algorithms are important when one does not have enough
explicit knowledge about the problem to directly write a program or a set
of rules that will solve the task, or when we don’t really know the form of
the true distribution from which the observed data are coming from. In some
cases one can obtain more information about the problem in the form of
examples of the task to be performed, usually a set of (input, output)
pairs (e.g. of patterns, or sequences). These examples tell the system
what it should produce in output in a certain situation represented by its
input, or what joint values are plausible for the observed variables. These
examples must then be combined with explicit knowledge about the problem to
design the system architecture and representation, and choose an
appropriate learning algorithm.
The LISA (machine learning lab) aims towards improving our understanding
of the principles that give rise to powerful learning and to intelligence,
which will be important to make significant progress on learning
algorithms and artificial intelligence (AI).
Acquiring the kind of complex knowledge necessary for AI requires
some form of learning, with the ability to discover hidden relationships
and statistical structure that may be highly complex, with many
interacting factors of variations explaining the observed high-dimensional
data that sensors can provide. According to us this is the main
challenge for machine learning and AI.
Like the brain, deep learning algorithms are based on several levels of
representation and processing, creating several levels of levels of
abstraction. Compared to learning algorithms based on shallower
architectures, deep learners have the potential to efficiently represent
highly complex functions and distributions. We explore various learning
algorithms for deep learning, based in particular on unsupervised
pre-training (e.g., various kinds of Boltzmann machines and auto-encoders).
Unsupervised pre-training allows to exploit very large quantities of
mostly unlabeled examples (such as documents, images, and videos from
the web). The learned representations capture the salient factors
of variation (and invariances) implicitly present in the data, and
can be exploited in the context of several supervised learning tasks
(multi-task learning, self-taught learning, semi-supervised learning).
See Yoshua Bengio’s research page ,
and these few papers, to learn a bit more:
|