# Conditional Density Estimation
with Extreme Observations

## Julie Carreau and Yoshua Bengio

My project is to develop methods of density estimation based on
subexponential distributions such as those found in the extreme value
theory. The goal is to model an unknown, asymmetric, fat-tailed density
(eventually a conditional density or the noise in a regression setting).
Current methods in robust statistics downweight the observations in the
tails symmetrically and are therefore biased in the case of asymmetry.

In order to make as few distributional assumptions as possible, I
would like to use a non-parametric method. However, the estimation of
the density becomes more and more difficult as we try to model further
in the tail of the density. A compromise would be to use a
non-parametric method to model the center of the density and a
parametric method to model the tail of the density, above a given
threshold.

The estimation of the distribution in the upper tail could be based on
the following theory. Distribution with Pareto-type tails are defined
by 1 - F(x) = x^{-alpha} L(x), where L(x) is a slowly varying
function and alpha controls the thickness of the tail. In order to
estimate alpha, one can use the technics from extreme value theory
(EVT).

The EVT is based on the modelling of maxima and minima
behaviour. The limit laws for maxima (minima) of random variables are one
of three types: Frechet, Gumbel or Weibull. The estimation of alpha is
based on sample maxima. Another possibility is the generalised Pareto
distribution (GPD) which models excesses over a given threshold.

The difficulty with the GPD is to set an appropriate threshold. One
possibility is to use order statistics. In the conditional setting, I
propose to use regression quantile. The level of quantile is chosen
through cross-validation. The GPD allows then to model parametrically
the tails of the density.