r6 - 27 Jul 2011 - 17:10:28 - EmmanuelBengio?You are here: TWiki >  Public Web  > EmotionClassification

Emotion Classification

This page intends to resume the work that has been done to classify emotions from data

Facial Emotion


The datasets used are a combination of the CMU/Kanade database (10734 examples), the MMI Faces database (9736 examples), JACFEE(130 examples), Radboud face DB(968 examples, only front was taken), and Labeled Faces in the Wild (13190 examples, no targets) alongside facetubes(Youtube Faces, ~1/2 million examples) for unsupervised tasks.

Except for LFW, each dataset contained examples with either FACS or an emotion tag.

Emotions are among the following: neutral, sad, disgust, anger, surprise, fear, contempt, happy.

Images from these datasets were cropped with OpenCV?'s face detector, then scaled down to either 32x32 or 48x48

Error measure

The results on this page were measured as follows.

For emotions, the target is a one-hot vector, the prediction should be a probability vector, the error rate is the percentage of differing argmax'es over a given set of these vectors.

For facs, the target is a binary vector, the predicted vector is a real valued vector thresholded at .5, the error rate is the percentage of per bit error over the set (both false positives and false negatives count as errors).



Model Facs error Emotion error Demo performance
MLP ~5.6% 17.5% miserable


A few different combinations were tried, input to emotions, input to hidden to emotions, input to hidden to facs, etc. What worked best is the following:

Input is a flattened 32x32 vector, FACS target is a 25 unit bit vector, emotion target is a 8 unit one-hot vector. The loss function is the sum of the binary crossentropy of the FACS and the categorical crossentropy with no coefficient between the two(seems to be what works best).

   | softsign
hidden units
   | sigmoid  |
   V          | linear
  FACS        |
   | linear   |
Intermediate ditto(B)
 emotion(A)   |
   |         /
   V        /
   V softmax(A+B)
Code is available at /u/bengioe/public/experiments/face_emotion/mlp.py . Best hyperparameters for this code are nhidden=256,learning rate=1e-4, no L1 or L2 regularization.
Edit | WYSIWYG | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r6 < r5 < r4 < r3 < r2 | More topic actions
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback