zbMATH — the first resource for mathematics

A hybrid approach of NN and HMM for facial emotion classification. (English) Zbl 1010.68911
Summary: Neural networks (NNs) are often combined with Hidden Markov Models (HMMs) in speech recognition for achieving superior performance. In this paper, this hybrid approach is employed in facial emotion classification. Gabor wavelets are employed to extract features from difference images obtained by subtracting the first frame showing a frontal face from the current frame. The NN, which takes the form of Multilayer perceptron (MLP), is used to classify the feature vector into different states of a HMM of a certain emotion sequence, i.e., neutral, intermediate and peak. In addition to using 1-0 as targets for the NN, a heuristic strategy of assigning variable targets 1-\(x\)-0 has also been applied. After training, we interpret the output values of the NN as the posterior of the HMM state and directly apply the Viterbi algorithm to these values to estimate the best state path. The experiments show that with variable targets for the NN, the HMM gives better results than that with 1-0 targets. The best HMM results are obtained for \(x\)=0.8 in 1-\(x\)-0.
68U99 Computing methodologies and applications
68T10 Pattern recognition, speech recognition
68U10 Computing methodologies for image processing
68T45 Machine vision and scene understanding
68T05 Learning and adaptive systems in artificial intelligence
Full Text: DOI