Analysis of key face parts to detect emotional expression using a neural network model

Psychological studies on recognition of facial expression reported that local parts of a face image such as eyes or mouth are important to detect the facial expression. On the other hand, artificial neural network technology has progressed greatly in facial recognition. However, the neural mechanism behind the recognition remains to be unknown. The purpose of this study is to extract important features from the neural network model after learning emotions from facial expression and to clarify which face parts are key to detect emotional expression. First, we trained a 2-layered neural network model with backpropagation for recognition of 7 kinds of emotional faces. Then, we found more weighted input pixels for the recognition by tracing and accumulating the synaptic weights linearly from the output layer toward the hidden layer. By this method, we extracted the 6 face-image filters for each emotional expression. Using the face-image filters divided into 10 parts, we designed a new analytical method to evaluate which facial part or parts are important for the discrimination of emotional expression. From this analysis, it can be concluded that key parts are very different for each emotion. For example, nose and mouth are effective for happy smile while strained cheek for sad face.