Simultaneous evolutionary neural network based automated video based facial expression analysis

  • Authors

    • S. Jeyalaksshmi
    • S. Prasanna
    2017-12-21
    https://doi.org/10.14419/ijet.v7i1.1.9211
  • Human computer interaction (HCI), facial expressions and emotions, simultaneous evolutionary neural network (SENN), adaboost, geometric features, local binary pattern (LBP), classification.

  • Abstract

    In real life scenario, facial expressions and emotions are nothing but responses to the external and internal events of human being. In Human Computer Interaction (HCI), recognition of end user’s expressions and emotions from the video streaming plays very important role. In such systems it is required to track the dynamic changes in human face movements quickly in order to deliver the required response system. In real time applications, this Facial Expression Recognition (FER) is very helpful like physical fatigue detection based on facial detection and expressions such as driver fatigue detection in order to prevent the accidents on road. Face expression based physical fatigue analysis or detection is out of scope of this work, but this work proposed a Simultaneous Evolutionary Neural Network (SENN) classification scheme is proposed for recognising human emotion or expression. In this work, at first, automatically detects and tracks facial landmarks in videos, and face is detected by using enhanced adaboost algorithm with haar features. Then, in order to describe facial expression modifications, geometric features are taken out and the Local Binary Pattern (LBP) is extracted to improve the detection accuracy and it has a much lower-dimensional size. With the aim of examining the temporal facial expression modifications, we apply SENN probabilistic classifiers, which examine the facial expressions in individual frames, and after that promulgate the likelihoods during the course of the video to take the temporal features of facial expressions such as glad, sad, anger, and fear feelings. The experimental results show that the performance of proposed SENN scheme is attained better results compared than existing recognition schemes like Time-Delay Neural Network with Support Vector Regression (TDNN-SVR) and SVR. 

  • References

    1. [1] Samad R & Sawada H, “Edge-based Facial Feature Extraction Using Gabor Wavelet and Convolution Filtersâ€, MVA, pp.430-433, (2011).

      [2] Thai LH, Nguyen NDT & Hai TS, “A facial expression classification system integrating canny, principal component analysis and artificial neural networkâ€, International Journal of Machine Learning and Computing, Vol.1, No.4, pp.388-393, (2011).

      [3] Sisodia P, Akhilesh V & Sachin K, “Human Facial Expression Recognition using Gabor Filter Bank with Minimum Number of Feature Vectorsâ€, International Journal of Applied Information Systems, Vol.5, No. 9, pp.9-13, (2013).

      [4] Jun Y & Zengfu W, “A Video-Based Facial Motion Tracking and Expression Recognition Systemâ€, Springer Science Business Media New York, (2016).

      [5] Dey A, “Contour based Procedure for Face Detection and Tracking from Videoâ€, 3rd Int'I Conf. on Recent Advances in Information Technology I RAIT, (2016).

      [6] Le DN, Van Chung L & Nguyen GN, “Performance evaluation of video-based face recognition approaches for online video contextual advertisement user-oriented systemâ€, Information Systems Design and Intelligent Applications, pp.287-295, (2016).

      [7] Haque MA, Irani R, Nasrollahi K & Moeslund TB, “Facial video-based detection of physical fatigue for maximal muscle activityâ€, IET Computer Vision, (2016).

      [8] Hajati F, Tavakolian M, Gheisari S, Gao Y & Mian AS, “Dynamic Texture Comparison Using Derivative Sparse Representation: Application to Video-Based Face Recognitionâ€, IEEE Transactions on Human-Machine Systems, (2017).

      [9] Chen J, Chen Z, Chi Z & Fu H, “Facial expression recognition in video with multiple feature fusionâ€, IEEE Transactions on Affective Computing, (2016).

      [10] Meng H, Bianchi-Berthouze N, Deng Y, Cheng J & Cosmas JP, “Time-delay neural network for continuous emotional dimension prediction from facial expression sequencesâ€, IEEE transactions on cybernetics, Vol.46, No.4, pp.916-929, (2016).

      [11] Chiranjeevi P, Gopalakrishnan V & Moogi P, “Neutral face classification using personalized appearance models for fast and robust emotion detectionâ€, IEEE Transactions on Image Processing, Vol.24, No.9, pp.2701-2711, (2015).

      [12] Hayat M & Bennamoun M, “An automatic framework for textured 3D video-based facial expression recognitionâ€, IEEE Transactions on Affective Computing, Vol.5, No.3, pp.301-313, (2014).

      [13] El Meguid MKA & Levine MD, “Fully Automated Recognition of Spontaneous Facial Expressions in Videos Using Random Forest Classifiersâ€, Affective Computing, Vol.5, pp.418-431, (2014).

      [14] Nicolaou MA, Gunes H & Pantic M, “Output-associative RVM regression for dimensional and continuous emotion predictionâ€, Image Vis. Comput. Best Autom. Face Gesture Recognit., Vol.30, No.3, pp.186–196, (2012).

      [15] Eyben F, Petridis S, Schuller B, Tzimiropoulos G, Zafeiriou S & Pantic M, “Audiovisual classification of vocal outbursts in human conversation using long-short-term memory networksâ€, IEEE International Conference on Acoustics, Speech and Signal Processing, pp.5844-5847, (2011).

      [16] Nicolaou MA, Gunes H & Pantic M, “Continuous prediction of spontaneous affect from multiple cues and modalities in valencearousal spaceâ€, IEEE Trans. Affect. Comput., Vol.2, No2, pp. 92–105, (2011).

      [17] Christopher ACK, Barrett F, Gur RE, Gur RC & Verma R, “Computerized Measurement of Facial Expression of Emotions in Schizophreniaâ€, Journal of Neuroscience Methods, (2007).

      [18] Cootes TF, Edwards GJ & Taylor CJ, “Active Appearance Modelsâ€, IEEE Trans on PAMI, Vol.23, No.6, pp.681–685, (2001).

      [19] Yang MH, Kriegman DJ & Ahuja N, “Detecting Faces in Images: A Surveyâ€, IEEE Trans on PAMI, Vol.24, No.1, pp.34–58, (2002).

      [20] Viola P & Jones M, “Robust Real-time Object Detectionâ€, International Journal of Computer Vision, Vol.57, No.2, pp.137–154, (2004).

      [21] Li SZ & Zhang Z, “FloatBoost Learning and Statistical Face Detectionâ€, IEEE Trans on PAMI, Vol.26, No.9, pp.1112–1123, (2004).

      [22] Wang P & Ji Q, “Learning Discriminant Features for Multi-View Face and Eye Detectionâ€, Computer Vision and Image Understanding, Vol.105, No.2, pp.99–111, (2007).

      [23] Blei DM, Ng AY & Jordan MI, “Latent dirichlet allocationâ€, J. Mach. Learn.Res., Vol.3, pp.993–1022, (2003).

      [24] Rocha M, Cortez P & Neves J, “Simultaneous evolution of neural network topologies and weights for classification and regressionâ€, International Work-Conference on Artificial Neural Networks, pp. 59-66, (2005).

      [25] Riedmiller M, “Advanced supervised learning in multi-layer perceptrons-from backpropagation to adaptive learning algorithmsâ€, Computer Standards & Interfaces, Vol.16, No.3, pp. 265-278, (1994).

      [26] Andreasen, NC, Scale for the Assessment of Negative Symptoms (SANS), Iowa City: University of Iowa, (1984).

      [27] Yeasin M, Bullot B & Sharma R, “From facial expression to level of interest: a spatio-temporal approachâ€, IEEE Conference on Computer Vision and Pattern Recognition, (2004).

      [28] Wolf L, Hassner T & Maoz I, “Face recognition in unconstrained videos with matched background similarityâ€, Proc. IEEE Comput. Soc.Conf. Comput. Vis. Pattern Recognit., pp.529–534, (2011).

  • Downloads

  • How to Cite

    Jeyalaksshmi, S., & Prasanna, S. (2017). Simultaneous evolutionary neural network based automated video based facial expression analysis. International Journal of Engineering & Technology, 7(1.1), 125-132. https://doi.org/10.14419/ijet.v7i1.1.9211

    Received date: 2018-01-19

    Accepted date: 2018-01-19

    Published date: 2017-12-21