| Summary: Emotion recognition has gained significant attention in various applications to develop reasoning and interaction between people. However, emotion recognition problems are very difficult two solve with classical methods, because emotions can change quickly because of complex external stimuli. In this thesis, both novel angle amplitude transform (AAT) methods and fusion methods for combining different types of input data were proposed for multimodal emotion recognition. In the first part of the thesis, AAT was applied to Electroencephalography (EEG) and Electrooculogram (EOG) signals. The obtained angle amplitude graph (AAG) images were then combined with two different approaches (FLF1 and FLF2) at the feature level, and classified with support vector machines (SVM). In the second part, EEG signals and facial images were used as input data. Also, two new AAT methods were applied to the EEG signals. Among all face images, the most unique one is determined automatically. Then, the two different input data were combined with two newly proposed approaches (Sensor level fusion (SLF) and FLF3). Finally, classification was performed with both k-nearest neighbor (kNN) and SVM. In addition to all these, a novel multimodal dataset KMED was created for multimodal emotion recognition studies. The performances of the proposed methods were tested on both KMED and DEAP datasets.
Key Words: Electroencephalography, Electrooculogram, Facial images, Multimodal emotion recognition, Angle amplitude transformation, Angle amplitude graph images, Sensor level fusion, Feature level fusion. |
