Introduction to EEG- and Speech-Based Emotion Recognition

Priyanka A. Abhang, Bharti W. Gawali, Suresh C. Mehrotra

• Publisher: Academic Press
• Year: 2016
• ISBN:
  9780128045312
• Topic: Medicina
• Subtopic: Fisiología

Introduction to EEG- and Speech-Based Emotion Recognition Methods examines the background, methods, and utility of using electroencephalograms (EEGs) to detect and recognize different emotions. By incorporating these methods in brain-computer interface (BCI), we can achieve more natural, efficient communication between humans and computers. This book discusses how emotional states can be recognized in EEG images, and how this is useful for BCI applications. EEG and speech processing methods are explored, as are the technological basics of how to operate and record EEGs. Finally, the authors include information on EEG-based emotion recognition, classification, and a proposed EEG/speech fusion method for how to most accurately detect emotional states in EEG recordings.

• Provides detailed insight on the science of emotion and the brain signals underlying this phenomenon
• Examines emotions as a multimodal entity, utilizing a bimodal emotion recognition system of EEG and speech data
• Details the implementation of techniques used for acquiring as well as analyzing EEG and speech signals for emotion recognition

Chapter 1

Introduction to Emotion, Electroencephalography, and Speech Processing

Abstract

This chapter briefly discusses the need for understanding and processing emotions through the brain and speech. It begins with information related to representation of emotions, and then describes the physiology of the brain and auditory system, their different parts, and their properties. Techniques to acquire signals produced by different parts of the brain and auditory system are briefly discussed. Finally, the organization of the book is summarized.

Keywords

BCI; Brain; EEG speech; Emotion recognition; Lobes; Speech mechanisms; Speech production

Outline

1.1 Introduction 1

1.2 Brain Physiology 3

1.2.1 Major Brain Areas 3

1.3 Lobes of the Brain and Their Functions 6

1.3.1 The Frontal Lobe 7

1.3.2 The Parietal Lobe 7

1.3.3 The Temporal Lobe 7

1.3.4 The Occipital Lobe 8

1.4 Electroencephalography 8

1.5 Human Auditory System 9

1.5.1 Speech Production Mechanism 9

1.6 Speech Processing 13

1.6.1 Speech Emotion Recognition 14

1.7 Organization of the Book 15

1.8 Conclusion 16

References 16

1.1. Introduction

In a very real sense, we have two minds, one that thinks and one that feels.

Daniel Goleman, Emotional Intelligence.

Emotions are intrinsically connected to the way that people interact with each other. Emotion constitutes a major influence for determining human behaviors. A human being can read the emotional state of another human, and behave in the best way to improve their communication at that moment. This is because emotions can be recognized through words, voice intonation, facial expression, and body language.

Emotions have been studied in scientific disciplines such as physiology, psychology, speech science, neuroscience, psychiatry, communication, and so on. As a result, distinctive perspectives on the concept of emotions have emerged, appropriate to the complexity and variety of emotions. It is important to see these different perspectives as complementary to each other.

In humans, emotions fundamentally involve physiological arousal, expressive behaviors, and conscious experience. Thus in that respect there are three views of emotions.

• Psychological (what one is thinking).

• Physiological (what one's body is doing).

• Expressive (how one reacts) in nature.

It is thought that emotions are predictable, and are settled in the different regions in the brain depending on what emotion is invoked. An emotional reaction can be separated into three major categories: behavioral, automatic, and hormonal.¹

In psychology, expression of emotion is viewed as a reaction to stimuli that involves characteristic physiological changes. According to psychology, an emotion is seen as a disturbance in the homeostatic baseline. Based on these changes, the properties of emotions can be represented as a three-dimensional construct. Essential dimensions of emotional states are measured by the features of activation.

• Arousal: measured as an intensity.

• Affect: valence of pressure, measured as positive or negative feeling after emotion perception.

• Power (control): measured as dominance or submissiveness in emotion expression.

Thus the psychology of emotions can be viewed as a complex experience of consciousness (psychology), bodily sensation (physiology), and behavior (active speech). Orientational dimensions of emotional states are captured by the features of activation, affection, and power.

The emotions generally represent a synthesis of subjective experience, expressive behavior, and neurochemical activity. In the physiology of emotion production mechanisms, it has been found that the nervous system is stimulated by the expression of high-arousal emotions such as anger, happiness, and fear. This phenomenon causes an increased heart rate, higher blood pressure, changes in respiratory pattern, greater subglottal air pressure in the lungs, and dryness in the mouth.²

Emotions are generally expressed in positive and negative ways. The positive emotions such as happiness, excitement, joy, etc. are pleasant and are seen as constructive in an individual, whereas negative emotions such as sadness, anger, fear, etc. are considered unpleasant and may be considered to be destructive for an individual.

According to Robert Plutchik,³ any emotion is based upon one of six primary emotions. Happiness, sadness, anger, disgust, fear, and surprise are considered as the main or basic emotions by most researchers and are known as archetypal emotions.⁴

1. Happiness is the emotion that expresses various degrees of positive feelings, ranging from satisfaction to extreme joy.

2. Sadness is the emotion that expresses a state of loss or difficulty. Sadness causes individuals to be slow at processing information.

3. Anger is the emotion that expresses dislike or opposition toward a person or thing that is causing aversion. Anger is sometimes displayed through sudden and overt aggressive acts.

4. Disgust is the emotion that expresses a reaction to things that are considered dirty, revolting, contagious, contaminated, or inedible. Disgust is associated with a distinct facial expression and a drop in heart rate.

5. Fear is the emotional reaction to an actual and specific source of danger. Fear is often confused with anxiety, which is an emotion that is often exaggerated and experienced even when the source of danger is not present or tangible.

6. Surprise is the emotion that arises when an individual comes across an unanticipated situation. A surprise emotion can be a positive, neutral, or negative experience. A human being can understand the emotional state of another human being and behave in the best manner to improve the communication in a certain situation. This is because emotions can be recognized through various modalities such as words, voice intonation, facial expression, body language and by brain signals.⁵

1.2. Brain Physiology

The brain is the central controlling organ of the human being. Various scientific studies have proved that some regions of the brain are involved in thinking of emotions, responding to extreme emotional stimuli, and viewing emotional situations. Nearly all vital activities necessary for survival, as well as all emotions, originate...