Romain Belmonte, Benjamin Allaert

Face Analysis Under Uncontrolled Conditions

From Face Detection to Expression Recognition

• Gebundenes Buch

Produktbeschreibung

Face analysis is essential for a large number of applications such as human-computer interaction or multimedia (e.g. content indexing and retrieval). Although many approaches are under investigation, performance under uncontrolled conditions is still not satisfactory. The variations that impact facial appearance (e.g. pose, expression, illumination, occlusion, motion blur) make it a difficult problem to solve. This book describes the progress towards this goal, from a core building block - landmark detection - to the higher level of micro and macro expression recognition. Specifically, the book addresses the modeling of temporal information to coincide with the dynamic nature of the face. It also includes a benchmark of recent solutions along with details about the acquisition of a dataset for such tasks.

Produktdetails

• Verlag: Wiley
• Seitenzahl: 320
• Erscheinungstermin: 17. Oktober 2022
• Englisch
• Abmessung: 240mm x 161mm x 22mm
• Gewicht: 648g
• ISBN-13: 9781789451115
• ISBN-10: 1789451116
• Artikelnr.: 65605233

Autorenporträt

Romain Belmonte is a postdoctoral researcher in computer science at the University of Lille, France. His current research interests include computer vision, deep learning and human behavior analysis. Benjamin Allaert is an associate professor at the IMT Nord Europe, the leading institute of technology in France. His research focuses on the development of digital simulations, deep learning and decision models to help humans better interact with their environment.

Inhaltsangabe

Preface xi
Romain BELMONTE and Benjamin ALLAERT
Part 1. Facial Landmark Detection 1
Introduction to Part 1 3
Romain BELMONTE, Pierre TIRILLY, IoanMarius BILASCO, Nacim IHADDADENE and
Chaabane DJERABA
Chapter 1. Facial Landmark Detection 13
Romain BELMONTE, Pierre TIRILLY, IoanMarius BILASCO, Nacim IHADDADENE and
Chaabane DJERABA
1.1. Facial landmark detection in still images 14
1.1.1.Generativeapproaches 14
1.1.2.Discriminative approaches 18
1.1.3.Deep learningapproaches 24
1.1.4.Handlingchallenges 34
1.1.5.Summary 40
1.2.Extendingfacial landmarkdetectionto videos 41
1.2.1.Trackingby detection 41
1.2.2.Box, landmarkand pose tracking 43
1.2.3.Adaptive approaches 45
1.2.4. Joint approaches 46
1.2.5. Temporal constrained approaches 47
1.2.6.Summary 49
1.3.Discussion 50
1.4.References 52
Chapter 2. Effectiveness of Facial Landmark Detection 67
Romain BELMONTE, Pierre TIRILLY, IoanMarius BILASCO, Nacim IHADDADENE and
Chaabane DJERABA
2.1.Overview 68
2.2.Datasets and evaluationmetrics 69
2.2.1. Image and videodatasets 69
2.2.2. Face preprocessing and data augmentation 73
2.2.3.Evaluationmetrics 75
2.2.4.Summary 77
2.3. Image andvideobenchmarks 77
2.3.1. Compiled results on 300W 77
2.3.2. Compiled results on 300VW 79
2.4.Cross-dataset benchmark 80
2.4.1.Evaluationprotocol 80
2.4.2.Comparisonof selected approaches 82
2.5.Discussion 86
2.6.References 88
Chapter 3. Facial Landmark Detection with Spatio-temporal Modeling 93
Romain BELMONTE, Pierre TIRILLY, IoanMarius BILASCO, Nacim IHADDADENE and
Chaabane DJERABA
3.1.Overview 94
3.2.Spatio-temporalmodelingreview 95
3.2.1.Hand-craftedapproaches 95
3.2.2.Deep learningapproaches 97
3.2.3.Summary 103
3.3.Architecturedesign 104
3.3.1. Coordinate regression networks 104
3.3.2.Heatmapregressionnetworks 106
3.4.Experiments 107
3.4.1.Datasets andevaluationprotocols 107
3.4.2. Implementationdetails 108
3.4.3.EvaluationonSNaP-2DFe 109
3.4.4. Evaluation on 300VW 111
3.4.5.Comparisonwith existingmodels 112
3.4.6. Qualitative results 112
3.4.7.Propertiesof the networks 114
3.5.Design investigations 114
3.5.1.Encoder-decoder 115
3.5.2. Complementarity between spatial and temporal information 117
3.5.3. Complementarity between local and global motion 119
3.6.Discussion 122
3.7.References 123
Conclusion to Part 1 133
Romain BELMONTE, Pierre TIRILLY, IoanMarius BILASCO, Nacim IHADDADENE and
Chaabane DJERABA
Part 2. Facial Expression Analysis 147
Introduction to Part 2 149
Benjamin ALLAERT, IoanMarius BILASCO and Chaabane DJERABA
Chapter 4. Extraction of Facial Features 157
Benjamin ALLAERT, IoanMarius BILASCO and Chaabane DJERABA
4.1. Introduction 157
4.2.Face detection 158
4.2.1.Point-of-interestdetectionalgorithms 160
4.2.2.Face alignment approaches 162
4.2.3.Synthesis 166
4.3.Face normalization 166
4.3.1.Dealingwith headpose variations 167
4.3.2.Dealingwith facial occlusions 170
4.3.3.Synthesis 172
4.4.Extractionof visual features 172
4.4.1.Facial appearancefeatures 172
4.4.2.Facial geometric features 174
4.4.3. Facial dynamics features 175
4.4.4.Facial segmentationmodels 177
4.4.5.Synthesis 179
4.5. Learning methods 179
4.5.1.Classification versus regression 180
4.5.2.Fusionmodel 182
4.5.3.Synthesis 184
4.6.Conclusion 185
4.7.References 186
Chapter 5. Facial Expression Modeling 191
Benjamin ALLAERT, IoanMarius BILASCO and Chaabane DJERABA
5.1. Introduction 191
5.2.Modelingof the affective state 192
5.2.1.Categoricalmodeling 192
5.2.2.Dimensionalmodeling 194
5.2.3.Synthesis 196
5.3. The challenges of facial expression recognition 197
5.3.1. The variation of the intensity of the expressions 197
5.3.2.Variationof facialmovement 199
5.3.3.Synthesis 200
5.4.The learningdatabases 201
5.4.1. Improvementof learningdata 201
5.4.2. Comparison of learning databases 203
5.4.3.Synthesis 205
5.5. Invariance to facial expression intensities 206
5.5.1.Macro-expression 206
5.5.2.Micro-expression 208
5.5.3.Synthesis 209
5.6. Invarianceto facialmovements 211
5.6.1. Pose variations (PV) and large displacements (LD) 211
5.6.2.Synthesis 214
5.7.Conclusion 215
5.8.References 216
Chapter 6. Facial Motion Characteristics 223
Benjamin ALLAERT, IoanMarius BILASCO and Chaabane DJERABA
6.1. Introduction 223
6.2.Characteristics of the facialmovement 225
6.2.1. Local constraint of magnitude and direction 226
6.2.2. Local constraint of the motion distribution 228
6.2.3.Motionpropagationconstraint 230
6.3.LMP 232
6.3.1. Local consistency of the movement 233
6.3.2.Consistencyof local distribution 236
6.3.3. Coherence in the propagationof themovement 238
6.4.Conclusion 241
6.5.References 242
Chapter 7. Micro- and Macro-Expression Analysis 243
Benjamin ALLAERT, IoanMarius BILASCO and Chaabane DJERABA
7.1. Introduction 243
7.2. Definition of a facial segmentation model 244
7.3.Feature vector construction 247
7.3.1.Motionfeaturesvector 247
7.3.2.Geometric featuresvector 248
7.3.3.Features fusion 249
7.4. Recognition process 250
7.5. Evaluation on micro- and macro-expressions 251
7.5.1.Learningdatabases 252
7.5.2. Micro-expression recognition 253
7.5.3. Macro-expressions recognition 255
7.5.4. Synthesis of experiments on micro- and macro-expressions 258
7.6. Same expression with different intensities 260
7.6.1.Data preparation 260
7.6.2.Fractional time analysis 263
7.6.3.Analysis on a different time frame 264
7.6.4. Synthesis of experiments on activation segments 265
7.7.Conclusion 265
7.8.References 266
Chapter 8. Towards Adaptation to Head Pose Variations 271
Benjamin ALLAERT, IoanMarius BILASCO and Chaabane DJERABA
8.1. Introduction 271
8.2.Learningdatabase challenges 273
8.3. Innovative acquisition system (SNaP-2DFe) 274
8.4. Evaluation of face normalization methods 276
8.4.1. Does the normalization preserve the facial geometry? 277
8.4.2. Does normalization preserve facial expressions? 280
8.5.Conclusion 283
8.6.References 284
Conclusion to Part 2 287
Benjamin ALLAERT, IoanMarius BILASCO and Chaabane DJERABA
List of Authors 293
Index 295