Computational Paralinguistics: Emotion, Affect and Personality in Speech

Book Preface

It might be safe to claim that 20 years ago, neither the term ‘computational paralinguistics’ nor the field it denotes existed. Some 10 years ago, the term did not yet exist either. However, in hindsight, the field had begun to exist if we think of the first steps towards the automatic processing of emotions in speech in the mid-1990s. For example, Picard’s book on Affective Computing published in 1997, and the International Speech Communication Association (ISCA) workshop on emotion and speech in 2000, just to mention some of the many topics and events related to and belonging to computational paralinguistics. The term ‘paralinguistics’ had already been coined in the 1950s – with different broad or narrow denotations; we will try and sketch this history in Part I of this book. Yet, in the realm of ‘hard core’ automatic (i.e., computational) processing of speech, the topic was still not fully acknowledged; as one of our colleagues said: ‘Emotion recognition, that’s esoterics with HMMs.’

Today, it might be safe to claim that computational paralinguistics has been established as a discipline in its own right – although surprisingly, not the term itself. It is only natural that, as a new and still somewhat exotic field, it has to cope with prejudices on the one hand, and unrealistic promises on the other hand.

This book represents the first attempt towards a unified overview of the field, its extremely ramified and diverse ‘genealogy’, its methodology, and the state of the art. ‘Computational paralinguistics’ is not an established subject that can be studied, and this fact is mirrored in the ‘scientific CVs’ of both authors. B.S. studied electrical engineering and information technology. However, his doctoral thesis dealt with one aspect of computational paralinguistics: the automatic recognition of human emotion in speech. During his habilitation period he broadened the scope of his work to ‘intelligent audio analysis’ – dealing with quite a number of further paralinguistic aspects, including those found in sung language and many other audio processing problems such as emotion in music and general sound. A.B. started within philology, came from diachronic phonology to phonetics in general to prosody in particular, and via prosody in the interface for and within syntax and semantics to the automatic processing of acted and very soon naturalistic emotions – realising, moreover, that he had been dealing with different topics for a long time that all can be located within (computational) paralinguistics. At the time of finalisation of the manuscript, A.B. was a professor in computer science.