Random String of Emotions
2018
Emotion recognition software analyse our emotions by deconstructing our facial expressions into temporal segments that produce the expression, called Action Units (AU - developed by prof. Paul Ekman) and deducting them into certain percentages of six basic emotion, happy, sad, angry, surprised, scared and disgusted. In this video work the artist uses this decoding system to turn the process around. Here - instead of detecting Action Units - a computer is used to generate a random string of these AU’s. In this way complex and maybe even non-existing emotion expressions are discovered. These randomly formed expressions, played in random order, is then analysed by professional emotion recognition software again.
Two channel video installation. video snippets are played at a random order (200 fragments of ~ 10 sec), with corresponding emotion recognition software (Face Reader)readings. 4K 55 inch screen, 17inch screen, computer, polyethylene foam around the frame.
Credits:
Actress: Marina Miller Dessau
Choreography: Marjolein Vogels
Sound design: Arnoud Traa
Percussion: Onno Govaert
Technique: Joerg Drefs, Sylvian Vriens
Camera: Tim Nowitzki
make up: Rafaela Siqueira
Software: Facereader by Noldus
This project is generously funded by Creative Industry Fund (NL), EMARE grant. And supported by Werkleitz Gesellschaft. The emotion software is FaceReader, with the help of the Usability Lab, Communication and Multimedia Design, Amsterdam University of Applied Science
‘By having computers generate and interpret arbitrary mixtures of action units, the artist evokes the detachment of AI from genuine emotional intelligence, and implicitly asks how a more genuine connection could be made. More interestingly, by enlisting computers to help find faces of the kind that Ekman and Friesen dismissed, she posits a creative role for AI in the expansion of our expressive range. “In this way complex and maybe even non-existing emotion expressions are discovered,” she writes on her website. Machines can “help us to visually find an emotion we don’t have a word for yet.”
The search for new emotions seems even more apt today than it did when Vogelaar began her project in 2018. Even then, the unprecedented consequences of anthropogenic climate change were triggering feelings that psychologists and pundits struggled to name. (Proposed neologisms included solastalgia and tierratrauma.) Simultaneously many children were starting to lose their ability to read facial expressions in real life as a result of the amount of time spent staring at screens. This potent combination of novel emotional stimuli and routine social distancing is vastly amplified and complicated by the coronavirus pandemic (which heightens emotions through isolation and blocks out expression with facemasks). This is a time when we may need to surface new emotions, and might just have the psycho-perceptual space to adopt them in future social interactions.
A Facial Action Coding System for future emotions could be compared to a dictionary of neologisms to which definitions have not yet been assigned, potentially evoking meanings through the combination of sound on the ears and needs in the world. However there are also important differences that make speculative facial action codes both problematic and beguiling. In the first place, words seem to be more arbitrary than facial expressions. Although some linguists have plausibly claimed that spoken language evolved from ideophones that sound like the phenomena they denote (and may even give us a feel for the phenomena by the way they shape our mouths), most of what we say is unintelligible to people who haven’t memorized the vocabulary. In contrast, facial expressions may be innate, and might induce the feelings they communicate.’..
Jonathan Keats, The Pandemic Triggers Unprecedented Emotions, These Artists Are Discovering New Ways To Express Human Feelings, Forbes 2020
The search for new emotions seems even more apt today than it did when Vogelaar began her project in 2018. Even then, the unprecedented consequences of anthropogenic climate change were triggering feelings that psychologists and pundits struggled to name. (Proposed neologisms included solastalgia and tierratrauma.) Simultaneously many children were starting to lose their ability to read facial expressions in real life as a result of the amount of time spent staring at screens. This potent combination of novel emotional stimuli and routine social distancing is vastly amplified and complicated by the coronavirus pandemic (which heightens emotions through isolation and blocks out expression with facemasks). This is a time when we may need to surface new emotions, and might just have the psycho-perceptual space to adopt them in future social interactions.
A Facial Action Coding System for future emotions could be compared to a dictionary of neologisms to which definitions have not yet been assigned, potentially evoking meanings through the combination of sound on the ears and needs in the world. However there are also important differences that make speculative facial action codes both problematic and beguiling. In the first place, words seem to be more arbitrary than facial expressions. Although some linguists have plausibly claimed that spoken language evolved from ideophones that sound like the phenomena they denote (and may even give us a feel for the phenomena by the way they shape our mouths), most of what we say is unintelligible to people who haven’t memorized the vocabulary. In contrast, facial expressions may be innate, and might induce the feelings they communicate.’..
Jonathan Keats, The Pandemic Triggers Unprecedented Emotions, These Artists Are Discovering New Ways To Express Human Feelings, Forbes 2020
