Hyper Emotional Skin is a fusion of art and technology, uniting the gorgeous 3D design work of Ines Alpha and my expertise in machine learning and sound design. The result is an interactive 3D mask that captures and reacts in real time to the emotions of its wearer, allowing an exploration of a visual and auditory manifestation of human emotions.
For this work, I designed and trained a machine learning model to be capable of recognizing five emotions: happiness, surprise, sadness, anger, and a neutral state. A key technical aspect was optimizing this model to function on an embedded system while maintaining accurate emotion detection.
Ines Alpha created five unique versions of the mask, each corresponding to a different emotion. These beautiful 3D forms dynamically evolve in real time in response to the emotions detected by the machine learning model.
The visual part of the experience is complemented by the sound design. For each emotion, I created a corresponding sound loop. For instance, the Shepard sound illusion was used to create a sensation of infinite rise for happiness, with its inverse used for sadness. These loops can overlay each other, facilitating a smooth transition between emotions as predicted by the machine learning model.
During the development process, two interaction modes were contemplated: the "muscular" mode, where the mask reacts almost instantly to facial expressions, and the "diffuse" mode, where the mask slowly adapts to the wearer's predominant emotion with more inertia. We opted for the first one which we found more interesting.
Hyper Emotional Skin also prompts philosophical inquiry into the materialization of emotions. The mask, reacting in a manner akin to muscles, serves as a physical extension of the body that expresses emotions. This leads to questions such as: can we gain control over this extension? Does this device influence the wearer's emotions? Furthermore, it introduces the idea that the emotions conveyed are those we choose to display (since the machine learning model is based on facial expressions), and not necessarily those we truly feel.
You can try the filter here!