Designing technologies that interpret your mood from your skin

Smart devices that measure electrical signals from your skin have the potential to tell you about your stress levels, help your sports performances and allow you to track your emotions.

An international team of researchers from Sweden and the UK have developed an innovative way of interpreting biological signals produced by the conductance of our skin. Using data obtained using a Philips wrist-worn wearable sensor device that also include an accelerometer to measure movement, the researchers’ system displays information in the form of colourful spiral graphics in real time on a smart phone, as well as a recording of data, for the wearer to interpret and reflect on.

Skin conductance is a measure of how much someone sweats, indicating their emotional reactions as well as physical reactions and is the basis for technologies such as lie detectors.

The prototype visualisation system called ‘Affective Health’, was developed by experts in Human-Computer Interaction looking into how new and emerging computer technologies, such as those using data extracted from sensors on the skin, can be used to design engaging visualisations on smart phones.

Dr Pedro Sanches, senior researcher at KTH – Royal Institute of Technology Stockholm, and lead researcher on the study said:

“Our bodies produce a wide range of signals that can be measured. Many useful devices that measure these signals, which we call biodata, have proliferated over the years – such as heart-rate monitors for sports. However, there are other areas of biodata that are yet to be fully developed – such as skin conductance or perspiration levels. Making sense of these kinds of biodata is not easy. People are unfamiliar with this kind of information and it is not clear how people would want to use it, or interact with devices that present this biodata.”

A study group of 23 people were given the Affective Health prototype to use for a month. Importantly, the researchers deliberately did not tell the participants what the devices were useful for. Instead, they gave guidelines that Affective Health could collect information relating to both physical and emotional reactions, how increased sweating increased the conductivity, and how this was represented by different colours. Participants were left to decide the best ways of using the technology.

The researchers found that this open design stage of the study, without providing pre-specified uses, led to some participants using the system as a tool to measure, and help manage, their stress levels.

While others, including elite athletes, used the device to get information on their training and recovery regimes.

Other uses included logging information on their lives, and tracking emotions. But interestingly, few would use the technology for more than one purpose.

Prof Kristina Höök, of Royal Institute of Technology in Sweden said:

“We were surprised at how much the wearers’ initial categorisation of the system coloured how they used it. If they looked upon it as a sports tool, they did not even ‘see’ the data that spoke of stress or emotional reactions. If they looked upon it as an emotion measuring tool, they did not see the data that spoke of social processes or exertion due to sports activities.

“It was also interesting to see how some would avoid engaging with data that spoke against their ideas of their personality traits. One person looked upon himself as a calm person, but in the data, there were plenty of peaks. He just could not reconcile those with his impression of himself.”

Although the open design phase helped reveal several different practices Affective Health prototypes could be used for, the prototype lacked some of the functions needed to make it a good tool for a specific role – such as a sports training system, or as a stress management tool.

The researchers found the need for a second, more tailored, step in the design process to make devices specific to particular roles.

The research is part of the AffecTech: Personal Technologies for Affective Health, Innovative Training Network, which is funded through the European Union’s Horizon2020 programme and led by Professor Corina Sas of Lancaster University’s School of Computing and Communications.

Professor Sas, said: “The study revealed insights for designers of emerging wearable technologies, and in particular of biodata-based wearable devices. Design is crucially important to help users understand their bodily responses.

“Our results suggest the value of a two-step approach for designing new technologies that present bio-data that are unfamiliar to users. A deliberately open initial design stage allows users to develop their own ideas of how these kinds of products could be used. This is followed by a second step that tailors the functions of the device for specific activities, such as wellbeing, health or productivity.”

###

The study is outlined in the paper ‘Ambiguity as a Resource to inform Proto-Practices: The Case of Skin Conductance’ published in the ACM flagship journal Transactions in Computer-Human Interaction (ToCHI) and was due to be presented at the CHI 2020 conference.

This research also received funding from the Swedish Foundation for Strategic Research.

The paper’s authors are: Pedro Sanches and Kristina Höök, of the Royal Institute of Technology in Sweden, Corina Sas of Lancaster University, and Anna Stahl of RISE.