The world’s first open-access database on responses to visual, auditory and haptic stimuli includes participants’ reactions to hearing a baby cry and to seeing work by artists including Piet Mondrian

Researchers from the Sussex Computer Human Interaction (SCHI) Lab at the University of Sussex have created the world’s first open-access database on participants’ responses to visual, auditory and haptic stimuli.

The work, including compositions of Mozart and Bach, works by Matisse and Mondrian along with sounds of a baby crying and a new set of mid-air haptic stimulations, has been published in the journal Scientific Data.

Participants were invited to sit comfortably in front of a computer screen with recording electrodes attached to their index and ring finger to measure the skin conductance of light electric current. They were then exposed to ten standardized sounds, ten new instrumental extracts from various compositions, ten standardized images, ten new abstract pictures and ten haptic stimuli delivered by a mid-air haptic device with their responses to each stimuli recorded.

Dr Emanuela Maggioni, co-author and Research Fellow at the SCHI Lab within the Department of Informatics, said: “Our database is the first that compares the emotions evoked by stimuli from multiple senses across the same group of participants. It is open for everyone that wants to find validated stimuli to evoke specific emotions.”

The SCHI Lab team hopes that by publishing the method for measuring the stimuli other academics around the world will help to build up the database, which could then be used for a wide range of applications leading to new developments in computer science, neuroscience, robotics and psychology.

The main author Dr Elia Gatti, now Postdoctoral Research Scientist at Oculus Research, sees the opportunity to guide industry-led research around multisensory experience design, based on this newly established dataset.

Dr Marianna Obrist, Head of the SCHI Lab, added: “We encourage other scientists to expand the database themselves, still maintaining the same experimental design. In the near future we aim to expand the database using other senses, such as smell, to evoke emotions.”

The study was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 programme

The researchers say that the research not only conbtributes to a more meaningful design of multisensory technologies but has also developed a methodological innovation that combines self-reporting and physiological measurements of participants' responses to stimuli.

This database is the first of its kind to compare how individuals respond to a range of different stimuli while also offering a rare dataset which shows both the individuals’ perceived response and their physical response to a stimuli.

To read more about the research paper Emotional ratings and skin conductance response to visual, auditory and haptic stimuli by Elia Gatti, Elena Calzolari, Emanuela Maggioni, and Marianna Obrist visit here.

The SCHI Lab specialises in researching Human-Computer Interaction (HCI), an area in which research on multisensory experiences makes a difference on how we design and interact with technology in the future. The interdisciplinary team, which is supported by the SenseX project funded by the European Research Council, explores tactile, gustatory, and olfactory experiences as novel interaction modalities.

For more information visit here.