A virtual reality game that integrates tactile experiences using biometric feedback | Techxplore

Over the past few decades, technological advances have enabled the development of increasingly sophisticated, immersive and realistic video games. One of the most noteworthy among these advances is virtual reality (VR), which allows users to experience games or other simulated environments as if they were actually navigating them, via the use of electronic wearable devices.

Most existing VR systems primarily focus on the sense of vision, using headsets that allow users to see what is happening in a game or in another simulated environment right before their eyes, rather than on a screen placed in front of them. While this can lead to highly engaging visual experiences, these experiences are not always matched by other types of sensory inputs.

Researchers at Nagoya University’s School of Informatics in Japan have recently created a new VR game that integrates immersive audiovisual experiences with tactile perception. This game, presented in a paper published in the Journal of Robotics, Networking and Artificial Life, uses a player’s biometric data to create a spherical object in the VR space that beats in alignment with his/her heart. The player can thus perceive the beating of his/her heart via this object visually, auditorily and tactually.

“Tactile sensation has a more significant effect on sensibility than audiovisual [perceptions],” the researchers wrote in their paper. “Therefore, if tactile and audiovisual senses are combined, we can create ‘virtual presence’ in the VR space.”

The goal of the VR game developed by the researchers is to break a series of targets by throwing a spherical object onto them. In the simulated environment, this spherical object is connected to a wire that appears to originate from where the player is standing.

Essentially, the researchers’ idea was to convey the feeling that the sphere is connected to a player’s heart and thus beats with its same rhythm, via a combination of visual, auditory and tactile stimuli. At the end of the game, after the player destroys all the targets, he is asked to cut the thread that connects him/her to the spherical object, using a pair of scissors that appear in the simulated environment and that can be handled via the controller. If the game produced the desired effect (i.e., eliciting the sense that the sphere is connected to the user’s heart), a player should have a strange feeling when cutting the thread or might even be hesitant to do so.

“In our system, a user’s biological information is linked to the audiovisual data presented,” the researchers explained in their paper. “A player plays with virtual objects through physical body movements and the system feeds back biometric information and delivers the change of biometric data through the sensory presentation to the user.”

The new multi-sensory VR game was developed using Unity, a renowned platform for designing video games created by Unity Technology Inc. The devices used to implement the game include a HTC Vive VR head-mounted display, controller and tracking system, as well as a tactile presentation device that was integrated inside the controller unit. The player’s biometric information is collected using a stethoscope with a built-in microphone, which is connected to a PC via a stereo mini-plug cable. To reduce microphone noise and improve the detection of the player’s heartbeat, the team also used a 500 Hz low-pass filter.

In the future, the researchers plan to conduct further studies aimed at evaluating the effectiveness of their game in producing its desired effect. The feedback they collected so far seems to be promising, as most users who played the game felt the connection between their heartbeat and the spherical object in the simulated environment.

“A user who tried the game for the preliminary survey said, ‘The sphere (corresponding to the heart) is alive because the changes due to physical movement are fed back by tactile feedback,” the researchers wrote in their paper. “Some comments have foreseen that it affects sensibility, such as ‘I get a feeling,’ and ‘I hesitate to cut the wire at the end.’ As future tasks, in addition to user evaluation, improvement of the execution speed of biological information analysis in the system will be significant for its broader implementation.”