The future for 3D-sound

Modern audio technology can radically change our understanding of computer games, music and movies i> h5>
I can not believe my ears. I took an ordinary gray cube around the head and was ready to swear that hear of it instrumental music is heard - as if it was a tiny speaker. The sounds of the violin suddenly became very loud when the cube is close to your ear and calmed down when he drew back away.

If the cube was placed behind me, it seemed that I turned my back to the string quartet. When I spent a cube over his head on it ran shivers, and it seemed that music plays from above. But by itself is silent cube. Music could be heard through the headphones, and when I took them off - all the illusion disappeared.

I'm in the big room IT-incubator of the University of Maryland, where a team of a startup VisiSonics, consisting of eight people. It was they who created the software which held my brain, causing him to hear 3D-sound through headphones. Grey cube, which I hold in my hand, is connected to a device to monitor its location relative to my head.

Technology RealSpace 3D, developed VisiSonics, uses spatial data from the tracking system to process music, so it seems as if it was coming from the cube. This demo simply shows that designed software capable of actually trick your brain - is strikingly what I had completely forgotten about the headphones.



Research Ramani Duraisvami (Ramani Duraiswami) helped him to create software that accurately reproduces the 3D-audio through headphones. Accurately reproduce 3D-audio can be by measuring individual characteristic transfer function of a so-called head (HRTF), which normally takes 2-3 hours. The prototype developed Duraisvami, which sits Zotkina Elena is able to measure the HRTF for a few tens of seconds, i>

This effect, which in VisiSonics have high expectations. The company hopes to develop audio engine for games, which will be used to create a virtual or augmented reality. I can say that the technology has already made a big step in that direction. Oculus VR, not so long ago bought Facebook, last October, has licensed for its RealSpace 3D head-mounted display Oculus Rift, and made it to the list of development tools audio software Oculus.

VisiSonics also negotiating with other companies. "The goal for 3D-sound great - make listening to music through earphones you thought that you do not have them", - says Ramani Duraisvami, one of the founders and a teacher computer science at the University of Maryland in College Park.

But the company does not intend to stop at the video games and virtual reality. "Where people use headphones - we'll be there to completely change their view of the sound," - added Duraisvami.

We perceive our ability to hear the sound in three dimensions for granted - even with your eyes closed, we can determine the position of the sound source. The brain interprets the sound is surprisingly good coming into our ears, which act as a spatially separated microphones. This means that it uses a different signal parameters such as the delay between receipt of it in right and left ear, to determine on which side the sound is coming. Repeat this effect with the use of headphones is not easy.

To achieve the effect for 3D-sound can be simply arranged around a large number of speakers. This approach is used by many theaters, but with headphones, this trick does not work out. The flourishing of mobile devices means that many of us are playing computer games, listening to music and watching movies, using headphones and software VisiSounds can play 3D-sound from any device.

Imagine how you listen to Beethoven's 9 on your mobile, and you feel that you are sitting in the best seats in the theater, listening to a live performance. Or imagine the horror while watching on your iPad, you suddenly begin to hear the footsteps of the villain right behind her.


One of the first demo, which created Haxton Rod (Rod Haxton), to show the possibilities of technology RealSpace 3D i>

Engineers have tried to reproduce the effect for 3D-sound for a century. Most of this time the scientists used a dummy head with a microphone in each ear for sound, trying to create a model of a human head.

VisiSonics found a way to get rid of the dummy and now use the software. We determine the source of the sound, turning your head in different directions. In a virtual or augmented reality, it is important not only to pinpoint the source of the sound, but plausible enough to change the sound depending on the position of the head. When you turn your head, VisiSonics software calculates in real time the parameters of the sound changes - this creates the effect of full immersion.

The software also takes into account the features of the room, determine the behavior of sound in space. You hear not only the sound but also its multiple reflections from the walls, furniture and other things in the room. Sound reverberation occupy last place in the determination of the distance between the person and the sound source and the transfer of feelings of distance. According Duraisvami, it is the precision with which the software recreates the reverberation and makes it sound so natural RealSpace 3D.

The third important component of software Duraisvami - is the ability to recreate how sound interacts with your body, even before it enters the ear canal. Sound differently reflected from the head, body and bends of the outer ear, depending on the source position. Scientists describe this feature of the transfer function of the head or HRTF (HRTF - head-related transfer function). If you want to accurately recreate the sound coming from a certain direction, then you have to take into account the HRTF.

Although the sound is processed VisiSonics, comes from the headphones, my brain sure that it goes gray cube, which I hold in my hand. «3D-sound - a" three-legged stool "- says Duraisvami. - If you remove one leg, then it will fall ».

With these advances in 3D-audio any music, movie or game can play a sound as if you really are in an open field, a cozy room or concert hall. You can release music and movies from the very beginning supported 3D-audio.

For example, Björk has recently released a panoramic musical clip with 3D-sound (the technology used in the video, it has been developed among the competitors VisiSounds). Singer on his page on Facebook to describe the feeling of watching a clip: "You seem to be on the beach and around you dense sit around 30 musicians." STUDIO , trying to capture panoramic movies with immersive, where all the action is happening right around you, this technology will need everything to be the most plausible.

Over the past decade the visual component of virtual reality has made a big step forward, but audio begins to "catch up" just now - and just in time, as several companies have recently announced (or have already released) new headset. "You can not and should not be" immersed "in virtual reality by half" - grins Duraisvami.

Another demo opportunities RealSpace 3D, prepared by an independent developer with plugins VisiSonics, helped me to understand why you need Oculus 3D-audio. When I first put on a head-mounted display, I immediately wanted to dissolve in it.

I was standing on a wooden bridge over the river, and grew up around the tropical trees. I turned his head to see lush vegetation of the world - a feeling similar to the study of the rich environment of computer games, but in this case you just feel like a part of it, and this feeling is reinforced with sound RealSpace 3D. I stood under the open night sky around chirping crickets, and on my right out of the chimney Flowing water stream. When I turned my head to get a better view landscape, the sound changes as needed, keeping the full sense of reality.

// player.vimeo.com/video/105967316 video>
This is a demo using the Oculus Rift helped me understand the importance of surround sound in the world of virtual reality i>

VisiSonics also tried to introduce surround sound in gaming worlds: the company believes that the player will be able to determine the location of enemies in the right direction or at the hearing. The team released a plug-in for RealSpace 3D game engine Unity, and is working to incorporate its technology into other engines. Due to the fact that RealSpace 3D can be used on all platforms (from iPhone to high-performance computers), software should provide high VisiSonics result using the available computing power.

VisiSonics - not the only company that operates in the market for 3D-sound. Among her competitors have projects such as Impulsonic and Two Big Ears - small, young start-ups. Another startup, engaged in creating sound for virtual reality, is called Thrive Audio and recently was acquired by Google - is another sign of the growing interest in the industry.

Haxton Rod (Rod Haxton), a leading engineer VisiSonics, said that the competition - it's good, and can contribute to the emergence of surround sound: "The sound quality - this is what will determine the choice of the buyer, as well as ease of use, and how much processing power required technology ".

Haxton explains the success of software VisiSonics study conducted Duraisvami Zotkin and Dmitri, an engineer from the University of Maryland, working in "part-time". "It's all the merit and genius Dmitri Ramani - exclaims Haxton. - It's going on the magic that is not understood by others ».

Magic - not magic, but Duraisvami Zotkin and took more than 10 years to bring the technology to the proper RealSpace 3D view, but it turned out more difficult to extract from it benefit. It all started with technology designed to help soldiers who have lost their sight.



VisiSonics was originally founded for the production of "audiokamery" - a device connected to a laptop in this image i>

Duraisvami began working at the University of Maryland in 1998 and decided to apply their knowledge of physics and engineering expertise in the field of audio studio. At first, he tried to create an artificial audiosredu for soldiers who have lost their sight. The aim was to give soldiers the ability to virtually feel like "sounds" street corner or office before they come into the real world. To play the sounds of the environment, Duraisvami developed "audiokameru" and software to it (which later migrated to RealSpace 3D).

Duraisvami and graduate student Adam O'Donavan (Adam O'Donovan) VisiSonics founded four years ago with the aim to produce and sell their audiokameru. Soon they realized that created processing technology can be used for other things, but found it difficult because it was not clear how to get users and companies to start using their operating time. "They said that if we come up with" bait "better, people reach for us - told Duraisvami. - But this is only part of the problem. "

Although their technology could have an effect on music or movies in the industry is not so easy to break. Duraisvami believes that virtual reality is very important: "It allows us to take a fresh look at familiar things for us».

For Duraisvami was obvious from the start that his technology works fine. As a tiny company located away from the Silicon Valley, to attract the attention of a virtual reality industry giants? "We wanted to start working with Oculus, - says Duraisvami. - We know that the technology is ideal for use in virtual reality, and our CEO Gregg Wilkes (Gregg Wilkes) literally chased them. " It took more than a year, these efforts have begun to bear fruit.

Haxton, Senior Software Engineer, joined VisiSonics in 2012, after years of work in the gaming industry. Fascinated by the potential of surround sound in games, he hastily prepared a presentation project, in which the player controls a huge rabbit. But when it came time to show the project at the conference with a lot of serious military simulations, he replaced the rabbit character with a gun, surrounded by futuristic tanks and helicopters - is more consistent environment. Although Haxton still added to a robot that went back and forth and singing a song MC Hammer «U Can not Touch This».

Duraisvami colleagues demonstrated his project at various conferences, but could not draw attention to themselves. That all changed when they arrived at the Game Developers Conference 2014 in San Francisco. On this show, they did not have a booth, so Haxton paced the show in a T-shirt with the words "Ask me if you want to see this presentation for 3D-sound for games", printed on a black background. Meanwhile, Duraisvami Wilkes and we borrowed some of unoccupied tables and trying to find workers Oculus VR, to show them a presentation. "In fact, we set up an ambush at these guys - says Duraisvami. - When the guys from Oculus tried VisiSonics, they wanted to buy a license from us ».

"That's how we come to this. We like the guerrillas, tried to seize any and force him to tell us what we wanted to hear, - says Haxton. - The success of the team at the trade fair was a tremendous breakthrough ».



Ramani Duraisvami (left) and Dmitry Zotkin (middle) audiosfere worked more than 10 years and continue to use their knowledge of physics and engineering expertise to improve the accuracy of the technology RealSpace 3D. To the right sits Yancheng Luo (Yuancheng Luo), a graduate student who after graduation worked in VisiSonics i>

Duraisvami continues to work on the measurement of the personal HRTF, to make the surround sound more authentic. Currently RealSpace 3D uses a generalized HRTF, which copes with its task, but it does not account for differences in shape and size of our heads and ears that affects the perception of sound.

HRTF measurement usually takes 2 or 3 hours, but Duraisvami able to reduce the time to a few tens of seconds. He says that their goal is personalization of music and sound perception using individual HRTF.

With these and other advantages of 3D-sound and virtual reality promises to get better and better. "If you look back 10 or 20 years, we will see the 8-bit games from Atari, - says Haxton. - Virtual reality to the next level, and a small company VisisSonics from College Park in Maryland is in the thick of things. "

Source: geektimes.ru/company/audiomania/blog/251530/