The history of the legend — the voice of Hawking

Eighty eight million six hundred seven thousand five hundred twenty four



For the first time, Stephen Hawking met Gordon Moore, one of the founders of Intel, at a conference in 1997. Moore noticed that the computer that Hawking used to communicate, had an AMD processor and asked him if he wanted instead a "real computer" with an Intel microprocessor. Since Intel provides Hawking custom personal computer and technical support, replacing his computer every two years.

Hawking has lost the ability to speak in 1985, when he caught pneumonia during a trip to CERN in Geneva. In the hospital he was transferred on artificial lung ventilation. The condition was critical. The doctors asked Hawking's wife Jane, to turn off the ventilator or not. She furiously refused. Hawking was taken to the hospital Addenbrookes in Cambridge, where doctors were able to contain the infection. To facilitate breathing Hawking, they also performed a tracheotomy, making a hole in the throat Hawking and placing the tube in his trachea. In the result, Hawking irreversibly lost the ability to speak.

Some time Hawking communicated using a spelling card, patiently indicating letters and forming words, raising an eyebrow. Martin king, a physicist who worked with Hawking on a new communication system, contacted a California company Words Plus, which has developed a program Equalizer, allowing users to select words and commands on a computer using a hand clicker. King appealed to the Director-General Words Plus Walter Voltou and asked what software can help Professor of physics in England with amyotrophic lateral sclerosis. Woltosz created an earlier version of Equalizer to help his godmother, who also suffered from ALS and lost the ability to write and speak.

"I asked Stephen Hawking this, but he couldn't give me a name without permission, told, Voltas. The next day he called and confirmed it. I said I'll do what I can."

The first Equalizer was working on Apple II, associated with a speech synthesizer, developed by Speech Plus. Then this system was adapted by David Mason, the engineer husband of one of Hawking's nurses, turning it into a portable system that can be installed on one of the arms of the wheelchair. With this new system, Hawking was able to communicate at a speed of 15 words per minute.

However, the nerve that allows him to move his fingers continued to deteriorate. By 2008, Hawking's hand was too weak to use the clicker. His assistant then has developed a device that could control the cheek. Attached to points infrared sensor can capture the motion of the muscles of the cheeks Hawking. Since then, Hawking has used to answer emails, surf the Internet, write books and communicate using only one muscle. However, this ability to communicate comes to an end. By 2011, it could issue only one-two words per minute, so sent a letter to Moore (the same one who is the author of Moore's law): "My audio input device is very very slow now. Whether Intel could help?".

Moore instructed Justin Rattner, the CTO of Intel, to deal with the problem. Rattner assembled a team of experts on human interaction with the computer in Intel Labs and brought her to Cambridge for a conference dedicated to the 70th anniversary of Hawking, January 8, 2012.

"I gathered a group of experts in the labs of Intel, said Rattner to the public. — We're going to gently apply advanced computer technology to improve communication speed Steven. We hope that this team will make a breakthrough and find a technique that will allow him to communicate at a level that was a few years ago."

Hawking was too ill to attend his own birthday party, so met with experts from Intel just a few weeks later, in his office in the Department of applied mathematics and theoretical physics at Cambridge University. In a team of five people included Horst Haussecker, Director, Experience Technology Lab, Lama Nachman, Director of the Anticipatory Computing Lab and project head, and Pete Denman, an interaction designer.

"Steven has always inspired me, says Denman, is also using a wheelchair. — After I broke my neck and became paralyzed, my mother gave me a copy of "a Brief history of time", which had just come out. She told me that people in a wheelchair can still do incredible things. Looking back, I realize how prophetic were those words."

After the Intel team introduced themselves, Haussecker took the lead, explaining why they were there and what were their plans. Haussecker talked about twenty minutes, until suddenly, Hawking spoke.

"He greeted us and said he was very happy to see us all, says Denman. — We were unaware that all the while he was typing. It took him twenty minutes to write a greeting in 30 words. We were pretty stunned. The moment was very sharp. We realized that the problem was much more serious than we thought".

At the time Hawking was using a computer interface in the form of a program, EZ Keys, an updated version of the previous programmes, also designed by Words Plus. She provided him with a keyboard on the screen and the basic algorithm for word prediction. The cursor automatically scanning the keyboard in rows or columns, Hawking was able to choose the desired character with a movement of his cheeks, stopping the cursor. EZ Keys also allowed Hawking to control the mouse in Windows and operate other applications in his computer. He could surf the Internet with Firefox and to write lectures with Notepad. He also had a webcam that he used with Skype.

Team Intel has decided to update the archaic system of Hawking, which would require the creation of new hardware.

"Justin thought we can use technologies like facial recognition, gestures, gaze and brain-computer interfaces, says Nachman. — Initially we fed him a lot of these wild ideas and have tried a lot of rare technologies."

Most often, these attempts were unsuccessful. Track view did not work with the look of Hawking because of his lowered eyelids. Before the Intel project, Hawking had experimented with EEG sensors that could read his brainwaves and potentially issue commands to the computer. Inexplicably, quite a strong brain signal and failed to find.

"We lit the letters on the screen and tried to identify the letter at the response of the brain, says wood. With me this worked well, but when Steven tried, everything was down the drain. He was not able to reproduce a strong enough signal distinguishable from noise."





"The more we watched him and listened to his comments, the more we end up with what he asks of us, in addition to the improvement of connection speed, to give it new functions that will enable it to better interact with their computer," says Nachman.

Back in the Intel lab after months of research, Denman prepared a 10 minute video for Hawking, which demonstrated new neurocomputer prototypes that the team wanted to try with Hawking, and asked him to answer.

"We came to the conclusion that the changes we want to make, not much will change the interaction of Hawking with his system, but it will change," says Denman. The changes included additions like the back button, which Hawking could use not only to delete characters but for backward navigation in the interface; predictive algorithm input words; the NAV of the next word, which allowed him to effectively select word by word instead of typing them.

The main change, according to Denman, was with the prototype. Hawking frequently missed keys while pressing the letter next to the one that actually wanted to press. He could miss the letter, remove it, fluff again, again to remove. It was terribly depressing and slowed down the entry process. Also this problem was compounded by Hawking's perfectionism. It was important for him to Express his thoughts correctly, so even punctuation had to be correct. He learned to be patient enough to be a perfectionist. He's not one of those who just wanted to send a message. He wanted to do it perfectly.

To solve the problem of missing letters, the Intel team added a prototype, which interpreted the intention of Hawking, and not his actual input, using an algorithm similar to that used in word processing and mobile phones. When the iPhone first came on the market, people began to complain about predictive text but quickly distrust turned into joy. The problem is that you need too much time to get used to it, and you need to trust the system to do their job. However, adding this feature can improve your speed and allow you to focus on the content.

At the end of the video the question was: "What is your level of excitement or fear?". In June that year, Hawking visited Intel Labs, where Denman and his team introduced him to a new system — ASTER (Assistive Text EditoR). "Your current software is a bit out of date, said Denman. — Actually, it is very out of date, but you got used to it, so we changed the method of working your predictive next word, and it almost always identifies the correct word even if the letters you typed is strikingly distant from him."

"This is a big step compared to previous version — said Hawking. I really like".

Engineers have installed a new user interface on the computer of Hawking. Denman thought they were on the right track. By September appeared first feedback: Hawking wasn't adapted to the new system. It was too complicated. Reset button and the one that was responsible for the blunders in letters, he introduced into the bewilderment and their need to change.

"He's one of the smartest guys in the world, but he's still not used to modern technology, says Denman. — He has never been able to use the iPhone. We tried to teach the famous and smartest 72 year old grandfather on the planet to a new way of interacting with technology".

Denman and the rest of the team realized that they need to start a new way to think about the problem. "We thought the developed software in the usual sense, when you deploy the network and trying to pull the fish up. We had no idea how the design will depend on Stephen. We needed to concentrate resources on the training of one person."

At the end of 2012, the Intel team have created a system that recorded the interaction of Hawking with his computer. They recorded tens of hundreds of hours of video, which included a range of different situations: Stephen prints, tired prints Stephen, Stephen uses the mouse, Steven tries to adjust the window size. "I looked at the footage again and again," says Denman. "Sometimes I run video with four times the speed and still found something new."

By September 2013, with the help of Jonathan wood, Hawking's assistant, they implemented another iteration of the user interface in Hawking's computer. "I thought we did it, we did it," says Denman. But next month, it became clear that Hawking's problems again. "One of his assistants called it "ASTER torture," recalls Denman. When they said that Steven tried to smile."

It took many months before the Intel team came up with a version that pleased Hawking. For example, Hawking now uses an adaptive predictor of the London startup SwiftKey, which allows him to select a word after printing the letters, whereas the previous system required Hawking to move to the bottom of the interface to select a word from the list. Its system of word prediction was too old. The new system was fast and efficient, but still had to teach Hawking to use it. At first he complained, and only much later engineers realized why: he already knew what words predicted his old system. He used to use the old predictions of the predictive system.

Intel worked with SwiftKey and included a lot of paperwork Hawking in the system, so in some cases he doesn't even have to type before the system will write the next words out of context. The phrase ‘the black hole’ does not require any printing. After ‘the’ automatically appears ‘black’. Selecting ‘black’ automatically appears ‘hole’.

The new version of the user interface Hawking (now known as ACAT — Assistive Contextually Aware Toolkit) includes contextual menus that provide Hawking with various shortcuts to speak, to find something or write a letter; and a new lecture Manager, which gives him control of the timing during conversations. Also there is a button ‘mute’, which allows Hawking to turn off his synthesizer.

"Because it manages the switch through the cheeks if he eats or moves, it generates a random output — explains wood. — There are times when he deliberately wants to say something random. He does this constantly, and sometimes it turns out quite awkward. I remember one time he accidentally typed "x x x x" in his speech synthesizer sounded like "axaxaxax".

Office wood near the office of Hawking. For the most part is a workshop. One wall is littered with electronic hardware and experimental prototypes. On the table there is a camera, part of the working project with Intel. "The idea is to have the camera trained on the face of Stephen, we read not only the movements of his cheeks, but other facial movements, says wood. — He can move his jaws up and down, left and right, control the mouse and perhaps even his wheelchair. It is a good idea, but they are still in development".

Another pilot project proposed by the manufacturers wheelchairs Hawking in the beginning of this year, it is a joystick that attaches to the chin Hawking and allows it to move wheelchair independently. "Steven loves this, says wood. — The problem of the contact of the chin Stephen and joystick. Because it is not moving the neck, very hard to plug and unplug the joystick". In General, however, Hawking could ride independently.





In the small gray box is stored the only copy of the speech synthesizer Hawking. It CallText 5010, the model that gave the Hawking in 1988, when he visited the company that produced them, Speech Plus. Card inside the synthesizer contains a processor that turns text to speech; this device was also used for automated systems answering phone calls in the 1980s.

"I'm trying to make a software version of the voice of Stephen that we would no longer rely on these old hardware cards," says wood. To do this, he had to trace the path of the command Speech Plus. In 1990, Speech Plus was sold to Centigram Communications. Centigram was acquired by Lernout and Hauspie Speech Products, which, in turn, in 2001, ScanSoft acquired. ScanSoft acquired Nuance Communications, a multinational company with 35 offices and 1,200 employees. Wood contacted her. "They had the software with the voice of Stephen from 1986. Looks like we found it on a backup tape Nuance".

Hawking is very attached to her voice. In 1988, when Speech Plus gave him a new synthesizer, the voice was so different that he asked to replace his original. His voice was created in the early 80-ies of the engineer at the Massachusetts Institute of technology Dennis Clutton, the pioneer algorithms that convert text to speech. He invented the DECtalk, one of the first devices for these tasks. Initially, it was written for three voices: Klatta, his wife and his daughter. A female voice was called "Beautiful Betty" (Beautiful Betty), children's "Baby kit" (Kit the Kid), and the male "Perfect Floor" (Perfect Paul). This voice says Stephen Hawking.

According to Wired

Source: hi-news.ru