Orchestra of robots — inventor Gil Weinberg about the musical collaboration of man and machine

We are afraid of competition of artificial intelligence in different areas, but not in creativity: the ability to create perfect humanity used to be considered its prerogative. Musician and Creator of the interactive robots Gil Weinberg defied the stereotypes of his creations can perceive someone else's music, to improvise and to establish an emotional connection with the listener. The inventor gave a lecture on Geek Picnic and told T&P, why a hit is impossible to calculate in advance how to listen to a live U2 concert in my own room and the robots can help us in the study of human nature.

— I read that the experiments you pushed the sad experience at a music school — can you tell us more about it?

— I had a very severe musical education. Teachers constantly told me what to do and how to hold hands. And insisted that I have no right to create their own music, not yet learned to carry someone else. So when I grew up, I began to experiment to disprove this concept. I think the children five or six years to study music and perform classical repertoire, but in parallel they need to develop creativity and explore the world of their feelings, even if they have no musical background. And I believe that knowledge and creativity should blend harmoniously. I have created several apps for iPhone for kids with no musical education. You can dance and move, or the movement and music are closely linked, and artificial intelligence in the application analysed these movements and respond to them in different ways, allowing to create melodies in different genres.

— Why have you switched to robots?

— I have bachelor's degree is in interdisciplinary program — musicology and computer science. Understood that it is necessary to continue research and went first to the master's degree and then to graduate school at the MIT Media Lab. At first I was interested in making different musical instruments, and robots have decided to do, having already made it into the Technological University of Georgia. Why I changed the concept? Too tired from the electronic sound: all computer music emanates from speakers and it sounds flat. I wanted more bright, rich sound like on a live concert. And when I got into Georgia Tech, one of my students was able to interest me in robotics, and later created my first robot, Haile. I always try to learn from their students.



The first creation of Weinberg became a robot drummer Haile. He was able to hear another drummer, recognize rhythms, and on their basis come up with their own compositions. In addition, technically, the robot can do what man can do: for example, at the same time to recapture the seven quarters of one "hand" and nine other, creating a complex rhythm.

— What is your musical talent in your opinion?

— I would not say that my robots got talent. Rather, they allow people to show their own talent. I use mathematical algorithms, which are not peculiar to people, so that they could Express themselves more fully, to create something new and unique. But we have a long way to go before we can call the robot talented.

But your robots look like real person. Can music to put the soul into a mechanical being?

— My robots behave, because I believe that music is something very personal, something that is born in communication and interaction. So we go to concerts, and not just listen to songs on the computer — we want to see how the music is born in real time as drummer and bass guitarist play the rhythm and lead guitarist improvises along with a vocalist. My robots are able to establish an emotional connection with the listener and other musicians. Of course, this raises questions. People who understand how machines can clean the house or build something, but they find it difficult to accept that a robot can create music. Many people ask me: how dare you give them something that is peculiar only to us?



The other robot Weinberg, Shimon, can play music on the marimba, to observe the actions of people using the camera and nod in time to the music. He is able to adapt to a partner, changing the tempo, volume and rhythm of the performance — the overall impression created by that man and robot are on the same wavelength. In addition, Weinberg and his colleagues taught Shimon to beat the style of famous jazz masters — Thelonious monk and John Coltrane. The Professor turns the music on the iPhone, and the robot begins to improvise, trying to imitate what he heard.

— What will be the music of the future?

— Now we see that the quality of the music gradually decreases. After the CD was MP3, which they conceded, and now people are listening to music on YouTube where the sound quality even worse. But then, I think people will want to return again to high-quality sound — perhaps, in some new sense of the word. So I see the future like you come to a music store and buy a player instead of an orchestra of robots. There will be guitar, bass, drums and everything you need. Then you get home and choose a song — for example, U2, and the robots are playing it live in your room. The problem is in the vocals: now we do not know how to make a robot voice sang with Bono. But I think this is a solvable problem.

— And the robots themselves will compose the tunes?

— They already know how. This is called "algorithmic composition" was invented decades ago — of course, then it was not done by robots and computers. The first of such experiments began in the 1960s, but it turned out that people usually don't like what composes artificial intelligence.

There is another trend: record labels using the computer to analyze all the songs that were popular in past decades, trying to capture the melodic patterns that make the song a hit. But I don't think they will succeed. There is something special about how there is a great melody. Music is close to mathematics, but, in addition, it has emotions. I think that we can bring in emotion, and robots to create interesting ideas and to play instruments like the person is not physically able. If we combine math, expression, emotions and physical ability, there will be something incredible.



One of the most famous inventions of the Weinberg — cyberpirates for drummer Jason Barnes from the British band Def Leppard, lost his right hand in an accident. Artificial hand gave Barnes a truly superhuman capabilities: it is capable not only to submit to his will and to beat the very complex rhythms, but also adapted to other group members and create their own improvisation. In addition, the prosthesis is able to play along with the party of Nobel laureate Richard Feynman, who was not only a great physicist but also a good musician. However, it is not without incidents: before flying to Saint Petersburg Jason passed his hand into the Luggage, and the staff of Pulkovo managed to lose the suitcase. A few hours later, but the concert at the Geek Picnic was held later than expected.

— You and Jason are planning more collaborations with dead composers?

— I chose Feynman, because it was very symbolic — the combination of music and science. In addition, few know about his passion for the drums. It is interesting that Jason owns the hand better and better: first, he's not very good at it, but practice works wonders, as with any tool. We still have so much to do and if I find an interesting idea, we will use it. But while I'm too busy developing robots.

— Perhaps a high-tech prostheses will be able to make people superheroes — example Jason is very convincing.

Six — time champion of the Paralympics runner Oscar Pistorius now competes in the sport — he has developed a special foot prostheses, Flex-Foot Cheetah. It sparked a debate. Before people would say: "Look, it's not fair — a healthy person is competing with a handicap!" Now the opposite is true: the robotic prostheses give the holder a competitive advantage. It's an interesting moral dilemma. Ask me if I'm going to deprive musicians of their work. But I didn't mean — I just wanted to expand our capabilities.

— Comparison with robots will help us to better understand what it means to be human?

When we play a musical instrument, for example piano, it's a subconscious process, we do not track the movement of each finger. But I, as the Creator of the robots, you have to think over every movement. And developing such a car, I can perform exactly how we create music and how its perception is connected with our body. So my research is not only a way to create something new, but also a tool of self-knowledge.

Daria Varlamova

Source: theoryandpractice.ru