GTA V connected to the platform OpenAI Universe for teaching AI autopilot

A little over a month since, as a nonprofit organization OpenAI Elon musk introduced the middleware Universe for training and learning AI. Theoretically, learning can happen in all of human information is available via the Internet. Using the software platform Universe intelligent agent uses a computer in exactly the same way as it does people: he will look at the pixels of the computer screen and interact with the virtual keyboard and mouse.

Now to the dozens of games available for training AI added the game Grand Theft Auto V, which has an exceptional realism.

You can create an agent AI of an unmanned vehicle on any framework for machine learning and is relatively easy to connect to the computer with the installed GTA V.



Agent AI is at the crossroads. In the console on the left displays the diagnosis status of pressing the brake pedal (true), and other current settings of the autopilot

Game GTA V, even with the disconnected violence, provides rich opportunities for learning AI for an unmanned vehicle. This is one of the largest and richest open worlds. The game action takes place on the island of San Andres has an area of almost 20% of the present Los Angeles. Here you can run various scenarios to test AI. Agents available 257 different types of cars, 7 types of bikes, 14 types of weather and the environment changed during the simulation.



The Island Of San Andreas

Thanks to the numerous mod, in GTA V you can download buildings from real cities, real cars, real road signs and other objects. Accordingly, the agent AI will learn to control the car in real road conditions.

Platform Universe
Open universal platform Universe — the continuation of systematic action OpenAI to create world open universal AI. In April this year, the organization released a public beta version of Gym OpenAI Toolkit for the development and comparison of algorithms for reinforcement learning. "Gym" OpenAI Gym consists of a large number of environments (simulator, humanoid robot, to games Atari). Is there a website for comparison and reproduction of results.

OpenAI considers that reinforcement learning is an important method of machine learning, which will greatly improve the AI. In the process of learning this method of the test system (the agent) learns by interacting with some environment. Unlike traditional supervised learning, a response to the decisions of the AI are the signals of a reinforcement, some reinforcement rules are generated dynamically and inaccessible to human understanding, that is based on the simultaneous activity of formal neurons.

Starting the agent AI on your computer with GTA V
The integration of the Universe with Grand Theft Auto V created and maintained the project DeepDrive, which is now moved to open source. The draft provides that the agent is started Universe on his own computer with an installed copy of the game.



Universe through the agent of the artificial intelligence gets access to a realistic three-dimensional game world of GTA V. the next video shows footage from the game, who transferred to the processing agent AI (artificially slowed down to 8 FPS, top left), the diagnostic information from the agent and the environment (bottom left), and also convenient to view person view from camera (right).

Integration of the AI in the game prohibits any acts of violence in GTA V.


To start the agent on your computer you want to run the server process GTA V. first you should install the Python library universe, and then connect the agent using the following code.

<code class="python">import universe import gym # register into Gym environments Universe from universe.spaces import joystick_event env = gym.make('gtav.SaneDriving-v0') env.configure(remotes='vnc://$host:$port') # point to the GTA V Universe server observation_n = env.reset() while True: steer = joystick_event.JoystickAxisXEvent(-1) # turn right throttle = joystick_event.JoystickAxisZEvent(-1) # go in reverse # Alternatively, use WASD keys to steer: ('KeyEvent', 'w', True) action_n = [[steer, throttle] for _ in observation_n] observation_n, reward_n, done_n, info = env.step(action_n) env.render()</code>As usual for the agents Universe, the AI uses a virtual keyboard, but in this case he is available and a virtual joystick. The best results AI shows it with a joystick.

DeepDrive
DeepDrive is a specialized platform for the development of AI for self-driving cars open source. It uses moderski frameworks and special equipment to turn GTA V into a normal car simulator. Available pre-trained agents with the ability to control the transport and the datasets on which they were trained.

Although the platform DeepDrive appeared before the Universe, but now its developer has decided that it would be wise to release your work on this open platform. It was the right move. Before agent startup required a Windows computer and many hours of environment settings (there used to intercept DirectX for screen capture and for writing the agents had to use the C++ interface to the Caffe for Windows). DeepDrive now installed in 20 minutes, runs Linux, and OS X, and to write agents in any framework for machine learning.

Published in the open access source code and AMI for GTA V, previously an FBI-trained driver. His training lasted for 21 hours (600 thousand images). The agent has basic driving skills and is a good start for your own experiments.


The joint efforts of thousands of agents AI, you can create a truly refined driver — a program that can operate this unmanned vehicle in the real world.

Source: geektimes.ru/post/284522/