Google uses machine learning to improve the efficiency of data centers

Internet giant uses machine learning and artificial intelligence to improve the efficiency of their data centers. As reported by Joe Kavanagh, vice president, Data Center at Google, the company began to apply neural networks to analyze the huge amount of data collected on the servers, and issuing recommendations for improving their work.

In fact, Google has built a computer that is aware of its data centers, even more than the engineers themselves. The human resource is not deducted from the account, but Kawa believes that the use of a neural network would give Google the opportunity to reach new horizons in the efficiency of the server farms due to go beyond what they can see and analyze the engineers.





Running Google is already some of the most energy efficient data centers in the world. Using artificial intelligence will give Google the opportunity to look into the future and simulate thousands of schemes of work of their data centers.

In the early stages of using neural networks allows Google to predict with accuracy PUE factor of 99, 6%. These recommendations for all the seeming insignificance led to substantial savings, since have been applied to thousands of servers.

Why Google turned to machine learning and neural networks? The main reason lies in the fact that data centers are constantly expanding, it becomes a challenge to Google, use sensors to collect millions of data points on the infrastructure and energy consumption.

"In such a dynamic environment as a data center, a person is sometimes difficult to see all the variables of the relationship," - says Cava. - "We have been working on the optimization of the data center. All certainly better ways already been implemented, but we must not stop! »

Meet the boy-genius h4> The neural network created by Jim Gao Google engineer Google, whose colleagues nicknamed the "boy genius" because of the ability to analyze large amounts of data. Gao analyzed the cooling systems by applying the principles of hydrodynamics and monitoring data to create a 3D model of the air flow inside the server room.

Gao believed that you can create a model that keeps track of a larger set of variables, including the utilization of IT equipment, weather conditions, the work of cooling towers, water pumps and heat exchangers that maintain the normal temperature of servers Google.

"Computers are good because they can see all the hidden data in history. Jim took the information that we collect every day and drove it through his model to come to an understanding of complex chains of interaction, to the meaning of which could not fail to notice the employees, being mere mortals "- writes in his blog Cava. "Through a series of trial and error, the model now gives Jim accuracy of 99, 6% in calculating PUE. This means it can now apply the model in search of new ways to increase the effectiveness of our actions. " The image below shows the correlation between the predicted (black curve) and actual (yellow curve) changes PUE.

How it works h4> Gao began work on machine learning as the above "20 percent project." Traditionally, Google allows its employees to spend part of their time working on the development of innovation, in addition to their core responsibilities. Gao was not an expert in artificial intelligence. To explore key issues in machine learning, Gao was course at Stanford Professor Andrew Un.

Neural network simulates the human brain, allowing the computer to understand and to "teach" the problem without having to program them explicitly. Search engine Google is often cited as an example of this type of training, which is also one of the key research areas in the company. "This model - no more than a set of calculations of differential equations - explained Cava. - "But you have to understand the math. The model begins with the study of the interaction of variables ».

To start Gao had to identify the key factors influencing the energy efficiency in data centers Google. He narrowed the number of indicators to 19 and designed a neural network, a machine learning system that can recognize patterns in large data sets.

"A huge number of combinations of equipment and settings complicates finding optimal efficiency," - writes in his report to Gao. - "In the data center running tasks can be implemented using a number of combinations of equipment (mechanical and electrical) and software (management strategies and settings). Check each combination for efficiency is virtually impossible - there is a time limit, frequent load fluctuations in the IT equipment, weather conditions, and the need to maintain stable operation DPC. »

Operates on a single server h4> As for the equipment, then at the request of Kava, the system does not require extraordinary computing power and runs on a single server, and could even work on a high-class desktop computer.

The system was launched on multiple data centers Google. Machine learning tool could offer a few changes that have led to a gradual improvement of PUE, including improved load distribution when the power infrastructure, and small changes in temperature water cooling system.

"Recent tests in data centers Google has shown that machine learning - an effective method of using existing sensor values ​​to simulate the distribution of energy in the data center and leads to substantial cost savings," - says Gao.

The machines do not take precedence h4> Kawa believes that this tool will help Google to model and develop other projects in the future. But do not worry, Google data centers will not soon acquire consciousness. Now the company is interested in automation, and even the recently acquired company-developer of robotics, but so far none of the data centers Google does not work solely on automated management ».

"We still need people to make the right conclusions about all this," - says Cava. - "And I still want to ensure that our engineers are familiar with these recommendations».

The biggest bonus of using neural networks will emerge in gryadushie years, the construction of a new server platform Google. "I foresee the use of this principle in the design of data centers," - says Cava. - "This advanced technology can be used in the design and further improvements. I think we will find other ways to apply ».

Google shared its approach to machine learning in the статье Gao, hoping that those who also runs a powerful data centers will be able to apply it in practice. "This mechanism - not something special that can only use Google or just Jim Gao," - says Cava. - "I would really like to see more widespread use of this technology. I think the whole industry will benefit from this. This amazing tool that allows you to become as efficient as possible. "

Source: habrahabr.ru/company/ua-hosting/blog/230627/