![]()
11
AI for Business – Minimize Costs with Deep Q-Learning
It's great that you can implement a deep Q-learning model to build a self-driving car. Really, once again, huge congratulations to you for that. But I also want you to be able to use deep Q-learning to solve a real-world business problem. With this next application, you'll be more than ready to add value to your work or business by leveraging AI. Even though we'll once again use a specific application, this chapter will provide you with a general AI framework, a blueprint containing the general steps of the process you have to follow when solving a real-world problem with deep Q-learning. This chapter is very important to you and for your career; I don't want you to close this book before you feel confident with the skills you'll learn here. Let's smash this next application together!
Problem to solve
When I said we were going to solve a real-world business problem, I didn't overstate the problem; the problem we're about to tackle with deep Q-learning is very similar to the following, which was solved in the real world via deep Q-learning.
In 2016, DeepMind AI minimized a big part of Google's yearly costs by reducing the Google Data Center's cooling bill by 40% using their DQN AI model (deep Q-learning). Check the link here:
https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40
In this case study, we'll do something very similar. We'll set up our own server environment, and we'll build an AI that controls the cooling and heating of the server so that it stays in an optimal range of temperatures while using the minimum of energy, therefore minimizing the costs.
Just as the DeepMind AI did, our goal will be to achieve at least 40% energy savings! Are you ready for this? Let's bring it on!
As ever, my first question to you is: What's our first step?
I'm sure by this point I don't need to spell out the answer. Let's get straight to building our environment!
Building the environment
Before we define the states, actions, and rewards, we need to set up the server and explain how it operates. We'll do that in several steps:
- First, we'll list all the environment parameters and variables by which the server is controlled.
- After that we'll set the essential assumptions of the problem, on which your AI will rely to provide a solution.
- Then we'll specify how you'll simulate the whole process.
- Finally, we'll explain the overall functioning of the server, and how the AI plays its role.
Parameters and variables of the server environment
Here is a list of all the parameters, which keep their values fixed, of the server environment:
- The average atmospheric temperature for each month.
- The optimal temperature range of the server, which we'll set as .
- The minimum temperature, below which the server fails to operate, which we'll set as .
- The maximum temperature, above which the server fails to operate, which we'll set as .
- The minimum number of users in the server, which we'll set as 10.
- The maximum number of users in the server, which we'll set as 100.
- The maximum change of users in the server per minute, which we'll set as 5; so every minute, the server can only have a change of 5 extra users or 5 fewer users at most.
- The minimum rate of data transmission in the server, which we'll set as 20.
- The maximum rate of data transmission in the server, which we'll set as 300.
- The maximum change of the rate of data transmission per minute, which we'll set as 10; so every minute, the rate of data transmission can only change by a maximum value of 10 in either direction.
Next, we'll list all the variables, which have values that fluctuate over time, of the server environment:
- The temperature of the server at a given minute.
- The number of users connected to the server at a given minute.
- The rate of data transmission at a given minute.
- The energy spent by the AI onto the server (to cool it down or heat it up) at a given minute.
- The energy that would be spent by the server's integrated cooling system to automatically bring the server's temperature back to the optimal range, whenever the server's temperature goes outside this optimal range. This is to keep...
