Analytics for the Internet of Things (IoT)

Andrew Minteer

• Publisher: Packt Publishing
• Year: 2017
• Edition: 1
• eBook ISBN:
  9781787127579
• Topic: Informatica
• Subtopic: Elaborazione di dati su cloud

Break through the hype and learn how to extract actionable intelligence from the flood of IoT dataAbout This Book• Make better business decisions and acquire greater control of your IoT infrastructure• Learn techniques to solve unique problems associated with IoT and examine and analyze data from your IoT devices• Uncover the business potential generated by data from IoT devices and bring down business costsWho This Book Is ForThis book targets developers, IoT professionals, and those in the field of data science who are trying to solve business problems through IoT devices and would like to analyze IoT data. IoT enthusiasts, managers, and entrepreneurs who would like to make the most of IoT will find this equally useful. A prior knowledge of IoT would be helpful but is not necessary. Some prior programming experience would be usefulWhat You Will Learn• Overcome the challenges IoT data brings to analytics• Understand the variety of transmission protocols for IoT along with their strengths and weaknesses• Learn how data flows from the IoT device to the final data set• Develop techniques to wring value from IoT data• Apply geospatial analytics to IoT data• Use machine learning as a predictive method on IoT data• Implement best strategies to get the most from IoT analytics• Master the economics of IoT analytics in order to optimize business valueIn DetailWe start with the perplexing task of extracting value from huge amounts of barely intelligible data. The data takes a convoluted route just to be on the servers for analysis, but insights can emerge through visualization and statistical modeling techniques. You will learn to extract value from IoT big data using multiple analytic techniques.Next we review how IoT devices generate data and how the information travels over networks. You'll get to know strategies to collect and store the data to optimize the potential for analytics, and strategies to handle data quality concerns.Cloud resources are a great match for IoT analytics, so Amazon Web Services, Microsoft Azure, and PTC ThingWorx are reviewed in detail next. Geospatial analytics is then introduced as a way to leverage location information. Combining IoT data with environmental data is also discussed as a way to enhance predictive capability. We'll also review the economics of IoT analytics and you'll discover ways to optimize business value.By the end of the book, you'll know how to handle scale for both data storage and analytics, how Apache Spark can be leveraged to handle scalability, and how R and Python can be used for analytic modeling.Style and approachThis book follows a step-by-step, practical approach to combine the power of analytics and IoT and help you get results quickly

Data Science for IoT Analytics

"Revenues are up 5% due to your little geospatial search trick," the VP of Connected Services says, "You know your former boss's position is still open. Maybe we should fill it from the inside..."

Your pulse quickens, you were hoping he might come to this conclusion. You deserve a promotion after what your analytics has brought to the company. You now have one person working for you focusing on geospatial analysis. You can just imagine what you could do with a whole team.

"There is something that we have been toying around with though," he continues, "With all this data we are collecting, we should be able to tap into machine learning models to predict equipment failures. Some think we should be hiring an outside consulting company to handle all of it. Sounds expensive to me. I sure wish we could coordinate this ourselves, work with data scientists of our own choosing... know anyone that might be up to it?"

He winks and walks off, hands behind his back, whistling a Brahms tune.

In this chapter, we introduce data science techniques such as machine learning, deep learning, and forecasting using ARIMA. Special focus is given on how to use these methods with IoT data. The core concepts for each will be reviewed along with examples in R. Deep learning will be described along with where to go to create an Amazon EC2 instance with TensorFlow.

This chapter covers a lot of ground, so hold onto your hats.

This chapter covers the following topics:

• Machine learning (ML):
  • Core concepts
  • Feature engineering
  • Validation methods
  • The Bias-variance tradeoff
  • Comparing models to find the best fit
  • Random forests
  • Gradient boosting machines
  • Anomaly detection
• Forecasting using ARIMA
• Deep learning:
  • Use cases with IoT data
  • Setting up and running a simple deep learning model using TensorFlow

Machine learning (ML)

As a wise uncle of a human-arachnid hybrid once said, With great power, comes great responsibility. This is very true of ML. There are many ways to go wrong. When in the hands of a skilled practitioner, it truly is a form of art. It can be used to do some incredible things on a grand scale, but it should come with a big caution sign. Use it carefully. Be paranoid and validate, validate, validate.

Although we will be going over some core concepts and providing code that you can take and run yourself, this is a big field with lots to learn. It takes years to skillfully and competently apply it. Each section in this chapter is really a book in itself. No, many books. If you plan to use it yourself on IoT data, read, read, read, and then read some more. This chapter is meant to able to provide you with a good foundation to have meaningful conversations with data scientists on the subject.

What is machine learning?

Ask a hundred experts for the definition of ML, and you are likely to get a hundred slightly different answers. Some will take a broad view and fully include deep learning, artificial intelligence, and some traditional statistical techniques, such as the sum of least squares linear regression. Others will be narrow and restrict their definition to a few modeling techniques, considering it separate from a related field called statistical learning.

Some will say it does not even exist in the real world, only in over-hyped media stories. They feel it is all the same traditional statistical analysis that has been done for decades. Some will consider the term ML as completely interchangeable with the term artificial intelligence, while others will consider them very separate things.

ML is an application of statistical techniques in an ordered set of steps (otherwise known as an algorithm). The statistical techniques are rarely new, many have been around for decades and some over a century. Many of the ML methods have also been around for several decades as well. What has changed is the dramatic decline in the cost of compute power along with a dramatic increase of computing capability. What would have taken months to calculate in 1980 now takes seconds or less.

With the availability of some strong open source statistical software libraries, such as R and Python, combined with the low cost and available speed of modern computing hardware, ML has become practical to implement in a large variety of applications. With an increased use of ML came refinements to the existing methods and developments of new algorithms. This has led to a significant increase in predictive capability and a golden age for ML.

We will use a (hopefully not overblown) analogy to help you think about this. With traditional statistical techniques, you are like a mechanical engineer, applying your expertise and knowledge of how things work to define a set of components that fit together. You use these methods to explicitly build your statistical model. You define the detail of each component based on testing and analysis of the data.

With ML, you become more like an agricultural engineer. A farmer of data models. In this chapter, we will define ML as a method that has three general components that, when combined, grow a program from the soil of the provided data. This set of statistical techniques learn a representation of the underlying function that determines the target values or categories. Learning, in this case, is adaptation and not cognition, as it would be with us humans. At no point does your computer have even an inkling of what it all means–it is all zeros and ones to it.

You as a ML practitioner.

Illustration: Jim Campbell; Inlander; https://www.inlander.com/spokane/farming-data/Content?oid=2136658

The true underlying function is never really known. So, the accuracy of the method can only be inferred from the error rates on new data examples. The goal of many of the ML algorithms (really a set of algorithms) is to iteratively find the right combination of levers to minimize these error rates. If the resulting ML model does a good job of this, then it is said to generalize well. More on this will be explained later.

Any ML model can be viewed as having three interrelated components:

• Representation
• Evaluation
• Optimization

We will cover each in the next sections.

Representation

Representation is how a model is formally constructed in a way which a computer can interpret. Examples are decision trees, support vector machines, and neural nets. ML models are commonly referred to by the name of the representation. A classifier is an instance in the set of possible models generated by the representation. When you make the choice of which representation to use, you are determining the possibilities of classifiers that your model is able to learn. The range of possibilities is known as the hypothesis space.

If the true (and remember, unknown) classifier model is not in the hypothesis space, it cannot be learned. Most representation models you will use have a large hypothesis space, so this is probably not going to be a problem. But you should be aware of it, as you may need to expand your choices of representation models in order to expand the collective hypothesis space. This may be necessary if you are getting poor predictive performance from your normal go-to set of ML models.

Evaluation

As the ML model tunes itself, there needs to be a way to evaluate how well it is doing. There needs to be a function that measures the performance to know which classifiers are good and which are bad. This is where the evaluation function comes into play. Some examples are accuracy, error rate, precisi...

Table of contents

• Title Page
• Copyright
• Credits
• About the Author
• About the Reviewer
• www.PacktPub.com
• Customer Feedback
• Preface
• Defining IoT Analytics and Challenges
• IoT Devices and Networking Protocols
• IoT Analytics for the Cloud
• Creating an AWS Cloud Analytics Environment
• Collecting All That Data - Strategies and Techniques
• Getting to Know Your Data - Exploring IoT Data
• Decorating Your Data - Adding External Datasets to Innovate
• Communicating with Others - Visualization and Dashboarding
• Applying Geospatial Analytics to IoT Data
• Data Science for IoT Analytics
• Strategies to Organize Data for Analytics
• The Economics of IoT Analytics
• Bringing It All Together