Intelligent Data Analysis

From Data Gathering to Data Comprehension

Deepak Gupta, Siddhartha Bhattacharyya, Ashish Khanna, Kalpna Sagar, Deepak Gupta, Siddhartha Bhattacharyya, Ashish Khanna, Kalpna Sagar

• Publisher: Wiley
• Year: 2020
• Print ISBN: 9781119544456
• eBook ISBN: 9781119544463
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Electrical Engineering & Telecommunications
• Index: Technology & Engineering

This book focuses on methods and tools for intelligent data analysis, aimed at narrowing the increasing gap between data gathering and data comprehension, and emphasis will also be given to solving of problems which result from automated data collection, such as analysis of computer-based patient records, data warehousing tools, intelligent alarming, effective and efficient monitoring, and so on. This book aims to describe the different approaches of Intelligent Data Analysis from a practical point of view: solving common life problems with data analysis tools.

1
Intelligent Data Analysis: Black Box Versus White Box Modeling

Sarthak Gupta, Siddhant Bagga, and Deepak Kumar Sharma

Division of Information Technology, Netaji Subhas University of Technology, New Delhi, India

1.1 Introduction

In the midst of all of the societal challenges of today's world, digital transformation is rapidly becoming a necessity. The number of internet users is growing at an unprecedented rate. New devices, sensors, and technologies are emerging every day. These factors have led to an exponential increase in the volume of data being generated. According to a recent research [1], users of the internet generate 2.5 quintillion bytes of data per day.

1.1.1 Intelligent Data Analysis

Data is only as good as what you make of it. The sheer amount of data being generated calls for methods to leverage its power. With the proper tools and methodologies, data analysis can improve decision making, lower the risks, and unearth hidden insights. Intelligent data analysis (IDA) is concerned with effective analysis of data [2, 3].

The process of IDA consists of three main steps (see Figure 1.1):

1. Data collection and preparation: This step involves acquiring data, and converting it into a format suitable for further analysis. This may involve storing the data as a table, taking care of empty or null values, etc.
2. Exploration: Before a thorough analysis can be performed on the data, certain characteristics are examined like number of data points, included variables, statistical features, etc. Data exploration allows analysts to get familiar with the dataset, and create prospective hypotheses. Visualization is extensively used in this step. Various visualization techniques will be discussed in depth later in this chapter.
3. Analysis: Various machine learning and deep learning algorithms are applied at this step. Data analysts build models that try to find the best possible fit to the data points. These models can be classified as white box or black box models.

A more comprehensive introduction to data analysis can be found in prior pieces of literature [4–6].

Figure 1.1 Data analysis process.

1.1.2 Applications of IDA and Machine Learning

IDA and machine learning can be applied to a multitude of products and services, since these models have the ability to make fast, data-driven decisions at scale. We're surrounded by live examples of machine learning in things we use in day-to-day life.

A primary example is web page ranking [7, 8]. Whenever we search for anything on a search engine, the results that we get are presented to us in the order of relevance. To achieve this, the search engine needs to “know” which pages are more relevant than others.

A related application is collaborative filtering [9, ...

Table of contents

• Cover
• Table of Contents
• List of Contributors
• Series Preface
• Preface
• 1 Intelligent Data Analysis: Black Box Versus White Box Modeling
• 2 Data: Its Nature and Modern Data Analytical Tools
• 3 Statistical Methods for Intelligent Data Analysis: Introduction and Various Concepts
• 4 Intelligent Data Analysis with Data Mining: Theory and Applications
• 5 Intelligent Data Analysis: Deep Learning and Visualization
• 6 A Systematic Review on the Evolution of Dental Caries Detection Methods and Its Significance in Data Analysis Perspective
• 7 Intelligent Data Analysis Using Hadoop Cluster – Inspired MapReduce Framework and Association Rule Mining on Educational Domain
• 8 Influence of Green Space on Global Air Quality Monitoring: Data Analysis Using K-Means Clustering Algorithm
• 9 IDA with Space Technology and Geographic Information System
• 10 Application of Intelligent Data Analysis in Intelligent Transportation System Using IoT
• 11 Applying Big Data Analytics on Motor Vehicle Collision Predictions in New York City
• 12 A Smart and Promising Neurological Disorder Diagnostic System: An Amalgamation of Big Data, IoT, and Emerging Computing Techniques
• 13 Comments-Based Analysis of a Bug Report Collection System and Its Applications
• 14 Sarcasm Detection Algorithms Based on Sentiment Strength
• 15 SNAP: Social Network Analysis Using Predictive Modeling
• 16 Intelligent Data Analysis for Medical Applications
• 17 Bruxism Detection Using Single-Channel C4-A1 on Human Sleep S2 Stage Recording
• 18 Handwriting Analysis for Early Detection of Alzheimer's Disease
• Index
• End User License Agreement