Big Data Analytics

Tools and Technology for Effective Planning

Arun K. Somani, Ganesh Chandra Deka, Arun K. Somani, Ganesh Chandra Deka

• Publisher: Chapman and Hall/CRC
• Year: 2017
• ISBN:
  9781315391243
• Edition: 1
• Topic: Informatique
• Subtopic: Bases de données

The proposed book will discuss various aspects of big data Analytics. It will deliberate upon the tools, technology, applications, use cases and research directions in the field. Chapters would be contributed by researchers, scientist and practitioners from various reputed universities and organizations for the benefit of readers.

1Challenges in Big Data

Pothireddy Venkata Lakshmi Narayana Rao, Pothireddy Siva Abhilash, and PS Pavan Kumar

Introduction

Background

Goals and Challenges of Analyzing Big Data

Paradigm Shifts

Organization of This Paper

Algorithms for Big Data Analytics

k-Means

Classification Algorithms: k-NN

Application of Big Data: A Case Study

Economics and Finance

Other Applications

Salient Features of Big Data

Heterogeneity

Noise Accumulation

Spurious Correlation

Coincidental Endogeneity

Impact on Statistical Thinking

Independence Screening

Dealing with Incidental Endogeneity

Impact on Computing Infrastructure

Literature Review

MapReduce

Cloud Computing

Impact on Computational Methods

First-Order Methods for Non-Smooth Optimization

Dimension Reduction and Random Projection

Future Perspectives and Conclusion

Existing Methods

Proposed Methods

Probabilistic Graphical Modeling

Mining Twitter Data: From Content to Connections

Late Work: Location-Specific Tweet Detection and Topic Summarization in Twitter

Tending to Big Data Challenges in Genome Sequencing and RNA Interaction Prediction

Single-Cell Genome Sequencing

RNA Structure and RNA–RNA Association Expectation

Identifying Qualitative Changes in Living Systems

Acknowledgments

References

Additional References for Researchers and Advanced Readers for Further Reading

Key Terminology and Definitions

Introduction

Enormous data guarantee new levels of investigative disclosure and financial quality. What is new about Big Data and how they vary from the conventional little or medium-scale information? This paper outlines the open doors and difficulties brought by Big Data, with accentuation on the recognized elements of Big Data and measurable and computational technique and in addition registering engineering to manage them.

Background

We are entering the time of Big Data, a term that alludes to the blast of data now accessible. Such a Big Data development is driven by the way that gigantic measures of high-dimensional or unstructured information are consistently delivered and are presented in a much less “luxurious” format than they used to be. For instance, in genomics we have seen an enormous drop in costs for sequencing of an entire genome [1]. This is likewise valid in many different scientific areas, for example, online network examination, biomedical imaging, high-recurrence money transactions, investigation of reconnaissance recordings, and retail deals. The current pattern for these vast amounts of information to be delivered and stored in an inexpensive manner is likely to keep up or even quicken in the future [2]. This pattern will have a profound effect on science, designing, and business. For instance, logical advances are turning out to be increasingly information driven, and specialists will increasingly consider themselves customers of information. The monstrous measures of high-dimensional information convey both open doors and new difficulties to information examination. Substantial measurable investigations for Big Data handling are turning out to be progressively essential.

Goals and Challenges of Analyzing Big Data

What are the purposes of violation depressed Big Data? As per Fan and Lu [3], two principal objectives of high-dimensional information investigation are to create powerful strategies that can precisely anticipate the future perceptions and in the meantime gain understanding into the relationship between the elements and reactions for experimental purposes. In addition, because of the extensive specimen size, Big Data offers an ascent to two more objectives: to comprehend heterogeneity and shared traits across various subpopulations.

At the end of the day, Big Data gives guarantees for:

1. Investigating the shrouded structures of every subpopulation of the information, which is generally not possible and may even be dealt with as “exceptions” when the specimen size is small; and
2. Extricating imperative regular elements across numerous subpopulations notwithstanding the expansive individual varieties of data.

What are the difficulties of investigating Big Data? Big Data is portrayed by high dimensionality and substantial specimen size. These two elements raise three one-of-a-kind difficulties:

1. High dimensionality brings clamor gathering, spurious relationships, and coincidental homogeneity;
2. High dimensionality consolidated with vast specimen size brings additional considerations, for example, regarding substantial computational expense and algorithmic flimsiness;
3. The gigantic examples in Big Data are regularly totaled from various sources at various times, utilizing distinctive advances. This creates issues regarding heterogeneity, trial varieties, and factual predispositions and obliges us to employ more versatile and hardy methodologies.

Paradigm Shifts

To handle the troubles of Big Data, we require new quantifiable derivation and computational techniques. As an example, various standard systems that perform well for moderate test sizes don’t scale to enormous amounts of data. Basically, various truthful methodologies that perform well for low-dimensional data are going up against basic troubles in separating high-dimensional data. To plot effective, truthful strategies for exploring and anticipating Big Data, we need to address Big Data issues, for instance, heterogeneity, hullabaloo gathering, spurious connections, and fortuitous endogeneity, despite changing the quantifiable precision and computational profitability.

With respect to exactness, estimation diminishment, and variable determination are critical parts in exploring high-dimensional data. We will address these disturbing, building issues. As a case in point, in a high-dimensional portrayal, Fan and Fan [4] and Pittelkow and Ghosh [5] reported ...