Machine Learning for Big Data Analysis

Siddhartha Bhattacharyya, Hrishikesh Bhaumik, Anirban Mukherjee, Sourav De, Siddhartha Bhattacharyya, Hrishikesh Bhaumik, Anirban Mukherjee, Sourav De

• Publisher: De Gruyter
• Year: 2018
• Print ISBN: 9783110550320
• eBook ISBN: 9783110550771
• Edition: 1
• Topic: Computer Science
• Subtopic: Digital Media
• Index: Computer Science

This volume comprises six well-versed contributed chapters devoted to report the latest fi ndings on the applications of machine learning for big data analytics. Big data is a term for data sets that are so large or complex that traditional data processing application software is inadequate to deal with them. The possible challenges in this direction include capture, storage, analysis, data curation, search, sharing, transfer, visualization, querying, updating and information privacy.

Big data analytics is the process of examining large and varied data sets - i.e., big data - to uncover hidden patterns, unknown correlations, market trends, customer preferences and other useful information that can help organizations make more-informed business decisions. This volume is intended to be used as a reference by undergraduate and post graduate students of the disciplines of computer science, electronics and telecommunication, information science and electrical engineering.

THE SERIES: FRONTIERS IN COMPUTATIONAL INTELLIGENCE

The series Frontiers In Computational Intelligence is envisioned to provide comprehensive coverage and understanding of cutting edge research in computational intelligence. It intends to augment the scholarly discourse on all topics relating to the advances in artifi cial life and machine learning in the form of metaheuristics, approximate reasoning, and robotics. Latest research fi ndings are coupled with applications to varied domains of engineering and computer sciences. This field is steadily growing especially with the advent of novel machine learning algorithms being applied to different domains of engineering and technology. The series brings together leading researchers that intend to continue to advance the fi eld and create a broad knowledge about the most recent research.

Till Blesik, Matthias Murawski, Murat Vurucu, and Markus Bick

1Applying big data analytics to psychometric micro-targeting

Till Blesik, Matthias Murawski, Murat Vurucu, Markus Bick, ESCP Europe Business School, Heubnerweg 8–10, 14059 Berlin, Germany, e-mails: {tblesik, mmurawski, mbick}@escpeurope.eu, [email protected]

Abstract: In this chapter we link two recent phenomena. First, innovations in technology have lowered the cost of data storage and enabled scalable parallel computing. Connected with social media, the Internet of Things applications and other sources, large data sets can easily be collected. These data sets are the basis for greatly improving our understanding of individuals and group dynamics. Second, events such as the election of Donald J. Trump as President of the United States of America and the exit of Great Britain from the European Union have shaped public debates on the influence of psychometric micro-targeting of voters. Generally, public authorities, but also other organizations, have a very high demand for information about individuals.

We combine these two streams, meaning the enormous amounts of data available and the demand for micro-targeting, aiming at answering the following question: How can big data analytics be used for psychometric profiling? We develop a conceptual framework of how Facebook data might be used to derive the psychometric traits of an individual user. Our conceptual framework includes the Facebook Graph API, a nonSQL Mongo Data Base for information storage and R scripts to reduce the dimensionality of large data sets by applying the latent Dirichlet allocation to determine correlations between reduced information with psychologically relevant words.

In this chapter we provide a hands-on introduction to psychometric trait analysis and present a scalable infrastructure solution as a proof of concept for the concepts presented here. We discuss two use cases and show how psychometric information, which could, for example, be used for targeted political messages, can be derived from Facebook data. Finally, potential further developments are outlined that could serve as starting points for future research.

Keywords: Big data, Big Five personality traits, Facebook, Politics, Psychometrics, Latent Dirichlet allocation

1.1Introduction

Technological innovations in the twentieth and twenty-first centuries have had immense impacts on society. The emergence of the Internet and the resulting permanent connectivity of individuals has changed not only the economy but also the way society functions in general [1]. Before the infiltration of measurable online actions, data were scarce. That is why statistical inference, analysing a data set derived from a larger population, was and still is very important. It helps make the best of scarce and expensive data.

Now, in the age of social media, the Internet of Things, e-commerce, online financial services, search engines, navigation systems and cloud computing, data are being collected from every individual to machines and processed by machine-to-machine interactions [2]. These data can be analysed in terms of joint correlations, which generates even more data, called meta-data. A single smartphone alone is already able to provide information about the purchasing habits, transportation preferences and routes, personal preferences and social surroundings of individual users.

It is not far-fetched to imagine how public authorities use their applications to listen to spoken words and translate them to written and, therefore, searchable text, mapping all movements and patterns of an individual and using correlations of in-between behaviour and preferences collected from social networks to identify, for instance, potential threats to the state. All of the international invasions of privacy carried out by the US National Security Agency (NSA) and similar organizations have shaped the public debate and our perception of technology dramatically since the Edward Snowden leaks in 2013. An Orwellian fantasy of mass surveillance seems to have become a reality in the shape of modern government.

Secret services are, however, also arguably relevant institutions within democracies. Based on a judicial system that follows a democratic constitution and aims at protecting a nation’s interests, secret services build their operations on judges and laws and usually use their applications in a political context of checks and balances to categorize and control enemies of the state. The nation’s “interest” and the nation’s “enemies” are, however, terms whose definitions greatly depend on the ideas of the current elected government. In fact, given the technology already in place, today’s governments are potentially able to build “psychometric” profiles of every single voter so as to influence their voting habits [3].

Psychometrics is a scientific approach to assessing the psychological traits of people. It has its roots in sociobiology [4]. The goal is to obtain a distribution within the population of each of the personality traits of the Big Five personality test. The Big Five traits are extraversion, agreeableness, openness to experience, conscientiousness and emotional stability/neuroticism [5, 6]. There are various ways to explore individual traits, for example by analysing the Facebook likes of a user and other forms of written texts such as status messages. This information can be connected to other demographic data such as age or gender. In the context of politics, messages sent to voters can potentially be adapted, based on the results of a psychometric analysis. A governor advocating lax gun laws would address a young mother in a different way than a gun enthusiast in the National Rifle Association (NRA). The young mother might receive a message advocating lax gun laws so that teachers can carry guns in educational settings to protect her children, while an NRA member would receive a message demonstrating the newest features of a military weapon that should be legalized.

Understanding the psychometric traits of each recipient can help define the content of a message. Psychometric analysis has been used for over a century, but in the context of big data and mass surveillance, it has gained new importance [4].

Based on these thoughts, we will investigate the opportunities for deriving the psychometric traits of individual users from Facebook data. More precisely, the research question can be summarized as follows:

How can Facebook data be extracted, stored, analysed, presented and used for micro-targeting within the Big Five model?

To answer this question, several data sources and calculations are used, as shown in Figure 1.1.

Fig. 1.1: Conceptual flowchart of this chapter

This chapter is structured as follows. The second section introduces the theoretical and historical foundations of psychometric analysis. The third section presents the research methodology while placing a specific focus on both the underlying statistical and technical infrastructure, including a presentation of how Facebook can be used as a data source for our study. We test our conceptual framework in Section 1.4, which includes two use cases covering the preparation of data, the extraction of patterns and corresponding final results. This chapter ends with a discussion of our results, the limitations of our approach and some concluding remarks.

1.2Psychometrics

This section contains a theoretical overview of the topic of psychometrics. We present its historical emergence and briefly mention some ethical issues related to psychometrics before two general schools, the functional and the trait schools, are discussed. Then the concept of the Big Five personality traits will be introduced. We will show how the Big Five traits are linked to politics and provide some recent research results on this linkage. This section ends with a presentation of new opportunities for psychometric assessment in an increasingly digital world.

1.2.1Historical emergence and ethical issues

Psychometrics is the science of psychological assessment. In this chapter, we mostly refer to the book by Rust and Golombok (2009), Modern Psychometrics – The Science of Psychological Assessment [4]. This book provides a comprehensive overview of psychometrics as well as a discussion of several practical aspects of the topic. Furthermore, important historical and ethical issues are presented. We summarize them in this subsection.

Generally, psychological assessment has diverse goals. Tests can potentially aim at recruiting the ideal candidates for a job or to create equality in educational settings by identifying learning disorders. Another controversial function is the goal of using psychometric profiling to build micro-targeted advertising to influence voting habits in democratic elections [3].

The roots of psychometrics reach back long before Darwin’s famous publications On the Origin of Species and The Descent of Man. Talent was assumed to be a divine gift that depends on the mere judgment and p...

Table of contents

• Cover
• Title Page
• Copyright
• Preface
• Dedication
• Contents
• 1 Applying big data analytics to psychometric micro-targeting
• 2 Keyframe selection for video indexing using an approximate minimal spanning tree
• 3 Deep learning techniques for image processing
• 4 Connecting cities using smart transportation: an overview
• 5 Model of intellectual analysis of multidimensional semi-structured data based on deep neuro-fuzzy networks
• 6 Image fusion in remote sensing based on sparse sampling method and PCNN techniques
• Index