Big Data, Big Analytics

Emerging Business Intelligence and Analytic Trends for Today's Businesses

Michael Minelli, Michele Chambers, Ambiga Dhiraj

• Publisher: Wiley
• Year: 2012
• Print ISBN: 9781118147603
• eBook ISBN: 9781118239155
• Edition: 1
• Topic: Business
• Subtopic: Business General
• Index: Business

Unique prospective on the big data analytics phenomenon for both business and IT professionals

The availability of Big Data, low-cost commodity hardware and new information management and analytics software has produced a unique moment in the history of business. The convergence of these trends means that we have the capabilities required to analyze astonishing data sets quickly and cost-effectively for the first time in history. These capabilities are neither theoretical nor trivial. They represent a genuine leap forward and a clear opportunity to realize enormous gains in terms of efficiency, productivity, revenue and profitability.

The Age of Big Data is here, and these are truly revolutionary times. This timely book looks at cutting-edge companies supporting an exciting new generation of business analytics.

• Learn more about the trends in big data and how they are impacting the business world (Risk, Marketing, Healthcare, Financial Services, etc.)
• Explains this new technology and how companies can use them effectively to gather the data that they need and glean critical insights
• Explores relevant topics such as data privacy, data visualization, unstructured data, crowd sourcing data scientists, cloud computing for big data, and much more.

Chapter 1
What Is Big Data and Why Is It Important?

Big Data is the next generation of data warehousing and business analytics and is poised to deliver top line revenues cost efficiently for enterprises. The greatest part about this phenomenon is the rapid pace of innovation and change; where we are today is not where we’ll be in just two years and definitely not where we’ll be in a decade.

Just think about all the great stories you will tell your grandchildren about the early days of the twenty-first century, when the Age of Big Data Analytics was in its infancy.

This new age didn’t suddenly emerge. It’s not an overnight phenomenon. It’s been coming for a while. It has many deep roots and many branches. In fact, if you speak with most data industry veterans, Big Data has been around for decades for firms that have been handling tons of transactional data over the years—even dating back to the mainframe era. The reasons for this new age are varied and complex, so let’s reduce them to a handful that will be easy to remember in case someone corners you at a cocktail party and demands a quick explanation of what’s really going on. Here’s our standard answer in three parts:

1. Computing perfect storm. Big Data analytics are the natural result of four major global trends: Moore’s Law (which basically says that technology always gets cheaper), mobile computing (that smart phone or mobile tablet in your hand), social networking (Facebook, Foursquare, Pinterest, etc.), and cloud computing (you don’t even have to own hardware or software anymore; you can rent or lease someone else’s).
2. Data perfect storm. Volumes of transactional data have been around for decades for most big firms, but the flood gates have now opened with more volume, and the velocity and variety—the three Vs—of data that has arrived in unprecedented ways. This perfect storm of the three Vs makes it extremely complex and cumbersome with the current data management and analytics technology and practices.
3. Convergence perfect storm. Another perfect storm is happening, too. Traditional data management and analytics software and hardware technologies, open-source technology, and commodity hardware are merging to create new alternatives for IT and business executives to address Big Data analytics.

Let’s make one thing clear. For some industry veterans, “Big Data” isn’t new. There are companies that have dealt with billions of transactions for many years. For example, John Meister, group executive of Data Warehouse Technologies at MasterCard Worldwide, deals with a billion transactions on a strong holiday weekend. However, even the most seasoned IT veterans are awestruck by recent innovations that give their team the ability to leverage new technology and approaches, which enable us to affordably handle more data and take advantage of the variety of data that lives outside of the typical transactional world—such as unstructured data.

Paul Kent, vice president of Big Data at SAS, is an R&D professional who has developed big data crunching software for over two decades. At the SAS Global Forum 2012, Kent explained that the ability to store data in an affordable way has changed the game for his customers:

People are able to store that much data now and more than they ever before. We have reached this tipping point where they don’t have to make decisions about which half to keep or how much history to keep. It’s now economically feasible to keep all of your history and all of your variables and go back later when you have a new question and start looking for an answer. That hadn’t been practical up until just recently. Certainly the advances in blade technology and the idea that Google brought to market of you take lots and lots of small Intel servers and you gang them together and use their potential in aggregate. That is the super computer of the future.

Let’s now introduce Misha Ghosh, who is known to be an innovator with several patents under his belt. Ghosh is currently an executive at MasterCard Advisors and before that he spent 11 years at Bank of America solving business issues by using data. Ghosh explains, “Aside from the changes in the actual hardware and software technology, there has also been a massive change in the actual evolution of data systems. I compare it to the stages of learning: dependent, independent, and interdependent.”

Using Misha’s analogy, let’s breakdown the three pinnacle stages in the evolution of data systems:

• Dependent (Early Days). Data systems were fairly new and users didn’t know quite know what they wanted. IT assumed that “Build it and they shall come.”
• Independent (Recent Years). Users understood what an analytical platform was and worked together with IT to define the business needs and approach for deriving insights for their firm.
• Interdependent (Big Data Era). Interactional stage between various companies, creating more social collaboration beyond your firm’s walls.

Moving from independent (Recent Years) to interdependent (Big Data Era) is sort of like comparing Starbucks to a hip independent neighborhood coffee shop with wizard baristas that can tell you when the next local environmental advisory council meet-up is taking place. Both shops have similar basic product ingredients, but the independent neighborhood coffee shop provides an approach and atmosphere that caters to social collaboration within a given community. The customers share their artwork and tips about the best picks at Saturday’s farmers market as they stand by the giant corkboard with a sea of personal flyers with tear off tabs . . . “Web Designer Available for Hire, 555-1302.”

One relevant example and Big Data parity to the coffee shop is the New York City data meet-ups with data scientists like Drew Conway, Jared Lander, and Jake Porway. These bright minds organize meet-ups after work at places like Columbia University and NYU to share their latest analytic application [including a review of their actual code] followed by a trip to the local pub for a few pints and more data chatter. Their use cases are a blend of Big Data corporate applications and other applications that actually turn their data skills into a helping hand for humanity.

For example, during the day Jared Lander helps a large healthcare organization solve big data problems related to patient data. By night, he is helping a disaster recovery organization with optimization analytics that help direct the correct supplies to areas where they are needed most. Does a village need bottled water or boats, rice or wheat, shelter or toilets? Follow up surveys six, 12, 18, and 24 months following the disaster help track the recovery and direct further relief efforts.

Another great example is Jake Porway, who decided to go full time to use Big Data to help humanity at DataKind, which is the company he co-founded with Craig Barowsky and Drew Conway. From weekend events to long-term projects, DataKind supports a data-driven social sector through services, tools, and educational resources to help with the entire data pipeline.

In the service of humanity, they were able to secure funding from several corporations and foundations such as EMC, O’Reilly Media, Pop Tech, National Geographic, and the Alfred P. Sloan Foundation. Porway described DataKind to us as a group of data superheroes:

I love superheroes, because they’re ordinary people who find themselves with extraordinary powers that they use to make the world a better place. As data and technology become more ubiquitous and the need for insights more pressing, ordinary data scientists are finding themselves with extraordinary powers. The world is changing and those who are stepping up to use data for the greater good have a real opportunity to change it for the better.

In summary, the Big Data world is being fueled with an abundance mentality; a rising tide lifts all boats. This new mentality is fueled by a gigantic global corkboard that includes data scientists, crowd sourcing, and opens source methodologies.

A Flood of Mythic “Start-Up” Proportions

Thanks to the three converging “perfect storms,” those trends discussed in the previous section, the global economy now generates unprecedented quantities of data. People who compare the amount of data produced daily to a deluge of mythic proportions are entirely correct. This flood of data represents something we’ve never seen before. It’s new, it’s powerful, and yes, it’s scary but extremely exciting.

The best way to predict the future is to create it!

—Peter F. Drucker

The influential writer and management consultant Drucker reminds us that the future is up to us to create. This is something that every entrepreneur takes to heart as they evangelize their start-up’s big idea that they ...

Table of contents

• Cover
• Table of Contents
• Title
• Copyright
• Dedication
• Foreword
• Preface
• Acknowledgments
• Chapter 1 : What Is Big Data and Why Is It Important?
• Chapter 2: Industry Examples of Big Data
• Chapter 3: Big Data Technology
• Chapter 4: Information Management
• Chapter 5: Business Analytics
• Chapter 6: The People Part of the Equation
• Chapter 7: Data Privacy and Ethics
• Conclusion
• Recommended Resources
• About the Authors
• Index
• End User License Agreement