Big Data

Understanding How Data Powers Big Business

Bill Schmarzo

• Publisher: Wiley
• Year: 2013
• ISBN:
  9781118740002
• Edition: 1
• Topic: Business
• Subtopic: Information Management
• Index: Business

Leverage big data to add value to your business

Social media analytics, web-tracking, and other technologies help companies acquire and handle massive amounts of data to better understand their customers, products, competition, and markets. Armed with the insights from big data, companies can improve customer experience and products, add value, and increase return on investment. The tricky part for busy IT professionals and executives is how to get this done, and that's where this practical book comes in. Big Data: Understanding How Data Powers Big Business is a complete how-to guide to leveraging big data to drive business value.

Full of practical techniques, real-world examples, and hands-on exercises, this book explores the technologies involved, as well as how to find areas of the organization that can take full advantage of big data.

• Shows how to decompose current business strategies in order to link big data initiatives to the organization's value creation processes
• Explores different value creation processes and models
• Explains issues surrounding operationalizing big data, including organizational structures, education challenges, and new big data-related roles
• Provides methodology worksheets and exercises so readers can apply techniques
• Includes real-world examples from a variety of organizations leveraging big data

Big Data: Understanding How Data Powers Big Business is written by one of Big Data's preeminent experts, William Schmarzo. Don't miss his invaluable insights and advice.

Chapter 1

The Big Data Business Opportunity

Every now and then, new sources of data emerge that hold the potential to transform how organizations drive, or derive, business value. In the 1980s, we saw point-of-sale (POS) scanner data change the balance of power between consumer package goods (CPG) manufacturers like Procter & Gamble, Unilever, Frito Lay, and Kraft—and retailers like Walmart, Tesco, and Vons. The advent of detailed sources of data about product sales, soon coupled with customer loyalty data, provided retailers with unique insights about product sales, customer buying patterns, and overall market trends that previously were not available to any player in the CPG-to-retail value chain. The new data sources literally changed the business models of many companies.

Then in the late 1990s, web clicks became the new knowledge currency, enabling online merchants to gain significant competitive advantage over their brick-and-mortar counterparts. The detailed insights buried in the web logs gave online merchants new insights into product sales and customer purchase behaviors, and gave online retailers the ability to manipulate the user experience to influence (through capabilities like recommendation engines) customers' purchase choices and the contents of their electronic shopping carts. Again, companies had to change their business models to survive.

Today, we are in the midst of yet another data-driven business revolution. New sources of social media, mobile, and sensor or machine-generated data hold the potential to rewire an organization's value creation processes. Social media data provide insights into customer interests, passions, affiliations, and associations that can be used to optimize your customer engagement processes (from customer acquisition, activation, maturation, up-sell/cross-sell, retention, through advocacy development). Machine or sensor-generated data provide real-time data feeds at the most granular level of detail that enable predictive maintenance, product performance recommendations, and network optimization. In addition, mobile devices enable location-based insights and drive real-time customer engagement that allow brick-and-mortar retailers to compete directly with online retailers in providing an improved, more engaging customer shopping experience.

The massive volumes (terabytes to petabytes), diversity, and complexity of the data are straining the capabilities of existing technology stacks. Traditional data warehouse and business intelligence architectures were not designed to handle petabytes of structured and unstructured data in real-time. This has resulted in the following challenges to both IT and business organizations:

• Rigid business intelligence, data warehouse, and data management architectures are impeding the business from identifying and exploiting fleeting, short-lived business opportunities.
• Retrospective reporting using aggregated data in batches can't leverage new analytic capabilities to develop predictive recommendations that guide business decisions.
• Social, mobile, or machine-generated data insights are not available in a timely manner in a world where the real-time customer experience is becoming the norm.
• Data aggregation and sampling destroys valuable nuances in the data that are key to uncovering new customer, product, operational, and market insights.

This blitz of new data has necessitated and driven technology innovation, much of it being powered by open source initiatives at digital media companies like Google (Big Table), Yahoo! (Hadoop), and Facebook (Hive and HBase), as well as universities (like Stanford, UC Irvine, and MIT). All of these big data developments hold the potential to paralyze businesses if they wait until the technology dust settles before moving forward. For those that wait, only bad things can happen:

• Competitors innovate more quickly and are able to realize compelling cost structure advantages.
• Profits and margins degenerate because competitors are able to identify, capture, and retain the most valuable customers.
• Market share declines result from not being able to get the right products to market at the right time for the right customers.
• Missed business opportunities occur because competitors have real-time listening devices rolling up real-time customer sentiment, product performance problems, and immediately-available monetization opportunities.

The time to move is now, because the risks of not moving can be devastating.

The Business Transformation Imperative

The big data movement is fueling a business transformation. Companies that are embracing big data as business transformational are moving from a retrospective, rearview mirror view of the business that uses partial slices of aggregated or sampled data in batch to monitor the business to a forward-looking, predictive view of operations that leverages all available data—including structured and unstructured data that may sit outside the four walls of the organization—in real-time to optimize business performance (see Table 1.1).

Table 1.1 Big Data Is About Business Transformation

Today's Decision Making	Big Data Decision Making
“Rearview Mirror” hindsight	“Forward looking” recommendations
Less than 10% of available data	Exploit all data from diverse sources
Batch, incomplete, disjointed	Real time, correlated, governed
Business Monitoring	Business Optimization

Think of this as the advent of the real-time, predictive enterprise!

In the end, it's all about the data. Insight-hungry organizations are liberating the data that is buried deep inside their transactional and operational systems, and integrating that data with data that resides outside the organization's four walls (such as social media, mobile, service providers, and publicly available data). These organizations are discovering that data—and the key insights buried inside the data—has the power to transform how organizations understand their customers, partners, suppliers, products, operations, and markets. In the process, leading organizations are transforming their thinking on data, transitioning from treating data as an operational cost to be minimized to a mentality that nurtures data as a strategic asset that needs to be acquired, cleansed, transformed, enriched, and analyzed to yield actionable insights. Bottom-line: companies are seeking ways to acquire even more data that they can leverage throughout the organization's value creation processes.

Walmart Case Study

Data can transform both companies and industries. Walmart is famous for their use of data to transform their business model.

The cornerstone of his [Sam Walton's] company's success ultimately lay in selling goods at the lowest possible price, something he was able to do by pushing aside the middlemen and directly haggling with manufacturers to bring costs down. The idea to “buy it low, stack it high, and sell it cheap” became a sustainable business model largely because Walton, at the behest of David Glass, his eventual successor, heavily invested in software that could track consumer behavior in real time from the bar codes read at Walmart's checkout counters.

He shared the real-time data with suppliers to create partnerships that allowed Walmart to exert significant pressure on manufacturers to improve their productivity and become ever more efficient. As Walmart's influence grew, so did its power to nearly dictate the price, volume, delivery, packaging, and quality of many of its suppliers' products. The upshot: Walton flipped the supplier-retailer relationship upside down.¹

Walmart up-ended the balance of power in the CPG-to-retailer value chain. Before they had access to detailed POS scanner data, the CPG manufacturers (such as Procter & Gamble, Unilever, Kimberley Clark, and General Mills,) dictated to the retailers how much product they would be allowed to sell, at what prices, and using what promotions. But with access to customer insights that could be gleaned from POS data, the retailers were now in a position where they knew more about their customers' behaviors—what products they bought, what prices they were willing to pay, what promotions worked the most effectively, and what products they tended to buy in the same market basket. Add to this information the advent of the customer loyalty card, and the retailers knew in detail what products at what prices under what promotions appealed to which customers. Soon, the retailers were dictating terms to the CPG manufacturers—how much product they wanted to sell (demand-based forecasting), at what prices (yield and price optimization), and what promotions they wanted (promotional effectiveness). Some of these retailers even went one step further and figured out how to monetize their POS data by selling it back to the CPG manufacturers. For example, Walmart provides a data service to their CPG manufacturer partners, called Retail Link, which provides sales and inventory data on the manufacturer's products sold through Walmart.

Across almost all organizations, we are seeing multitudes of examples where data coupled with advanced analytics can transform key organizational business processes, such as:

• Procurement: Identify which suppliers are most cost-effective in delivering products on-time and without damages.
• Product Development: Uncover product usage insights to speed product development processes and improve new product launch effectiveness.
• Manufacturing: Flag machinery and process variances that might be indicators of quality problems.
• Distribution: Quantify optimal inventory levels and optimize supply chain activities based on external factors such as weather, holidays, and economic conditions.
• Marketing: Identify which marketing promotions and campaigns are most effective in driving customer traffic, engagement, and sales, or use attribution analysis to optimize marketing mixes given marketing goals, customer behaviors, and channel behaviors.
• Pricing and Yield Management: Optimize prices for “perishable” goods such as groceries, airline seats, concert tickets and fashion merchandise.
• Merchandising: Optimize merchandise markdown based on current buying patterns, inventory levels, and product interest insights gleaned from social media data.
• Sales: Optimize sales resource assignments, product mix, commissions modeling, and account assignments.
• Store Operations: Optimize inventory levels given predicted buying patterns coupled with local demographic, weather, and events data.
• Human Resources: Identify the characteristics and behaviors of your most successful and effective employees.

The Big Data Business Model Maturity Index

Customers often ask me:

• How far can big data take us from a business perspective?
• What could the ultimate endpoint look like?
• How do I compare to others with respect to my organization's adoption of big data as a business enabler?
• How far can I push big data to power—or even transform—my value creation processes?

To help address these types of questions, I've created the Big Data Business Model Maturity Index. This index provides a benchmark against which organizations can measure themselves as they look at what big data-enabled opportunities may lay ahead. Organizations can use this index to:

• Get an idea of where they stand with respect to exploiting big data and advanced analytics to power their value creation processes and business models (their current state).
• Identify where they want to be in the future (their desired state).

Organizations are moving at different paces with respect to how they are adopting big data and advanced analytics to create competitive advantages for themselves. Some organizations are moving very cautiously because they are unclear where and how to start, and which of the bevy of new technology innovations they need to deploy in order to start their big data journeys. Others are moving at a more aggressive pace to integrate big data and advanced analytics into their existing business processes in order to improve their organizational decision-making capabilities.

However, a select few are looking well beyond just improving their existing business processes with big data. These organizations are aggressively looking to identify and exploit new data monetization opportunities. That is, they are seeking out business opportunities where they can either sell their data (coupled with analytic insights) to others, integrate advanced analytics into their products to create “intelligent” products, or leverage the insights from big data to transform their customer relationships and customer experience.

Le...