Getting Started with Greenplum for Big Data Analytics

Sunila Gollapudi

• Publisher: Packt Publishing
• Year: 2013
• eBook ISBN: 9781782177043
• Topic: Informatica
• Subtopic: Visualizzazione di dati

In Detail

Organizations are leveraging the use of data and analytics to gain a competitive advantage over their opposition. Therefore, organizations are quickly becoming more and more data driven. With the advent of Big Data, existing Data Warehousing and Business Intelligence solutions are becoming obsolete, and a requisite for new agile platforms consisting of all the aspects of Big Data has become inevitable. From loading/integrating data to presenting analytical visualizations and reports, the new Big Data platforms like Greenplum do it all. It is now the mindset of the user that requires a tuning to put the solutions to work.

"Getting Started with Greenplum for Big Data Analytics" is a practical, hands-on guide to learning and implementing Big Data Analytics using the Greenplum Integrated Analytics Platform. From processing structured and unstructured data to presenting the results/insights to key business stakeholders, this book explains it all.

"Getting Started with Greenplum for Big Data Analytics" discusses the key characteristics of Big Data and its impact on current Data Warehousing platforms. It will take you through the standard Data Science project lifecycle and will lay down the key requirements for an integrated analytics platform. It then explores the various software and appliance components of Greenplum and discusses the relevance of each component at every level in the Data Science lifecycle.

You will also learn Big Data architectural patterns and recap some key advanced analytics techniques in detail. The book will also take a look at programming with R and integration with Greenplum for implementing analytics. Additionally, you will explore MADlib and advanced SQL techniques in Greenplum for analytics. This book also elaborates on the physical architecture aspects of Greenplum with guidance on handling high-availability, back-up, and recovery.

Approach

Standard tutorial-based approach

Who this book is for

"Getting Started with Greenplum for Big Data" Analytics is great for data scientists and data analysts with a basic knowledge of Data Warehousing and Business Intelligence platforms who are new to Big Data and who are looking to get a good grounding in how to use the Greenplum Platform. Its assumed that you will have some experience with database design and programming as well as be familiar with analytics tools like R and Weka.

Getting Started with Greenplum for Big Data Analytics

---

Table of Contents

Getting Started with Greenplum for Big Data Analytics

Credits

Foreword

About the Author

Acknowledgement

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Instant Updates on New Packt Books

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Errata

Piracy

Questions

1. Big Data, Analytics, and Data Science Life Cycle

Enterprise data

Classification

Features

Big Data

So, what is Big Data?

Multi-structured data

Data analytics

Data science

Data science life cycle

Phase 1 – state business problem

Phase 2 – set up data

Phase 3 – explore/transform data

Phase 4 – model

Phase 5 – publish insights

Phase 6 – measure effectiveness

References/Further reading

Summary

2. Greenplum Unified Analytics Platform (UAP)

Big Data analytics – platform requirements

Greenplum Unified Analytics Platform (UAP)

Core components

Greenplum Database

Hadoop (HD)

Chorus

Command Center

Modules

Database modules

HD modules

Data Integration Accelerator (DIA) modules

Core architecture concepts

Data warehousing

Column-oriented databases

Parallel versus distributed computing/processing

Shared nothing, massive parallel processing (MPP) systems, and elastic scalability

Shared disk data architecture

Shared memory data architecture

Shared nothing data architecture

Data loading patterns

Greenplum UAP components

Greenplum Database

The Greenplum Database physical architecture

The Greenplum high-availability architecture

High-speed data loading using external tables

External table types

Polymorphic data storage and historic data management

Data distribution

Hadoop (HD)

Hadoop Distributed File System (HDFS)

Hadoop MapReduce

Chorus

Greenplum Data Computing Appliance (DCA)

Greenplum Data Integration Accelerator (DIA)

References/Further reading

Summary

3. Advanced Analytics – Paradigms, Tools, and Techniques

Analytic paradigms

Descriptive analytics

Predictive analytics

Prescriptive analytics

Analytics classified

Classification

Forecasting or prediction or regression

Clustering

Optimization

Simulations

Modeling methods

Decision trees

Association rules

The Apriori algorithm

Linear regression

Logistic regression

The Naive Bayesian classifier

K-means clustering

Text analysis

R programming

Weka

In-database analytics using MADlib

References/Further reading

Summary

4. Implementing Analytics with Greenplum UAP

Data loading for Greenplum Database and HD

Greenplum data loading options

External tables

gpfdist

gpload

Hadoop (HD) data loading options

Sqoop 2

Greenplum BulkLoader for Hadoop

Using external ETL to load data into Greenplum

Extraction, Load, and Transformation (ELT) and Extraction, Transformation, Load, and Transformation (ETLT)

Greenplum target configuration

Sourcing large volumes of data from Greenplum

Unsupported Greenplum data types

Push Down Optimization (PDO)

Greenplum table distribution and partitioning

Distribution

Data skew and performance

Optimizing the broadcast or redistribution motion for data co-location

Partitioning

Querying Greenplum Database and HD

Querying Greenplum Database

Analyzing and optimizing queries

The ANALYZE function

The EXPLAIN function

Dynamic Pipelining in Greenplum

Querying HDFS

Hive

Pig

Data communication between Greenplum Database and Hadoop (using external tables)

Data Computing Appliance (DCA)

Storage design, disk protection, and fault tolerance

Master server RAID configurations

Segment server RAID configurations

Monitoring DCA

Greenplum Database management

In-database analytics options (Greenplum-specific)

Window functions

The PARTITION BY clause

The ORDER BY clause

The OVER (ORDER BY…) clause

Creating, modifying, and dropping functions

User-defined aggregates

Using R with Greenplum

DBI Connector for R

PL/R

Using Weka with Greenplum

Using MADlib with Greenplum

Using Greenplum Chorus

Pivotal

References/Further reading

Summary

Index

Getting Started with Greenplum for Big Data Analytics

Copyright © 2013 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: October 2013

Production Reference: 1171013

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-...

Table of contents

• Getting Started with Greenplum for Big Data Analytics