Apache Hive Essentials

Dayong Du

• Verlag: Packt Publishing
• Jahr: 2015
• ISBN:
  9781783558575
• Thema: Informatica
• Thema: Data mining

Apache Hive Essentials

---

Table of Contents

Apache Hive Essentials

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Overview of Big Data and Hive

A short history

Introducing big data

Relational and NoSQL database versus Hadoop

Batch, real-time, and stream processing

Overview of the Hadoop ecosystem

Hive overview

Summary

2. Setting Up the Hive Environment

Installing Hive from Apache

Installing Hive from vendor packages

Starting Hive in the cloud

Using the Hive command line and Beeline

The Hive-integrated development environment

Summary

3. Data Definition and Description

Understanding Hive data types

Data type conversions

Hive Data Definition Language

Hive database

Hive internal and external tables

Hive partitions

Hive buckets

Hive views

Summary

4. Data Selection and Scope

The SELECT statement

The INNER JOIN statement

The OUTER JOIN and CROSS JOIN statements

Special JOIN – MAPJOIN

Set operation – UNION ALL

Summary

5. Data Manipulation

Data exchange – LOAD

Data exchange – INSERT

Data exchange – EXPORT and IMPORT

ORDER and SORT

Operators and functions

Transactions

Summary

6. Data Aggregation and Sampling

Basic aggregation – GROUP BY

Advanced aggregation – GROUPING SETS

Advanced aggregation – ROLLUP and CUBE

Aggregation condition – HAVING

Analytic functions

Sampling

Summary

7. Performance Considerations

Performance utilities

The EXPLAIN statement

The ANALYZE statement

Design optimization

Partition tables

Bucket tables

Index

Data file optimization

File format

Compression

Storage optimization

Job and query optimization

Local mode

JVM reuse

Parallel execution

Join optimization

Common join

Map join

Bucket map join

Sort merge bucket (SMB) join

Sort merge bucket map (SMBM) join

Skew join

Summary

8. Extensibility Considerations

User-defined functions

The UDF code template

The UDAF code template

The UDTF code template

Development and deployment

Streaming

SerDe

Summary

9. Security Considerations

Authentication

Metastore server authentication

HiveServer2 authentication

Authorization

Legacy mode

Storage-based mode

SQL standard-based mode

Encryption

Summary

10. Working with Other Tools

JDBC / ODBC connector

HBase

Hue

HCatalog

ZooKeeper

Oozie

Hive roadmap

Summary

Index

Apache Hive Essentials

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: February 2015

Production reference: 1210215

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78355-857-5

www.packtpub.com

Credits

Author

Dayong Du

Reviewers

Puneetha B M

Hamzeh Khazaei

Nitin Pradeep Kumar

Balaswamy Vaddeman

Commissioning Editor

Ashwin Nair

Acquisition Editor

Shaon Basu

Content Development Editor

Merwyn D'souza

Technical Editor

Taabish Khan

Copy Editors

Sameen Siddiqui

Laxmi Subramanian

Project Coordinator

Neha Bhatnagar

Proofreaders

Paul Hindle

Jonathan Todd

Indexer

Monica Ajmera Mehta

Production Coordinator

Aparna Bhagat

Cover Work

Aparna Bhagat

About the Author

Dayong Du is a big data practitioner, leader, and developer with expertise in technology consulting, designing, and implementing enterprise big data solutions. With more than 10 years of experience in enterprise data warehouse, business intelligence, and big data and analytics, he has provided his data intelligence expertise in various industries, such as media, travel, telecommunications, and so on. He is currently working with QuickPlay Media in Toronto, Canada, to build enterprise big data intelligence reporting for online media services and content providers. He has a master's degree in computer science from Dalhousie University, and he holds the Cloudera C...