Apache Mahout Essentials

Jayani Withanawasam

• Éditeur: Packt Publishing
• Année: 2015
• ISBN:
  9781783554997
• Édition: 1
• Sujet: Ciencia de la computación
• Sous-sujet: Programación en Java

Apache Mahout Essentials

---

Table of Contents

Apache Mahout Essentials

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. Introducing Apache Mahout

Machine learning in a nutshell

Features

Supervised learning versus unsupervised learning

Machine learning applications

Information retrieval

Business

Market segmentation (clustering)

Stock market predictions (regression)

Health care

Using a mammogram for cancer tissue detection

Machine learning libraries

Open source or commercial

Scalability

Languages used

Algorithm support

Batch processing versus stream processing

The story so far

Apache Mahout

Setting up Apache Mahout

How Apache Mahout works?

The high-level design

The distribution

From Hadoop MapReduce to Spark

Problems with Hadoop MapReduce

In-memory data processing with Spark and H2O

Why is Mahout shifting from Hadoop MapReduce to Spark?

When is it appropriate to use Apache Mahout?

Summary

2. Clustering

Unsupervised learning and clustering

Applications of clustering

Computer vision and image processing

Types of clustering

Hard clustering versus soft clustering

Flat clustering versus hierarchical clustering

Model-based clustering

K-Means clustering

Getting your hands dirty!

Running K-Means using Java programming

Data preparation

Understanding important parameters

Cluster visualization

Distance measure

Writing a custom distance measure

K-Means clustering with MapReduce

MapReduce in Apache Mahout

The map function

The reduce function

Additional clustering algorithms

Canopy clustering

Fuzzy K-Means

Streaming K-Means

The streaming step

The ball K-Means step

Spectral clustering

Dirichlet clustering

Text clustering

The vector space model and TF-IDF

N-grams and collocations

Preprocessing text with Lucene

Text clustering with the K-Means algorithm

Topic modeling

Optimizing clustering performance

Selecting the right features

Selecting the right algorithms

Selecting the right distance measure

Evaluating clusters

The initialization of centroids and the number of clusters

Tuning up parameters

The decision on infrastructure

Summary

3. Regression and Classification

Supervised learning

Target variables and predictor variables

Predictive analytics' techniques

Regression-based prediction

Model-based prediction

Tree-based prediction

Classification versus regression

Linear regression with Apache Spark

How does linear regression work?

A real-world example

The impact of smoking on mortality and different diseases

Linear regression with one variable and multiple variables

The integration of Apache Spark

Setting up Apache Spark with Apache Mahout

An example script

Distributed row matrix

An explanation of the code

Mahout references

The bias-variance trade-off

How to avoid over-fitting and under-fitting

Logistic regression with SGD

Logistic functions

Minimizing the cost function

Multinomial logistic regression versus binary logistic regression

A real-world example

An example script

Testing and evaluation

The confusion matrix

The area under the curve

The Naïve Bayes algorithm

The Bayes theorem

Text classification

Naïve assumption and its pros and cons in text classification

Improvements that Apache Mahout has made to the Naïve Bayes classification

A text classification coding example using the 20 newsgroups' example

Understand the 20 newsgroups' dataset

Text classification using Naïve Bayes – a MapReduce implementation with Hadoop

Text classification using Naïve Bayes – the Spark implementation

The Markov chain

Hidden Markov Model

A real-world example – developing a POS tagger using HMM supervised learning

POS tagging

HMM for POS tagging

HMM implementation in Apache Mahout

HMM supervised learning

The important parameters

Returns

The Baum Welch algorithm

A code example

The important parameters

The Viterbi evaluator

The Apache Mahout references

Summary

4. Recommen...