Learning Apache Mahout

Chandramani Tiwary

• Editore: Packt Publishing
• Anno: 2015
• ISBN:
  9781783555215
• Argomento: Ciencia de la computación
• Categoria: Algoritmos de programación

Learning Apache Mahout

---

Table of Contents

Learning Apache Mahout

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Introduction to Mahout

Why Mahout

Simple techniques and more data is better

Sampling is difficult

Community and license

When Mahout

Data too large for single machine

Data already on Hadoop

Algorithms implemented in Mahout

How Mahout

Setting up the development environment

Configuring Maven

Configuring Mahout

Configuring Eclipse with the Maven plugin and Mahout

Mahout command line

A clustering example

Reuter's raw data file

A classification example

Mahout API – a Java program example

The dataset

Parallel versus in-memory execution mode

Summary

2. Core Concepts in Machine Learning

Supervised learning

Determine the objective

Decide the training data

Create and clean the training set

Feature extraction

Train the models

Bagging

Boosting

Validation

Holdout-set validation

K-fold cross validation

Evaluation

Bias-variance trade-off

Function complexity and amount of training data

Dimensionality of the input space

Noise in data

Unsupervised learning

Cluster analysis

Objective

Feature representation

Feature normalization

Row normalization

Column normalization

Rescaling

Standardization

A notion of similarity and dissimilarity

Euclidean distance measure

Squared Euclidean distance measure

Manhattan distance measure

Cosine distance measure

Tanimoto distance measure

Algorithm for clustering

A stopping criteria

Frequent pattern mining

Measures for identifying interesting rules

Support

Confidence

Lift

Conviction

Things to consider

Actionable rules

What association to look for

Recommender system

Collaborative filtering

Cold start

Scalability

Sparsity

Content-based filtering

Model efficacy

Classification

Confusion matrix

ROC curve and AUC

Features of ROC graphs

Evaluating classifier using the ROC curve

Area-based accuracy measure

Euclidian distance comparison

Example

Regression

Mean absolute error

Root mean squared error

R-square

Adjusted R-square

Recommendation system

Score difference

Precision and recall

Clustering

The internal evaluation

The intra-cluster distance

The inter-cluster distance

The Davies–Bouldin index

The Dunn index

The external evaluation

The Rand index

F-measure

Summary

3. Feature Engineering

Feature engineering

Feature construction

Categorical features

Merging categories

Converting to binary variables

Converting to continuous variables

Continuous features

Binning

Binarization

Feature standardization

Rescaling

Mean standardization

Scaling to unit norm

Feature transformation derived from the problem domain

Ratios

Frequency

Aggregate transformations

Normalization

Mathematical transformations

Feature extraction

Feature selection

Filter-based feature selection

Wrapper-based feature selection

Backward selection

Forward selection

Recursive feature elimination

Embedded feature selection

Dimensionality reduction

Summary

4. Classification with Mahout

Classification

White box models

Black box models

Logistic regression

Mahout logistic regression command line

Getting the data

Model building via command line

Splitting the dataset

Train the model command line option

Interpreting the output

Testing the model

Prediction

Adaptive regression model

Code example with logistic regression

Train the model

The LogisticRegressionParameter and CsvRecordFactory classes

A code example without the parameter class

Testing the online regression model

Getting predictions from OnlineLogisticRegression

A CrossFoldLearner example

Random forest

Bagging

Random subsets of features

Out-of-bag error estimate

Random forest using the command line

Predictions from random forest

Naïve Bayes classifier

Numeric features with naïve Bayes

Command line

Summary

5. Frequent Pattern Mining and Topic Modeling

Frequent pattern mining

Building FP Tree

Constructing the tree

Identify...