Learning Probabilistic Graphical Models in R

David Bellot

• Publisher: Packt Publishing
• Year: 2016
• eBook ISBN: 9781784392055
• Edition: 1
• Topic: Computer Science
• Subtopic: Data Modelling & Design
• Index: Computer Science

Familiarize yourself with probabilistic graphical models through real-world problems and illustrative code examples in R

About This Book

• Predict and use a probabilistic graphical models (PGM) as an expert system
• Comprehend how your computer can learn Bayesian modeling to solve real-world problems
• Know how to prepare data and feed the models by using the appropriate algorithms from the appropriate R package

Who This Book Is For

This book is for anyone who has to deal with lots of data and draw conclusions from it, especially when the data is noisy or uncertain. Data scientists, machine learning enthusiasts, engineers, and those who curious about the latest advances in machine learning will find PGM interesting.

What You Will Learn

• Understand the concepts of PGM and which type of PGM to use for which problem
• Tune the model's parameters and explore new models automatically
• Understand the basic principles of Bayesian models, from simple to advanced
• Transform the old linear regression model into a powerful probabilistic model
• Use standard industry models but with the power of PGM
• Understand the advanced models used throughout today's industry
• See how to compute posterior distribution with exact and approximate inference algorithms

In Detail

Probabilistic graphical models (PGM, also known as graphical models) are a marriage between probability theory and graph theory. Generally, PGMs use a graph-based representation. Two branches of graphical representations of distributions are commonly used, namely Bayesian networks and Markov networks. R has many packages to implement graphical models.

We'll start by showing you how to transform a classical statistical model into a modern PGM and then look at how to do exact inference in graphical models. Proceeding, we'll introduce you to many modern R packages that will help you to perform inference on the models. We will then run a Bayesian linear regression and you'll see the advantage of going probabilistic when you want to do prediction.

Next, you'll master using R packages and implementing its techniques. Finally, you'll be presented with machine learning applications that have a direct impact in many fields. Here, we'll cover clustering and the discovery of hidden information in big data, as well as two important methods, PCA and ICA, to reduce the size of big problems.

Style and approach

This book gives you a detailed and step-by-step explanation of each mathematical concept, which will help you build and analyze your own machine learning models and apply them to real-world problems. The mathematics is kept simple and each formula is explained thoroughly.

Learning Probabilistic Graphical Models in R

---

Table of Contents

Learning Probabilistic Graphical Models in R

Credits

About the Author

About the Reviewers

www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Probabilistic Reasoning

Machine learning

Representing uncertainty with probabilities

Beliefs and uncertainty as probabilities

Conditional probability

Probability calculus and random variables

Sample space, events, and probability

Random variables and probability calculus

Joint probability distributions

Bayes' rule

Interpreting the Bayes' formula

A first example of Bayes' rule

A first example of Bayes' rule in R

Probabilistic graphical models

Probabilistic models

Graphs and conditional independence

Factorizing a distribution

Directed models

Undirected models

Examples and applications

Summary

2. Exact Inference

Building graphical models

Types of random variable

Building graphs

Probabilistic expert system

Basic structures in probabilistic graphical models

Variable elimination

Sum-product and belief updates

The junction tree algorithm

Examples of probabilistic graphical models

The sprinkler example

The medical expert system

Models with more than two layers

Tree structure

Summary

3. Learning Parameters

Introduction

Learning by inference

Maximum likelihood

How are empirical and model distribution related?

The ML algorithm and its implementation in R

Application

Learning with hidden variables – the EM algorithm

Latent variables

Principles of the EM algorithm

Derivation of the EM algorithm

Applying EM to graphical models

Summary

4. Bayesian Modeling – Basic Models

The Naive Bayes model

Representation

Learning the Naive Bayes model

Bayesian Naive Bayes

Beta-Binomial

The prior distribution

The posterior distribution with the conjugacy property

Which values should we choose for the Beta parameters?

The Gaussian mixture model

Definition

Summary

5. Approximate Inference

Sampling from a distribution

Basic sampling algorithms

Standard distributions

Rejection sampling

An implementation in R

Importance sampling

An implementation in R

Markov Chain Monte-Carlo

General idea of the method

The Metropolis-Hastings algorithm

MCMC for probabilistic graphical models in R

Installing Stan and RStan

A simple example in RStan

Summary

6. Bayesian Modeling – Linear Models

Linear regression

Estimating the parameters

Bayesian linear models

Over-fitting a model

Graphical model of a linear model

Posterior distribution

Implementation in R

A stable implementation

More packages in R

Summary

7. Probabilistic Mixture Models

Mixture models

EM for mixture models

Mixture of Bernoulli

Mixture of experts

Latent Dirichlet Allocation

The LDA model

Variational inference

Examples

Summary

A. Appendix

References

Books on the Bayesian theory

Books on machine learning

Papers

Index

Learning Probabilistic Graphical Models in R

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: April 2016

Production reference: 1270416

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78439-205-5

www.packtpub.com

Credits

Author

David Bellot

Reviewers

Mzabalazo Z. Ngwenya

Prabhanjan Tattar

Acquisition Editor

Divya Poojari

Content Development Editor

Trusha Shriyan

Technical Editor

Vivek Arora

Copy Editor

Stephen Copestake

Project Coordinator

Kinjal Bari

Proofreader

Safis Editing

Indexer

Mariammal Chettiyar

Graphics

Abhinash Sahu

Production Coordinator

Nilesh Mohite

Cover Work

Nilesh Mohite

About the Author

David Bellot is a PhD graduate in computer science from INRIA, France, with a focus on Bayesian machine learning. He was a postdoctoral fellow at the University of California, Berkeley, and worked for companies such as Intel, Orange, and Barclays Bank. He currently works in the financial industry, where he develops financial market prediction algorithms using machine learning. He is also a contributo...

Table of contents

• Learning Probabilistic Graphical Models in R