A Student's Guide to Bayesian Statistics

Ben Lambert

• Publisher: SAGE Publications Ltd
• Year: 2018
• eBook ISBN: 9781526418265
• Edition: 1
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Supported by a wealth of learning features, exercises, and visual elements as well as online video tutorials and interactive simulations, this book is the first student-focused introduction to Bayesian statistics. Without sacrificing technical integrity for the sake of simplicity, the author draws upon accessible, student-friendly language to provide approachable instruction perfectly aimed at statistics and Bayesian newcomers. Through a logical structure that introduces and builds upon key concepts in a gradual way and slowly acclimatizesstudents to using R and Stan software, the book covers:

• An introduction to probability and Bayesian inference
• Understanding Bayes? rule
• Nuts and bolts of Bayesian analytic methods
• Computational Bayes and real-world Bayesian analysis
• Regression analysis and hierarchical methods

This unique guide will help students develop the statistical confidence and skills to put the Bayesian formula into practice, from the basic concepts of statistical inference to complex applications of analyses.

1 How Best to use this Book

Chapter contents

• 1.1 The purpose of this book 2
• 1.2 Who is this book for? 2
• 1.3 Prerequisites 2
• 1.4 Book outline 3
• 1.5 Route planner – suggested journeys through Bayesland 4
• 1.6 Video 5
• 1.7 Problem sets 6
• 1.8 R and Stan 6
• 1.9 Why don’t more people use Bayesian statistics? 6
• 1.10 What are the tangible (non-academic) benefits of Bayesian statistics? 7
• 1.11 Suggested further reading 8

1.1 The Purpose of this Book

This book aims to be a friendlier introduction to Bayesian analysis than other texts available out there. Whenever we introduce new concepts, we keep the mathematics to a minimum and focus instead on the intuition behind the theory. However, we do not sacrifice content for the sake of simplicity and aim to cover everything from the basics up to the advanced topics required for applied research. Overall, this book seeks to plug a gap in the existing literature (see Figure 1.1).

To help readers along the way, we have developed a number of interactive elements which are accessible through the book’s website, as well as example code for readers to peruse and, if they so desire, to run themselves. We also supplement key ideas with videos, which approach topics from different angles, and examples.

At the end of each chapter, there are problem sets, which allow the student to build up practical experience of Bayesian analysis. Whenever appropriate these problem sets will also be supplemented with video material.

Figure 1.1 This book’s niche.

1.2 Who is this Book for?

This book is for anyone who has ever tried and failed at statistics, particularly Bayesian statistics.

The text is aimed at anyone who has completed high school mathematics and wants to conduct Bayesian inference on real-world data. We assume no previous knowledge of probability (which is central to Bayesian analysis) and devote the entirety of Chapter 3 to this topic. We do not require that the student be versed in Frequentist statistics, as we aim to build an alternative and complementary path to a shared goal. After Chapter 2 we refrain from frequent comparisons between these two approaches.

While we start at the beginning of statistical inference, we hope to provide a guide of practical use for the types of analysis encountered in real life.

1.3 Prerequisites

Knowledge of the following is strongly recommended to allow the reader to get the most out of this book:

• Algebra: Manipulation of symbolic expressions is widespread throughout the text.
• Products and summations: These are mainly used for writing down likelihood and log-likelihood functions.

There is some differentiation in this book, although it is fairly limited and used mostly in sections concerning maximum likelihood. A note on integration: At early stages of this book’s development, it contained many integrals. In teaching this material, we have realised that students can be discouraged by the sight of these mathematical behemoths. Fortunately, since modern Bayesian inference relies on computational sampling rather than hard calculation (see Part IV), an intimate knowledge of integrals is no longer essential. In this book, we keep the use of integrals to a minimum, apart from mainly those cases where we provide a motivation for Markov chain Monte Carlo (MCMC).

The only other prerequisite concerns the practical application of Bayesian analysis. Knowledge of the open source statistical software R [29] would be useful. We do not classify this item with those above, because we use only the basic functionality of this language and also document any use of this language thoroughly. This language is widely used for statistical analysis and, because of its popularity, there are excellent free online resources that can be used to learn it. Here we list just a few of the available resources:

• Coursera (www.coursera.org) has a number of great lecture courses with associated problem sets available for learning R. We recommend the courses by Roger Peng at Johns Hopkins University.
• Try R (http://tryr.codeschool.com) is a short interactive introductory lesson on the basics of R.
• Data Camp’s free Introduction to R (www.datacamp.com/courses/free-introduction-to-r) provides 4 hours of interactive lectures on the basics of R.
• The R Guide (http://cran.r-project.org/doc/contrib/Owen-TheRGuide.pdf) is a nice written guide to R.

While none of these are essential, if you have difficulty following the examples in this text, we recommend that you try the above resources.

1.4 Book Outline

We have written this text to make each chapter as self-contained as possible. While, at times, the reader may feel that this makes the text repetitive, this approach has two purposes: first to help keep topics self-contained, but also because we believe that some ideas are worth encountering, and re-encountering, at different points along the way in learning about Bayes.

The book is divided into five parts:

• Part I: An introduction to Bayesian inference
• Part II: Understanding the Bayesian formula
• Part III: Analytic Bayesian methods
• Part IV: A practical guide to doing real-life Bayesian analysis: Computational Bayes
• Part V: Hierarchical models and regression

Part I provides an introduction to the purpose of statistical inference, then compares and contrasts the Bayesian and Frequentist approaches to it. Bayesian inference is based on probability distributions. Hence, it is imperative to understand these types of mathematical object. The latter half of this part is devoted to this topic. Part II introduces the reader to the constituent elements of the Bayesian inference formula, and in doing so provides an all-round introduction to the practicalities of doing Bayesian inference. Part III aims to equip the reader with knowledge of the most practically relevant probability distributions for Bayesian inference. These objects come under two categories (although some distributions fall into both): prior distributions and likelihood distributions. Knowledge of these distributions is essential for understanding existing research papers and books which use Bayesian statistics, as well as necessary to conduct Bayesian inference in practice. The rest of this part is concerned with introducing the reader to ‘nice’ combinations of distributions, which allow for a pen-and-paper deduction of quantities of interest. This is important as a stepping stone to computational methods, but also because these types of model are a good place to start before implementing more nuanced models. Part IV introduces the reader to the modern methods of undertaking Bayesian analysis, through computational Markov chain Monte Carlo. This part provides an intuitive explanation of some of the most important algorithmic tools used in computational methods. It also introduces the reader to the statistical programming language that we use for many applied examples in this text: Stan. This part is essential reading for anyone who wants to conduct serious real-world Bayesian analysis of data. Assuming this computational knowledge, Part V introduces the reader to an important Bayesian paradigm known as hierarchical models. It also provides an in-depth introduction to Bayesian regression modelling for linear and generalised linear models.

Each chapter has two introductory summaries: the chapter mission statement and chapter goals. The former is usually a one- or two-sentence summary of the material to be covered in the chapter. The goals section is more detailed and links together material encountered in previous chapters. At the end of each chapter, there are also two summary sections: a chapter summary and short list of chapter outcomes. These provide the reader with a description of the skills acquired as well as a perspective on the material’s position within the book’s overall goals.

1.5 Route Planner – Suggested Journeys Through Bayesland

In the style of most good guide books, we suggest itineraries that offer routes through select parts of Bayesland. These journeys are meant to be shortish paths towards gaining a better understanding of particular elements of Bayesian statistics. Like most short trips they are not as all-encompassing as a more prolonged stay, but can nonetheless be useful and fun in their own right. We offer the following trips through Bayesland, which the reader can choose based on their time constraints, goals and pre-existing knowledge:

• The long-weekender (introductory) provides a short introduction to the principles of Bayesian inference. Chapter 2 introduces you to the theory behind statistical inference and provides a gentle comparison between Bayesian and Frequentist approaches. If you have extra time, and knowledge of probability distributions, then try your hand at Chapter 7.
• The 2-week basic package trip (introductory), consisting of Parts I and II, provides a full introduction to Bayesian statistics from the ground up.
• The 2-week refresher (intermediate) aims to provide a good grounding in Bayesian inference for someone with some experience in statistics. Read Chapter 2 to get your bearings. Depending on your knowledge of the Bayesian formula, Part II can be either read or left behind. Part III should be read almost in full, as this will get you up to speed with many of the tools necessary to understand research papers. To this end, you can probably avoid reading Chapter 11, on objective Bayes.
• The Bayes summer 1-weeker (intermediate) is a short course that provides some background information for anyone who wants to use Bayesian inference in their own work. Read Chapters 8 and 9 to get an idea of some of the distributional tools which are available to us and how they can be used. Next read Chapter 12, which explains some of the issues with analytical Bayesian inference and a motivation for Markov chain Monte Carlo.
• The 3-week full practical swing (intermediate-expert) is if you are happy with your knowledge of the Bayesian inference formula and the distributions used in Bayesian analysis, and you want to skip ahead to Part IV, which introduces computational methods. This introduces you to the motivation behind computational sampling and provides an introduction to Stan, which is the statistical language used in this text to do sampling via MCMC. If you have time, then you may want to progress to Part V, where there are more applied examples that use Stan.
• The ‘I need to do Bayesian analysis now’ 3-day leg (intermediate-expert) is tailored to those practitioners who need to carry out Bayesian data analysis fast. The most likely audience here consists of those in research, either academic or corporate, who have existing knowledge of Bayesian statistics. Skip a...

Table of contents

• Cover
• Half Title
• Publisher Note
• Title Page
• Copyright Page
• Acknowledgements
• Contents
• Online resources
• Acknowledgements
• About the Author
• 1 How Best to use this Book
• Part I An Introduction to Bayesian Inference
• 2 The Subjective Worlds of Frequentist and Bayesian Statistics
• 3 Probability – The Nuts and Bolts of Bayesian Inference
• Part II Understanding the Bayesian Formula
• 4 Likelihoods
• 5 Priors
• 6 The Devil is in the Denominator
• 7 The Posterior – The Goal of Bayesian Inference
• Part III Analytic Bayesian Methods
• 8 An Introduction to Distributions for the Mathematically Uninclined
• 9 Conjugate priors
• 10 Evaluation of model fit and hypothesis testing
• 11 Making Bayesian analysis objective?
• Part IV A practical guide to doing real-life Bayesian analysis: computational Bayes
• 12 Leaving conjugates behind: Markov chain Monte Carlo
• 13 Random Walk Metropolis
• 14 Gibbs sampling
• 15 Hamiltonian Monte Carlo
• 16 Stan
• Part V Hierarchical models and regression
• 17 Hierarchical models
• 18 Linear regression models
• 19 Generalised linear models and other animals
• References
• Index