Bayesian Modeling and Computation in Python

Osvaldo A. Martin, Ravin Kumar, Junpeng Lao

• Publisher: Chapman and Hall/CRC
• Year: 2021
• Print ISBN: 9780367894368
• eBook ISBN: 9781000520071
• Edition: 1
• Topic: Economics
• Subtopic: Statistics for Business & Economics
• Index: Economics

Bayesian Modeling and Computation in Python aims to help beginner Bayesian practitioners to become intermediate modelers. It uses a hands on approach with PyMC3, Tensorflow Probability, ArviZ and other libraries focusing on the practice of applied statistics with references to the underlying mathematical theory.

The book starts with a refresher of the Bayesian Inference concepts. The second chapter introduces modern methods for Exploratory Analysis of Bayesian Models. With an understanding of these two fundamentals the subsequent chapters talk through various models including linear regressions, splines, time series, Bayesian additive regression trees. The final chapters include Approximate Bayesian Computation, end to end case studies showing how to apply Bayesian modelling in different settings, and a chapter about the internals of probabilistic programming languages. Finally the last chapter serves as a reference for the rest of the book by getting closer into mathematical aspects or by extending the discussion of certain topics.

This book is written by contributors of PyMC3, ArviZ, Bambi, and Tensorflow Probability among other libraries.

1 Bayesian Inference

DOI: 10.1201/9781003019169-1

Modern Bayesian statisticsA statistic (not plural) or sample statistic is any quantity computed from a sample. Sample statistics are computed for several reasons including estimating a population (or data generating process) parameter, describing a sample, or evaluating a hypothesis. The sample mean (also known as empirical mean) is a statistic, the sample variance (or empirical variance) is another example. When a statistic is used to estimate a population (or data generating process) parameter, the statistic is called an estimator. Thus, the sample mean can be an estimator and the posterior mean can be another estimator. is mostly performed using computer code. This has dramatically changed how Bayesian statistics was performed from even a few decades ago. The complexity of models we can build has increased, and the barrier of necessary mathematical and computational skills has been lowered. Additionally, the iterative modeling process has become, in many aspects, much easier to perform and more relevant than ever. The popularization of very powerful computer methods is really great but also demands an increased level of responsibility. Even if expressing statistical methods is easier than ever, statistics is a field full of subtleties that do not magically disappear by using powerful computational methods. Therefore having a good background about theoretical aspects, especially those relevant in practice, is extremely useful to effectively apply statistical methods. In this first chapter, we introduce these concepts and methods, many, which will be further explored and expanded throughout the rest of the book.

1.1 Bayesian Modeling

A conceptual model is a representation of a system, made of the composition of concepts that are used to help people know, understand, or simulate the object or process the model represents [39]. Additionally, models are human-designed representations with very specific goals in mind. As such, it is generally more convenient to talk about the adequacy of the model to a given problem than its intrinsic correctness. Models exist solely as an aid to a further goal.

When designing a new car, a car company makes a physical model to help others understand how the product will look when it is built. In this case, a sculptor with prior knowledge of cars, and a good estimate of how the model will be used, takes a supply of raw material such as clay, uses hand tools to sculpt a physical model. This physical model can help inform others about various aspects of the design, such as whether the appearance is aesthetically pleasing, or if the shape of the car is aerodynamic. It takes a combination of domain expertise and sculpting expertise to achieve a useful result. The modeling process often requires building more than one model, either to explore different options or because the models are iteratively improved and expanded as a result of the interaction with other members of the car development team. These days it is also common that in addition to a physical car model, there is a digital model built-in Computer-Aided Design software. This computer model has some advantages over a physical one. It is simpler and cheaper to use for digital for crash simulations versus testing on physical cars. It is also easier to share this model with colleagues in different offices.

These same ideas are relevant in Bayesian modeling. Building a model requires a combination of domain expertise and statistical skill to incorporate knowledge into some computable objectives and determine the usefulness of the result. Data is the raw material, and statistical distributions are the main mathematical tools to shape the statistical model. It takes a combination of domain expertise and statistical expertise to achieve a useful result. Bayesian practitioners also build more than one model in an iterative fashion, the first of which is primarily useful for the practitioner themselves to identify gaps in their thinking, or shortcomings in their models. These first sets of models are then used to build subsequent improved and expanded models. Additionally, the use of one inference mechanism does not obviate the utility for all others, just as a physical model of a car does not obviate the utility of a digital model. In the same way, the modern Bayesian practitioner has many ways to express their ideas, generate results, and share the outputs, allowing a much wider distribution of positive outcomes for the practitioner and their peers.

1.1.1 Bayesian Models

Bayesian models, computational or otherwise, have two defining characteristics:

• Unknown quantities are described using probability distributions ¹. We call these quantities parameters ².
• Bayes’ theorem is used to update the values of the parameters conditioned on the data. We can also see this process as a reallocation of probabilities.

At a high-l...

Table of contents

• Cover Page
• Half-Title Page
• Series Page
• Title Page
• Copyright Page
• Dedication Page
• Contents
• Foreword
• Preface
• Symbols
• 1 Bayesian Inference
• 2 Exploratory Analysis of Bayesian Models
• 3 Linear Models and Probabilistic Programming Languages
• 4 Extending Linear Models
• 5 Splines
• 6 Time Series
• 7 Bayesian Additive Regression Trees
• 8 Approximate Bayesian Computation
• 9 End to End Bayesian Workflows
• 10 Probabilistic Programming Languages
• 11 Appendiceal Topics
• Glossary
• Bibliography
• Index