Bayesian Analysis of Linear Models

Broemeling

• Publisher: CRC Press
• Year: 2017
• eBook ISBN: 9781351464475
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

With Bayesian statistics rapidly becoming accepted as a way to solve applied statisticalproblems, the need for a comprehensive, up-to-date source on the latest advances in thisfield has arisen.Presenting the basic theory of a large variety of linear models from a Bayesian viewpoint,Bayesian Analysis of Linear Models fills this need. Plus, this definitive volume containssomething traditional-a review of Bayesian techniques and methods of estimation, hypothesis,testing, and forecasting as applied to the standard populations ... somethinginnovative-a new approach to mixed models and models not generally studied by statisticianssuch as linear dynamic systems and changing parameter models ... and somethingpractical-clear graphs, eary-to-understand examples, end-of-chapter problems, numerousreferences, and a distribution appendix.Comprehensible, unique, and in-depth, Bayesian Analysis of Linear Models is the definitivemonograph for statisticians, econometricians, and engineers. In addition, this text isideal for students in graduate-level courses such as linear models, econometrics, andBayesian inference.

1

BAYESIAN INFERENCE FOR THE GENERAL LINEAR MODEL

INTRODUCTION

Many books begin with the general linear model and this is no exception. The general linear model includes many useful and interesting special cases. For example, independent normal sequences of random variables, simple and multiple regression models, models for designed experiments, and analysis of covariance models are all special cases of the general linear model which will be explained in more detail in the following chapters.

This chapter introduces the prior, posterior, and predictive analysis of the linear model and thereby lays the foundation for the remainder of the book where the analyses are reported with other models.

THE PARAMETRIC INFERENCE PROBLEM

Let θ be a p × 1 vector of real parameters, Y = (y₁, y₂, …, y_n)’a n × 1 vector of observations, X a n × p known design matrix. Then the general linear model is

$\text{Y}=\text{Xθ}+\text{e},$

(1.1)

where e ~ Ν(0, τ⁻¹I_n), and τI_n is the precision matrix of e, which has covariance matrix σ²I_n, and σ² = τ⁻¹ > 0 is unknown.

This is the general linear model and our objective is to provide inferences for θ and τ when observing s = (y₁, y₂, …, y_n), where y_i is the i-th observation. The word inference is somewhat of a vague word, but it usually implies a procedure which extracts information about θ from the sample s.

Introductory statistics books explain inferential techniques in terms of point and interval estimation, tests of hypotheses, and forecasts.

For the Bayesian, all inferences are based on the posterior distribution of θ, which is given by Bayes theorem.

BAYES THEOREM

Suppose one’s prior information about θ is represented by a probability density function ξ(θ, τ), θ ∈ R^p, τ > 0, then Bayes theorem combines this information with the information contained in the sample. The likelihood function for θ and τ is

$\text{L(θ,}\text{τ}\text{|s)}\propto {\text{τ}}^{\text{n/2}}\text{exp}-\frac{\text{τ}}{2}(\text{Y}-\text{X}\text{θ})\prime (\text{Y}-\text{X}\text{θ}),$

(1.2)

where θ ∈ R^p and τ > 0, where ∝ means proportional to (as a function of τ and θ). The likelihood function is one’s sample information about the parameters and is the conditional density function of the sample random variables given θ and τ.

Bayes theorem gives the conditional density of θ given s

$\begin{array}{cc}\text{ξ(θ,}\text{τ|s)}\propto \text{L(θ,}\text{τ/s)ξ(θ,}\text{τ),}& \begin{array}{cc}\text{θ}\in {\text{R}}^{\text{p}},& \text{τ > 0}\text{.}\end{array}\end{array}$

(1.3)

The posterior density of θ is ξ(θ, τ |s) and represents one’s knowledge of θ and τ after observing the sample s. On the other hand our information about θ and τ before s is observed is contained in the prior density.

Note the posterior density (1.3) is written with a proportional symbol and ξ is used to denote both the prior and posterior densities. If one uses an equality sign, the posterior density is

$\begin{array}{cc}\text{ξ (θ,}\text{τ|s) = K}\cdot \text{L(θ,}\text{τ/s) ξ (θ,}\text{τ),}& \begin{array}{cc}\text{θ}\in {\text{R}}^{\text{p}},& \text{τ > 0}\end{array}\end{array}$

(1.4)

where K is the normalizing constant and is given by

${\text{k}}^{-1}={\displaystyle {\int }_0^{\infty }{\displaystyle \underset{{\text{R}}^{\text{p}}}{\int }\text{L(θ, τ|s)ξ(θ, τ)dθ dτ,}}}$

which is the marginal probability density of Y.

Obviously it is easier to omit the normalizing constant and use the proportional symbol and this convention will be adhered to in this book.

Before continuing the analysis, one must find the posterior density of θ and τ.

PRIOR INFORMATION

The prior information about the parameters θ and τ is given in two ways. The first is when ξ(θ, τ) is a normal-gamma prior density, namely,

$\begin{array}{cc}\text{ξ (θ,}{\text{τ) = ξ}}_1{\text{(θ|τ) ξ}}_2\text{(τ),}& \begin{array}{cc}\text{θ}\in {\text{R}}^{\text{p}},& \text{τ > 0,}\end{array}\end{array}$

(1.5)

where

$\begin{array}{cc}{\text{ξ}}_1\text{(θ|τ)}\propto {\text{τ}}^{\text{p/2}}\text{exp -}\frac{\text{τ}}{2}\text{(θ}-\text{μ)' p (θ}-\text{μ),}& \text{θ}\in {\text{R}}^{\text{p}}\end{array}$

(1.6)

and μ is a p × 1 given vector and p is a known p × p positive definite matrix. Thus ξ₁ is the conditional density of θ given τ and is normal with mean vector μ and precision matrix τp.

The marginal prior density of τ is gamma with parameters α > 0 and β > 0.

$\begin{array}{cc}{\text{ξ}}_2(\text{τ})\propto {\text{τ}}^{\text{α-1}}\text{exp}-\text{τβ},& \text{τ > 0}\text{.}\end{array}$

(1.7)

What does this imply about the marginal pri...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Preface
• Introduction
• 1 BAYESIAN INFERENCE FOR THE GENERAL LINEAR MODEL
• 2 LINEAR STATISTICAL MODELS AND BAYESIAN INFERENCE
• 3 THE TRADITIONAL LINEAR MODELS
• 4 THE MIXED MODEL
• 5 TIME SERIES MODELS
• 6 LINEAR DYNAMIC SYSTEMS
• 7 STRUCTURAL CHANGE IN LINEAR MODELS
• 8 MULTIVARIATE LINEAR MODELS
• 9 LOOKING AHEAD
• APPENDIX
• Index