![]()
CHAPTER 1
INTRODUCTION TO BAYESIAN INFERENCE
1.1 INTRODUCTION: BAYESIAN MODELING IN THE 21 ST CENTURY
The beginning of the 21 st century found Bayesian statistics to be fashionable in science. But until the late 1980s, Bayesian statistics were considered only as an interesting alternative to the “classical” theory. The main difference between the classical statistical theory and the Bayesian approach is that the latter considers parameters as random variables that are characterized by a prior distribution. This prior distribution is combined with the traditional likelihood to obtain the posterior distribution of the parameter of interest on which the statistical inference is based. Although the main tool of Bayesian theory is probability theory, for many years Bayesians were considered as a heretic minority for several reasons. The main objection of “classical” statisticians was the subjective view point of the Bayesian approach introduced in the analysis via the prior distribution. However, as history had proved, the main reason why Bayesian theory was unable to establish a foothold as a well accepted quantitative approach for data analysis was the intractabilities involved in the calculation of the posterior distribution. Asymptotic methods had provided solutions to specific problems, but no generalization was possible. Until the early 1990s two groups of statisticians had (re)discovered Markov chain Monte Carlo (MCMC) methods (Gelfand and Smith, 1990; Gelfand et al., 1990). Physicists were familiar with MCMC methodology from the 1950s. Nick Metropolis and his associates had developed one of the first electronic supercomputers (for those days) and had been testing their theories in physics using Monte Carlo techniques. Implementation of the MCMC methods in combination with the rapid evolution of personal computers made the new computational tool popular within a few years. Bayesian statistics suddenly became fashionable, opening new highways for statistical research. Using MCMC, we can now set up and estimate complicated models that describe and solve problems that could not be solved with traditional methods.
Since 1990, when MCMC first appeared in statistical science, many important related papers have appeared in the literature. During 1990–1995, MCMC-related research focused on the implementation of new methods in various popular models [see, e.g., Gelman and Rubin (1992), Gelfand, Smith and Lee (1992), Gilks and Wild (1992), Dellaportas and Smith (1993)]. The development of MCMC methodology had also promoted the implementation of random effects and hierarchical models.
Green’s (1995) publication on reversible jump Markov chain Monte Carlo (RJMCMC) algorithm boosted research on model averaging, selection and model exploration algorithms [see, e.g., Dellaportas and Forster (1999), Dellaportas et al. (2002), Sisson (2005), Hans et al. (2007)]. During the same period, the early versions of BUGS software appeared. BUGS was computing-language-oriented software in which the user only needed to specify the structure of the model. Then, BUGS was using MCMC methods to generate samples from the posterior distribution of the specified model. The most popular version of BUGS (v.05) was available via the Internet in 1996 [manual date August 14, 1996; see, Spiegelhalter et al. (1996a)]. Currently WinBUGS version 1.4.3 is available via the WinBUGS project Webpage (Spiegelhalter et al., 2003d). Many add-ons, utilities, and variations of the package are also available. The development of WinBUGS had proved valuable for the implementation of Bayesian models in a wide variety of scientific disciplines. In parallel, many workshops and courses have been organized on Bayesian inference, data analysis, and modeling using WinBUGS software. WinBUGS is a key factor in the growing popularity of Bayesian methods in science.
Development, extensions, and improvement of MCMC methods have also been considered in statistical research since the mid-1990s. Automatic samplers, which will be directly applicable in any set of data, are within this frame of research and have led to the slice sampler (Higdon, 1998; Damien et al., 1999). Various samplers designed for mod...
