Generalized Linear Models

P. McCullagh, John A. Nelder, D.R. Cox, N. Reid, Valerie Isham, R.J. Tibshirani, Thomas A. Louis, Howell Tong, Niels Keiding

• Publisher: Chapman and Hall/CRC
• Year: 2019
• eBook ISBN: 9781351445849
• Edition: 2
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

The success of the first edition of Generalized Linear Models led to the updated Second Edition, which continues to provide a definitive unified, treatment of methods for the analysis of diverse types of data. Today, it remains popular for its clarity, richness of content and direct relevance to agricultural, biological, health, engineering, and ot

CHAPTER 1

Introduction

1.1 Background

In this book we consider a class of statistical models that is a natural generalization of classical linear models. Generalized linear models include as special cases, linear regression and analysis-of-variance models, logit and probit models for quantal responses, log-linear models and multinomial response models for counts and some commonly used models for survival data. It is shown that the above models share a number of properties, such as linearity, that can be exploited to good effect, and that there is a common method for computing parameter estimates. These common properties enable us to study generalized linear models as a single class, rather than as an unrelated collection of special topics.

Classical linear models and least squares began with the work of Gauss and Legendre (Stigler, 1981, 1986) who applied the method to astronomical data. Their data were usually measurements of continuous quantities such as the positions and magnitudes of the heavenly bodies and, at least in the astronomical investigations, the variability in the observations was largely the effect of measurement error. The Normal, or Gaussian, distribution was viewed as a mathematical construct developed to describe the properties of such errors; later in the nineteenth century the same distribution was used to describe the variation between individuals in a biological population in respect of a character such as height, an application quite different in kind from its use for describing measurement error, and leading to the numerous biological applications of linear models.

Gauss introduced the Normal distribution of errors as a device for describing variability, but he showed that many of the important properties of least-squares estimates depend not on Normality but on the assumptions of constant variance and independence. A closely related property applies to all generalized linear models. In other words, although we make reference at various points to standard distributions such as the Normal, binomial, Poisson, exponential or gamma, the second-order properties of the parameter estimates are insensitive to the assumed distributional form: the second-order properties depend mainly on the assumed variance-to-mean relationship and on uncorrelatedness or independence. This is fortunate because, in applications, one can rarely be confident that all aspects of the assumed distributional form are correct.

Another strand in the history of statistics is the development of methods for dealing with discrete events rather than with continuously varying quantities. The enumeration of probabilities for various configurations in games of cards and dice was a matter of keen interest for gamblers in the eighteenth century. From their pioneering work grew methods for dealing with data in the form of counts of events. In the context of rare events, the basic distribution is that named after Poisson. This distribution has been applied to diverse kinds of events: a famous example concerns unfortunate soldiers kicked to death by Prussian horses (Bortkewitsch, 1898). The annual number of such incidents during the period 1875–1894 was observed to be consistent with the Poisson distribution having mean about 0.7 per corps per year. There is, however, some variation in this figure between corps and between years. Routine laboratory applications of the Poisson model include the monitoring of radioactive tracers by emission counts, counts of infective organisms as measured by the number of events observed on a slide under a microscope, and so on.

Closely related to the Poisson model are models for the analysis of counted data in the form of proportions or ratios of counts. The Bernoulli distribution is often suitable for modelling the presence or absence of disease in a patient, and the derived binomial distribution may be suitable as a model for the number of diseased patients in a fixed pool of patients under study. In medical and pharmaceutical trials it is usually required to study not primarily the incidence of a particular disease, but how the incidence is affected by factors such as age, social class, housing conditions, exposure to pollutants, and any treatment procedures under study. Generalized linear models permit us to study patterns of systematic variation in much the same way as ordinary linear models are used to study the joint effects of treatments and covariates.

Some continuous measurements encountered in practice have non-Normal error distributions, and the class of generalized linear models includes distributions useful for the analysis of such data. The simplest examples are perhaps the exponential and gamma distributions, which are often useful for modelling positive data that have positively skewed distributions, such as occur in studies of survival times.

Before looking in more detail at the history of individual instances of generalized linear models, we make some general comments about statistical models and the part they play in the analysis of data, whether experimental or observational.

1.1.1 The problem of looking at data

Suppose we have a number of measurements or counts, together with some associated structural or contextual information, such as the order in which the data were collected, which measuring instruments were used, and other differences in the conditions under which the individual measurements were made. To interpret such data, we search for a pattern, for example that one measuring instrument has produced consistently higher readings than another. Such systematic effects are likely to be blurred by other variation of a more haphazard nature. The latter variation is usually described in statistical terms, no attempt being made to model or to predict the actual haphazard contribution to each observation.

Statistical models contain both elements, which we will call systematic effects and random effects. The value of a model is that often it suggests a simple summary of the data in terms of the major systematic effects together with a summary of the nature and magnitude of the unexplained or random variation. Such a reduction is certainly helpful, for the human mind, while it may be able to encompass say 10 numbers easily enough, finds 100 much more difficult, and will be quite defeated by 1000 unless some reducing process takes place.

Thus the problem of looking intelligently at data demands the formulation of patterns that are thought capable of describing succinctly not only the systematic variation in the data under study, but also for describing patterns in similar data that might be collected by another investigator at another time and in another place.

1.1.2 Theor...

Table of contents

• Cover
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• Preface to the first edition
• Preface
• 1 Introduction
• 2 An outline of generalized linear models
• 3 Models for continuous data with constant variance
• 4 Binary data
• 5 Models for polytomous data
• 6 Log-linear models
• 7 Conditional likelihoods*
• 8 Models with constant coefficient of variation
• 9 Quasi-likelihood functions
• 10 Joint modelling of mean and dispersion
• 11 Models with additional non-linear parameters
• 12 Model checking
• 13 Models for survival data
• 14 Components of dispersion
• 15 Further topics
• Appendices
• References
• Index of data sets
• Author index
• Subject index