Transformation and Weighting in Regression

Raymond J. Carroll, David Ruppert

• Publisher: Chapman and Hall/CRC
• Year: 2017
• Print ISBN: 9780412014215
• eBook ISBN: 9781351407267
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

This monograph provides a careful review of the major statistical techniques used to analyze regression data with nonconstant variability and skewness. The authors have developed statistical techniques--such as formal fitting methods and less formal graphical techniques-- that can be applied to many problems across a range of disciplines, including pharmacokinetics, econometrics, biochemical assays, and fisheries research. While the main focus of the book in on data transformation and weighting, it also draws upon ideas from diverse fields such as influence diagnostics, robustness, bootstrapping, nonparametric data smoothing, quasi-likelihood methods, errors-in-variables, and random coefficients. The authors discuss the computation of estimates and give numerous examples using real data. The book also includes an extensive treatment of estimating variance functions in regression.

CHAPTER 1

Introduction

1.1 Preliminaries

When modeling data it is often assumed that, in the absence of randomness or error, one can predict a response y from a predictor x through the deterministic relationship

$y=f\left(x,\beta \right)$

(1.1)

where β is a regression parameter. The equation (1.1) is often a theoretical (biological or physical) model, but it may also be an empirical model that seems to work well in practice, e.g., a linear regression model. In either case, once we have determined β then the system will be completely specified.

Observed data almost never fit a model exactly, so that β cannot be determined exactly. Such a situation occurs when the relationship (1.1) holds exactly, but we are able to observe the response y or the predictor x only with error. This is the useful perspective behind the ‘errors-in-variables’ literature, although it is not the standard explanation why observed data do not fit models exactly. Most statistical analyses assume that we can measure the predictor x accurately, but that, given a value of x, the relationship with the observed response y is not deterministic. Partly this might be due to measurement error in y, but there are also other factors to consider. The lack of determinism could be a physical fact, e.g., the physical process generating an observed y from x might not be deterministic. Also, the model might not be exact either because of slight misspecification or because certain other predictors might have been omitted.

Understanding the variability of the response is important. Efficient estimation of the unknown parameter β requires knowledge of the structure of the errors. Also, understanding the distribution of y may be important in itself, particularly in prediction and calibration problems.

A good example of these issues is the relationship between the number of fish in a spawning population (‘spawners’ denoted by S) and the number of new fish eventually recruited into the fishery (‘recruits’ denoted by R). A standard theoretical deterministic model is due to Ricker (1954), and can be written as

$R={\beta }_{1}S\exp \left(-{\beta }_{2}S\right)=f_{\text{RK}}\left(S,\beta \right)$

(1.2)

for β₂ ⩾ 0. This model is discussed at length in Chapter 4. In practice, the spawner–recruit relationship is not deterministic, and in fact there is considerable error in the equation. Partly, this is due to difficulties in measuring R and S, but perhaps even more important is the existence of unobserved environmental factors, e.g., ocean currents, pollution, predator depletion, and availability of food. Estimating the parameter β is important, but just as important for managing the fishery is understanding the distribution of the recruitment given the size of the spawning population. Only after this distribution is understood can one begin to assess the impact of different management strategies on the long-term productivity of the fishery.

In Ruppert et al. (1985) and Reish et al. (1985), a stochastic model for the Atlantic menhaden fishery is used to analyze the risk inherent in management strategies. The risk is largely due to the stochastic nature of recruitment, though estimation error in the parameters is also a factor. Certain harvesting policies, such as constant-catch policies where the same size catch is taken every year, are optimal for a deterministic fishery, but perform surprisingly poorly in stochastic situations. The main point is that the size and nature of management risk can only be assessed if a realistic model of recruitment variability (conditional on the size of the spawning population) is used.

1.2 The classical regression model

The usual regression model makes four basic assumptions and the analyses are often based on a fifth. The first four assumptions are that the model is correctly specified on average, and that the errors are independent, have the same distribution, and have constant variability, i.e.

$E\left(y\right)=\text{expected value of}y=f\left(x,\beta \right)$

(1.3)

$y-f\left(x,\beta \right)=\epsilon ,\mathrm{Variance}\left(y\right)=\text{Variance}\left(\epsilon \right)={\sigma }^2$

(1.4)

$\begin{array}{l}\text{The errors}\epsilon \text{have the same distribution for each}\\ \text{given value of}x\end{array}$

(1.5)

$\begin{array}{l}\text{Given}x\text{, the errors}\epsilon =y-f\left(x,\beta \right)\text{are independently}\\ \text{distributed}\end{array}$

(1.6)

Of course, (1.5) implies (1.4). Sometimes only (1.4) is assumed, so we list it separately. Most often, the parameter β is estimated by least squares, the basic motivation for which is that it is efficient if the errors are normally distributed. This leads to the fifth assumption, i.e.

$\text{The errors are normally distributed}$

(1.7)

To see how these assumptions might work in practice, we consider the spawner-recruit relationship for the Skeena River sockeye salmon. The data are described in Chapter 4 and are plotted in Figure 4.1. In Figure 4.3, we plot the predicted values against the residuals. An examination of these two figures suggests that at least two of the major classical assumptions are strongly violated. The first assumption (1.3) seems reasonable, as the Ricker curve appears adequate. Whether the independence assumption (1.6) holds is not clear here, but it can be taken as a working hypothesis especially in light of the small size of the data set. A check of the serial correlations of the residuals suggests that (1.6) is not seriously violated. As can be seen from either figure, the variability of the data increases as a function of the size of the spawning population, thus violating assumptions (1.4) and (1.5). The normality assumption (1.7) seems adequate, although there may be some right-skewness.

The major purpose of this book is to discuss methods for analyzing regression data when the classical assumptions of constant variance and/or normality are violated.

Having observed that the variances are not constant, we might take as a working hypothesis that the variances are proportional to the square of the mean. This is a model with a constant coefficient of variation (standard deviation divided by mean). This model involves no additional parameters, but requires the a priori assumption about the variance function. A more flexible approach is to use a variance model with unknown parameters, for example to assume that the variance is proportional to some (unknown) power of the mean. To obtain a better estimate of the regression parameter β and to understand the distribution of the size of the recruit population given a spawning population size, two basic strategies can be employed.

First, we might continue to assume normally distributed obs...

Table of contents

• Cover
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• Preface
• 1 Introduction
• 2 Generalized least squares and the analysis of heteroscedasticity
• 3 Estimation and inference for variance functions
• 4 The transform-both-sides methodology
• 5 Combining transformations and weighting
• 6 Influence and robustness
• 7 Technical complements
• 8 Some open problems
• References
• Author Index
• Subject Index