Statistics with Confidence

Confidence Intervals and Statistical Guidelines

Douglas Altman, David Machin, Trevor Bryant, Martin Gardner, Douglas Altman, David Machin, Trevor Bryant, Martin Gardner

• Publisher: BMJ Books
• Year: 2013
• ISBN:
  9781118702505
• Edition: 2
• Topic: Medicine
• Subtopic: Biostatistics

This highly popular introduction to confidence intervals has been thoroughly updated and expanded. It includes methods for using confidence intervals, with illustrative worked examples and extensive guidelines and checklists to help the novice.

Part I

Estimation and confidence intervals

1

Estimating with confidence

MARTIN J GARDNER, DOUGLAS G ALTMAN

Editors’ note: this chapter is reproduced from the first edition (with minor adjustments). It was closely based on an editorial published in 1988 in the British Medical Journal. Chapter 2 describes developments in the use of confidence intervals in the medical literature since 1988.

Statistical analysis of medical studies is based on the key idea that we make observations on a sample of subjects and then draw inferences about the population of all such subjects from which the sample is drawn. If the study sample is not representative of the population we may well be misled and statistical procedures cannot help. But even a well-designed study can give only an idea of the answer sought because of random variation in the sample. Thus results from a single sample are subject to statistical uncertainty, which is strongly related to the size of the sample. Examples of the statistical analysis of sample data would be calculating the difference between the proportions of patients improving on two treatment regimens or the slope of the regression line relating two variables. These quantities will be imprecise estimates of the values in the overall population, but fortunately the imprecision can itself be estimated and incorporated into the presentation of findings. Presenting study findings directly on the scale of original measurement, together with information on the inherent imprecision due to sampling variability, has distinct advantages over just giving P values usually dichotomised into “significant” or “non-significant”. This is the rationale for using confidence intervals.

The main purpose of confidence intervals is to indicate the (im)precision of the sample study estimates as population values. Consider the following points for example: a difference of 20% between the percentages improving in two groups of 80 patients having treatments A and B was reported, with a 95% confidence interval of 6% to 34% (see chapter 5). Firstly, a possible difference in treatment effectiveness of less than 6% or of more than 34% is not excluded by such values being outside the confidence interval—they are simply less likely than those inside the confidence interval. Secondly, the middle half of the 95% confidence interval (from 13% to 27%) is more likely to contain the population value than the extreme two quarters (6% to 13% and 27% to 34%)—in fact the middle half forms a 67% confidence interval. Thirdly, regardless of the width of the confidence interval, the sample estimate is the best indicator of the population value—in this case a 20% difference in treatment response.

The British Medical Journal now expects scientific papers submitted to it to contain confidence intervals when appropriate.¹ It also wants a reduced emphasis on the presentation of P values from hypothesis testing (see chapter 3). The Lancet,³ the Medical Journal of Australia,⁴ the American Journal of Public Health,⁵ and the British Heart Journal,⁶ have implemented the same policy, and it has been endorsed by the International Committee of Medical Journal Editors.⁷ One of the blocks to implementing the policy had been that the methods needed to calculate confidence intervals are not readily available in most statistical textbooks. The chapters that follow present appropriate techniques for most common situations. Further articles in the American Journal of Public Health and the Annals of Internal Medicine have debated the uses of confidence intervals and hypothesis tests and discussed the interpretation of confidence intervals.^8–14

So when should confidence intervals be calculated and presented? Essentially confidence intervals become relevant whenever an inference is to be made from the study results to the wider world. Such an inference will relate to summary, not individual, characteristics—for example, rates, differences in medians, regression coefficients, etc. The calculated interval will give us a range of values within which we can have a chosen confidence of it containing the population value. The most usual degree of confidence presented is 95%, but any suggestion to standardise on 95%¹⁵

Thus, a single study usually gives an imprecise sample estimate of the overall population value in which we are interested. This imprecision is indicated by the width of the confidence interval: the wider the interval the less the precision. The width depends essentially on three factors. Firstly, the sample size: larger sample sizes will give more precise results with narrower confidence intervals (see chapter 3). In particular, wide confidence intervals emphasise the unreliability of conclusions based on small samples. Secondly, the variability of the characteristic being studied: the less variable it is (between subjects, within subjects, from measurement error, and from other sources) the more precise the sample estimate and the narrower the confidence interval. Thirdly, the degree of confidence required: the more confidence the wider the interval.

1 Langman MJS. Towards estimation and confidence intervals. BMJ 1986;292:716.

2 Anonymous. Report with confidence [Editorial]. Lancet 1987;i:488.

3 Bulpitt CJ. Confidence intervals. Lancet 1987;i:494–7.

4 Berry G. Statistical significance and confidence intervals. Med J Aust 1986;144:618–19

5 Rothman KJ, Yankauer A. Confidence intervals vs significance tests: quantitative interpretation (Editors’ note). AmJ Public Health 1986;76:587–8.

6 Evans SJW, Mills P, Dawson J. The end of the P value? By Heart J 1988;60:177–80.

7 International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. BMJ 1988;296:401–5.

8 DeRouen TA, Lachenbruch PA, Clark VA, et al. Four comments received on statistical testing and confidence intervals. Am J Public Health 1987;77:237–8.

9 Anonymous. Four comments received on statistical testing and confidence intervals. Am J Public Health 1987;77:238.

10 Thompson WD. Statistical criteria in the interpretation of epidemiological data. Am J Public Health 1987;77:191–4.

11 Thompson WD. On the comparison of effects. Am J Public Health 1987;77:491–2.

12 Poole C. Beyond the confidence interval. AmJ Public Health 1987;77:195–9.

13 Poole C. Confidence intervals exclude nothing. Am J Public Health 1987;77:492–3.

14 Braitman, LE. Confidence intervals extract clinically useful information from data. Ann Intern Med 1988;108:296–8.

15 Gardner MJ, Altman DG. Using confidence intervals. Lancet 1987;i:746.

2

Confidence...