The Foundations of Statistics

Leonard J. Savage

• Publisher: Dover Publications
• Year: 2012
• eBook ISBN: 9780486137100
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

Classic analysis of the foundations of statistics and development of personal probability, one of the greatest controversies in modern statistical thought. Revised edition. Calculus, probability, statistics, and Boolean algebra are recommended.

CHAPTER 1

Introduction

1 The role of foundations

It is often argued academically that no science can be more secure than its foundations, and that, if there is controversy about the foundations, there must be even greater controversy about the higher parts of the science. As a matter of fact, the foundations are the most controversial parts of many, if not all, sciences. Physics and pure mathematics are excellent examples of this phenomenon. As for statistics, the foundations include, on any interpretation of which I have ever heard, the foundations of probability, as controversial a subject as one could name. As in other sciences, controversies over the foundations of statistics reflect themselves to some extent in everyday practice, but not nearly so catastrophically as one might imagine. I believe that here, as elsewhere, catastrophe is avoided, primarily because in practical situations common sense generally saves all but the most pedantic of us from flagrant error. It is hard to judge, however, to what extent the relative calm of modern statistics is due to its domination by a vigorous school relatively well agreed within itself about the foundations.

Although study of the foundations of a science does not have the role that would be assigned to it by naive first-things-firstism, it has a certain continuing importance as the science develops, influencing, and being influenced by, the more immediately practical parts of the science.

2 Historical background

The concept and problem of inductive inference have been prominent in philosophy at least since Aristotle. Mathematical work on some aspects of the problem of inference dates back at least to the early eighteenth century. Leibniz is said to be the first to publish a suggestion in that direction, but Jacob Bernoulli’s posthumous Ars Conjectandi (1713) [B12] seems to be the first concerted effort.† This mathematical work has always revolved around the concept of probability; but, though there was active interest in probability for nearly a century before the publication of Ars Conjectandi, earlier activity seems not to have been concerned with inductive inference.

In the present century there has been and continues to be extraordinary interest in mathematical treatment of problems of inductive inference. For reasons I cannot and need not analyze here, this activity has been strikingly concentrated in the English-speaking world. It is known under several names, most of which stress some aspect of the subject that seemed of overwhelming importance at the moment when the name was coined. "Mathematical statistics," one of its earliest names, is still the most popular. In this name, "mathematical" seems to be intended to connote rational, theoretical, or perhaps mathematically advanced, to distinguish the subject from those problems of gathering and condensing numerical data that can be considered apart from the problem of inductive inference, the mathematical treatment of which is generally relatively trivial. The name "statistical inference" recognizes that the subject is concerned with inductive inference. The name "statistical decision" reflects the idea that inductive inference is not always, if ever, concerned with what to believe in the face of inconclusive evidence, but that at least sometimes it is concerned with what action to decide upon under such circumstances. Within this book, there will be no harm in adopting the shortest possible name, "statistics."

It is unanimously agreed that statistics depends somehow on probability. But, as to what probability is and how it is connected with statistics, there has seldom been such complete disagreement and breakdown of communication since the Tower of Babel. There must be dozens of different interpretations of probability defended by living authorities, and some authorities hold that several different interpretations may be useful, that is, that the concept of probability may have different meaningful senses in different contexts. Doubtless, much of the disagreement is merely terminological and would disappear under sufficiently sharp analysis. Some believe that it would all disappear, or even that they have themselves already made the necessary analysis.

Considering the confusion about the foundations of statistics, it is surprising, and certainly gratifying, to find that almost everyone is agreed on what the purely mathematical properties of probability are. Virtually all controversy therefore centers on questions of interpreting the generally accepted axiomatic concept of probability, that is, of determining the extramathematical properties of probability.

The widely accepted axiomatic concept referred to is commonly ascribed to Kolmogoroff [K7] and goes by his name. It should be mentioned that there is some dissension from it on the part of a small group led by von Mises [V2]. There are also a few minor technical variations on the Kolmogoroff system that are sometimes of interest; they will be discussed in § 3.4.

I would distinguish three main classes of views on the interpretation of probability, for the purposes of this book, calling them objectivistic, personalistic, and necessary. Condensed descriptions of these three classes of views seem called for here. If some readers find these descriptions condensed to the point of unintelligibility, let them be assured that fuller ones will gradually be developed as the book proceeds.

Objectivistic views hold that some repetitive events, such as tosses of a penny, prove to be in reasonably close agreement with the mathematical concept of independently repeated random events, all with the same probability. According to such views, evidence for the quality of agreement between the behavior of the repetitive event and the mathematical concept, and for the magnitude of the probability that applies (in case any does), is to be obtained by observation of some repetitions of the event, and from no other source whatsoever.

Personalistic views hold that probability measures the confidence that a particular individual has in the truth of a particular proposition, for example, the proposition that it will rain tomorrow. These views postulate that the individual concerned is in some ways "reasonable," but they do not deny the possibility that two reasonable individuals faced with the same evidence may have different degrees of confidence in the truth of the same proposition.

Necessary views hold that probability measures the extent to which one set of propositions, out of logical necessity and apart from human opinion, confirms the truth of another. They are generally regarded by their holders as extensions of logic, which tells when one set of propositions necessitates the truth of another.

After what has been said about the intensity and complexity of the controversy over the probability concept, you must realize that the short taxonomy above is bound to infuriate any expert on the foundations of probability, but I trust it may do the less learned more good than harm.

The great burst of statistical research in the English-speaking world in the present century has revolved around objectivistic views on the interpretation of probability. As will shortly be explained, any purely objectivistic view entails a severe difficulty for statistics. This difficulty is recognized by members of the British-American School, if I may use that name without its being taken too literally or at all nationalistically, and is regarded by them as a great, though not insurmountable, obstacle; indeed, some of them see it as the central problem of statistics.

The difficulty in the objectivistic position is this. In any objectivistic view, probabilities can apply fruitfully only to repetitive events, that is, to certain processes; and (depending on the view in question) it is either meaningless to talk about the probability that a given proposition is true, or this probability can be only 1 or 0, according as the proposition is in fact true or false. Under neither interpretation can probability serve as a measure of the trust to be put in the proposition. Thus the existence of evidence for a proposition can never, on an objectivistic view, be expressed by saying that the proposition is true with a certain probability. Again, if one must choose among several courses of action in the light of experimental evidence, it is not meaningful, in terms of objective probability, to compute which of these actions is most promising, that is, which has the highest expected income. Holders of objectivistic views have, therefore, no recourse but to argue that it is not reasonable to assign probabilities to the truth of propositions or to calculate which of several actions is the most promising, and that the need expressed by the attempt to set up such concepts must be met in other ways, if at all.

The British-American School has had great success in several respects. The number of its adherents has rapidly increased. It has contributed many procedures of strong intuitive appeal and (one feels) of lasting worth. These have found widespread application in many sciences, in industry, and in commerce. The success of the school may pragmatically be taken as evidence for the correctness of the general view on which it is based. Indeed, anyone who overthrows that view must either discredit the procedures to which it has led, or show, as I hope to show in this book, that they are on the whole consistent with the alternative proposed.

Some, I among them, hold that the grounds for adopting an objectivistic view are not overwhelmingly strong; that there are serious logical objections to any such view; and, most important of all, that the difficulty a strictly objectivistic view meets in statistics reflects real inadequacy.

3 General outline of this book

This book presents a theory of the foundations of statistics which is based on a personalistic view of probability derived mainly from the work of Bruno de Finetti, as expressed for example in [D2]. The theory is presented in a tentative spirit, for I realize that the serious blemishes in it apparent to me are not the only ones that will be discovered by critical readers. A theory of the foundations of statistics that appears contrary to the teaching of the most productive statisticians will properly be regarded with extraordinary caution. Other views on probability will, of course, be discussed in this book, partly for their own interest and partly to explain the relationship between the personalistic view on which this book is based and other views.

The book is organized into seventeen chapters, of which the present introduction is the first. Chapters 2–7 are, so to speak, concerned with the foundations at a relatively deep level. They develop, explain, and defend a certain abstract theory of the behavior of a highly idealized person faced with uncertainty. That theory is shown to have as implications a theory of personal probability, corresponding to the personalistic view of probability basic to this book, and also a theory of utility due, in its modern form, to von Neumann and Morgenstern [V4].

There is a transition, occurring in Chapter 8 and maintained throughout the rest of the book, to a shallower level of the foundations of statistics; I might say from pre-statistics to statistics proper. In those later chapters, it is recognized that the theory developed in the earlier ones is too highly idealized for immediate application. Some compromises have to be made, and the appropriate ones are sought in an analysis of some of the inventions and ideas of the British-American School. It will, I hope, be demonstrated thereby that the superficially incompatible systems of ideas associated on the one hand with a personalistic view of probability and on the other with the objectivistically inspired developments of the British-American School do in fact lend each other mutual support and clarification.

 

† Valuable information on this and other topics of the early philosophic history of probability is attractively presented in Keynes' treatise [K4], especially in Chapters VII, XXIII, and the bibliography.

CHAPTER 2

Preliminary Considerations on Decision in the Face of Uncertainty

1 Introduction

Decisions made in the face of uncertainty pervade the life of every individual and organization. Even animals might be said continually to make such decisions, and the psychological mechanisms by which men decide may have much in common with those by which animals do so. But formal reasoning presumably plays no role in the decisions of animals, little in those of children, and less than might be wished in those of men. It may be said to be the purpose of this book, and indeed of statistics generally, to discuss the implications of reasoning for the making of decisions.

Reasoning is commonly associated with logic, but it is obvious, as many have pointed out, that the implications of what is ordinarily called logic are meager indeed when uncertainty is to be faced. It has therefore often been asked whether logic cannot be extended, by principles as acceptable as those of logic itself, to bear more fully on uncertainty. An attempt to extend logic in this way will be begun in this chapter, differing in two important respects from most, but not all, other attempts.

First, since logic is concerned with implications among propositions, many have thought it natural to extend logic by setting up criteria for the extent to which one proposition tends to imply, or provide evidence for, another. It seems to me obvious, however, that what is ultimately wanted is criteria for deciding among possible courses of action; and, therefore, generalization of the relation of implicati...

Table of contents

• Cover
• Title Page
• Copyright Page
• Dedication
• Preface to the Dover Edition
• Preface to the First Edition
• Acknowledgement
• Contents
• 1. Introduction
• 2. Preliminary Considerations on Decision in the face of uncertainty
• 3. PERSONAL PROBABILITY
• 4. Critical comments on personal probability
• 5. Utility
• 6. Observation
• 7. Partition problems
• 8. Statistics proper
• 9. Introduction to the minimax theory
• 10. A Personalistic reinterpretation of the minimax theory
• 11. The parallelism between the minimax theory and the theory of two-person games
• 12. The Mathematics of minimax problems
• 13. Objections to the Minimax Rules
• 14. The minimax theory applied to observations
• 15. Point Estimation
• 16. Testing
• 17. Interval Estimation and related topics
• Appendix 1. Expected value
• Appendix 2. Convex Functions
• Appendix 3. Bibliographic Material
• Appendix 4. Bibliographic Supplement
• Technical Symbols
• Author Index
• General Index