Statistical Inference

S.D. Silvey

• Publisher: Routledge
• Year: 2017
• eBook ISBN: 9781351414494
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

Statistics is a subject with a vast field of application, involving problems which vary widely in their character and complexity.However, in tackling these, we use a relatively small core of central ideas and methods. This book attempts to concentrateattention on these ideas: they are placed in a general settingand illustrated by relatively simple examples, avoidingwherever possible the extraneous difficulties of complicatedmathematical manipulation.In order to compress the central body of ideas into a smallvolume, it is necessary to assume a fair degree of mathematicalsophistication on the part of the reader, and the book is intendedfor students of mathematics who are already accustomed tothinking in rather general terms about spaces and functions

1 Introduction

1.1 Preliminaries

The theory of probability and statistics is concerned with situations in which an observation is made and which contain an element of inherent variability outwith the observer’s control. He knows that if he repeated the observation under conditions which were identical in so far as he could control them, the second observation would not necessarily agree exactly with the first. Thus when a scientist repeats an experiment in a laboratory, no matter how careful he may be in ensuring that experimental conditions do not vary from one repetition to the next, there will be variation in the observations resulting from the different repetitions, variation which in this context is often referred to as experimental error. Similarly, two apparently identical animals will not react in exactly the same way to some given treatment. Indeed it is the case that almost all situations in which observations are taken contain this element of variability. So the field of application of the theory which will be introduced in this book is extremely wide.

The possibility of formulating a mathematical theory to assist in the interpretation of observations in this kind of situation arises from the following phenomenon. Suppose we consider an experiment which can be repeated and whose result is ‘an observation’. This observation may belong to some stated set E of possible observations or it may not. If the observation belongs to E we shall say that ‘the event E has occurred’. Now suppose that the experiment is repeated n times and on each occasion we note whether or not E occurs. It transpires in practice that the proportion of times that a stated event E occurs in n independent repetitions of the experiment seems to settle down, as n increases, to a fixed number, depending of course on E. This is the phenomenon which is popularly referred to as the ‘law of averages’ and it is this law which underlies the whole theory of probability. It leads to the description of the inherent variability in the kind of situation we are discussing by a probability distribution or measure over the set of possible observations.

It will be assumed that the reader is familiar with the notion of a probability distribution and the way in which the ‘law of averages’ motivates its underlying axioms. (For a full account of this he may consult such books as those by Lindgren, 1962, Lindley, 1965, Feller, 1968, Meyer, 1965.) However since there are slight differences in usage we shall now explain the interpretation to be given to certain terms which will occur repeatedly.

1.1.2 Sample spaces

Possible observations in a situation under investigation will be represented in a mathematical set or space called a sample space. It is not necessary that there be a one-to-one correspondence between elements of a sample space and possible observations. Indeed, in a sophisticated treatment an observation is represented by a set of points for reasons which we need not elaborate at this stage. It will be sufficient for our purposes, at least initially, to regard a sample space as a set in which each possible observation is represented by a distinct element or point. It would be unnecessarily restrictive and would lead to clumsiness to insist that every point in a sample space should represent a possible observation and so we shall allow the possibility that a sample space is bigger than is absolutely essential for the representation we have in mind. We shall denote this space by X, its typical point by x, and we shall refer to ‘the observation x’. Of course X will vary according to the situation being investigated. It may be that each possible observation is a real number, in which case X may be taken as the set of real numbers. It happens frequently that each possible observation is an ordered set of n real numbers, in which case x = (x₁, x₂, …, x_n) and X may be taken as real n-space. It may even be that X is a space of functions, as, for example, when an observation is the curve traced by a barograph over a specified period of time.

1.1.3 Events

A subset E of a sample space X represents a real-life event, namely the event that the observation made belongs to the set of observations represented by E. Of course in everyday language a given event may not be described in this somewhat pedantic manner, but such a description of it is always possible. (To take an almost trivial example, if we consider rolling a die and observing the number on the face appearing uppermost, the event ‘an even face turns up’ is the event ‘the observation made belongs to the set {2, 4, 6} of possible observations’). Even if an event is described in everyday language it simplifies matters to think of it as simply a subset of a sample space, and because this habit of thought is so useful we shall often identify an event with the subset representing it and refer simply to ‘the event E’.

We shall assume that the reader is familiar with the interpretation of the standard set operations in terms of events, for example, E₁ ∪ E₂ is the event ‘either E₁ or E₂’.

1.1.4 Probability distributions

As we have already said, the inherent variability in the kind of situation which concerns us is described by a probability distribution on the set of possible o...

Table of contents

• Cover
• Title Page
• Copyright Page
• Table of Contents
• Preface
• 1 Introduction
• 2 Minimum-Variance Unbiased Estimation
• 3 The Method of Least Squares
• 4 The Method of Maximum Likelihood
• 5 Confidence Sets
• 6 Hypothesis Testing
• 7 The Likelihood-Ratio Test and Alternative ‘Large-Sample’ Equivalents of it
• 8 Sequential Tests
• 9.1 The Kolmogorov-Smirnov test
• 10 The Bayesian Approach
• 11 An Introduction to Decision Theory
• Appendix A Some Matrix Results
• Appendix Β The Linear Hypothesis
• References
• Index