Probability, Statistics, and Stochastic Processes

Peter Olofsson, Mikael Andersson

• Publisher: Wiley
• Year: 2012
• Print ISBN: 9780470889749
• eBook ISBN: 9781118231326
• Edition: 2
• Topic: Mathematics
• Subtopic: Operations
• Index: Mathematics

Praise for the First Edition

"... an excellent textbook... well organized and neatly written."
— Mathematical Reviews

"... amazingly interesting..."
— Technometrics

Thoroughly updated to showcase the interrelationships between probability, statistics, and stochastic processes, Probability, Statistics, and Stochastic Processes, Second Edition prepares readers to collect, analyze, and characterize data in their chosen fields.

Beginning with three chapters that develop probability theory and introduce the axioms of probability, random variables, and joint distributions, the book goes on to present limit theorems and simulation. The authors combine a rigorous, calculus-based development of theory with an intuitive approach that appeals to readers' sense of reason and logic. Including more than 400 examples that help illustrate concepts and theory, the Second Edition features new material on statistical inference and a wealth of newly added topics, including:

• Consistency of point estimators
  
  

• Large sample theory
  
  

• Bootstrap simulation
  
  

• Multiple hypothesis testing
  
  

• Fisher's exact test and Kolmogorov-Smirnov test
  
  

• Martingales, renewal processes, and Brownian motion
  
  

• One-way analysis of variance and the general linear model
  
  

Extensively class-tested to ensure an accessible presentation, Probability, Statistics, and Stochastic Processes, Second Edition is an excellent book for courses on probability and statistics at the upper-undergraduate level. The book is also an ideal resource for scientists and engineers in the fields of statistics, mathematics, industrial management, and engineering.

Chapter 1

Basic Probability Theory

1.1 Introduction

Probability theory is the mathematics of randomness. This statement immediately invites the question “What is randomness?” This is a deep question that we cannot attempt to answer without invoking the disciplines of philosophy, psychology, mathematical complexity theory, and quantum physics, and still there would most likely be no completely satisfactory answer. For our purposes, an informal definition of randomness as “what happens in a situation where we cannot predict the outcome with certainty” is sufficient. In many cases, this might simply mean lack of information. For example, if we flip a coin, we might think of the outcome as random. It will be either heads or tails, but we cannot say which, and if the coin is fair, we believe that both outcomes are equally likely. However, if we knew the force from the fingers at the flip, weight and shape of the coin, material and shape of the table surface, and several other parameters, we would be able to predict the outcome with certainty, according to the laws of physics. In this case we use randomness as a way to describe uncertainty due to lack of information.¹

Next question: “What is probability?” There are two main interpretations of probability, one that could be termed “objective” and the other “subjective.” The first is the interpretation of a probability as a limit of relative frequencies; the second, as a degree of belief. Let us briefly describe each of these.

For the first interpretation, suppose that we have an experiment where we are interested in a particular outcome. We can repeat the experiment over and over and each time record whether we got the outcome of interest. As we proceed, we count the number of times that we got our outcome and divide this number by the number of times that we performed the experiment. The resulting ratio is the relative frequency of our outcome. As it can be observed empirically that such relative frequencies tend to stabilize as the number of repetitions of the experiment grows, we might think of the limit of the relative frequencies as the probability of the outcome. In mathematical notation, if we consider n repetitions of the experiment and if S_n of these gave our outcome, then the relative frequency would be f_n = S_n/n, and we might say that the probability equals lim _n→∞ f_n. Figure 1.1 shows a plot of the relative 3 frequency of heads in a computer simulation of 100 hundred coin flips. Notice how there is significant variation in the beginning but how the relative frequency settles in toward

quickly.

Figure 1.1 Consecutive relative frequencies of heads in 100 coin flips.

The second interpretation, probability as a degree of belief, is not as easily quantified but has obvious intuitive appeal. In many cases, it overlaps with the previous interpretation, for example, the coin flip. If we are asked to quantify our degree of belief that a coin flip gives heads, where 0 means “impossible” and 1 means “with certainty,” we would probably settle for

unless we have some specific reason to believe that the coin is not fair. In some cases it is not possible to repeat the experiment in practice, but we can still imagine a sequence of repetitions. For example, in ...

Table of contents

• Cover
• Title Page
• Copyright
• Preface
• Preface to the First Edition
• The Book
• The People
• Chapter 1: Basic Probability Theory
• Chapter 2: Random Variables
• Chapter 3: Joint Distributions
• Chapter 4: Limit Theorems
• Chapter 5: Simulation
• Chapter 6: Statistical Inference
• Chapter 7: Linear Models
• Chapter 8: Stochastic Processes
• Appendix A: Tables
• Appendix B: Answers to Selected Problems
• Further Reading
• Index