Applied Probability Models with Optimization Applications

Sheldon M. Ross

• Publisher: Dover Publications
• Year: 2013
• Print ISBN: 9780486673141
• eBook ISBN: 9780486318646
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

"A clarity of style and a conciseness of treatment which students will find most welcome. The material is valuable and well organized … an excellent introduction to applied probability." — Journal of the American Statistical Association.
This book offers a concise introduction to some of the stochastic processes that frequently arise in applied probability. Emphasis is on optimization models and methods, particularly in the area of decision processes. After reviewing some basic notions of probability theory and stochastic processes, the author presents a useful treatment of the Poisson process, including compound and nonhomogeneous Poisson processes. Subsequent chapters deal with such topics as renewal theory and Markov chains; semi-Markov, Markov renewal, and regenerative processes; inventory theory; and Brownian motion and continuous time optimization models.
Each chapter is followed by a section of useful problems that illustrate and complement the text. There is also a short list of relevant references at the end of every chapter. Students will find this a largely self-contained text that requires little previous knowledge of the subject. It is especially suited for a one-year course in applied probability at the advanced undergraduate or beginning postgraduate level. 1970 edition.

1

INTRODUCTION TO STOCHASTIC PROCESSES

1.1. Random Variables and Probability Theory

In order to understand the theory of stochastic processes, it is necessary to have a firm grounding in the basic concepts of probability theory. As a result, we shall briefly review some of these concepts and at the same time establish some useful notation.

The distribution function F( ) of the random variable X is defined for any real number x by

A random variable X is said to be discrete if its set of possible values is countable. For discrete random variables, the probability mass function p(x) is defined by

Clearly,

A random variable is called continuous if there exists a function f(x), called the probability density function, such that

for every Borel set B. Since

, it follows that

The expectation or mean of the random variable X, denoted by EX, is defined by

provided the above integral exists.

Equation (1) also defines the expectation of any function of X, say h(X). Since h(X) is itself a random variable, it follows from (1) that

where F_h is the distribution function of h(X). However, it can be shown that this is identical to

, i.e.,

The above equation is sometimes known as the law of the unconscious statistician [as statisticians have been accused of using the identity (2) without realizing that it is not a definition].

The variance of the random variable X is defined by

Jointly Distributed Random Variables

The joint distribution F of two random variables X and Y is defined by

The distributions

and

are called the marginal distributions of X and Y. X and Y are said to be independent if

for all real x and y. It can be shown that X and Y are independent if and only if

for all functions g and h for which the above expectations exist.

Two jointly distributed random variables X and Y are said to be uncorrected if their covariance, defined by

is zero. It follows that independent random variables are uncorrected. However, the converse is not true. (Give an example.)

The random variables X and Y are said to be jointly continuous if there ...

Table of contents

• Cover
• Title Page
• Dedication
• Copyright Page
• Preface
• Table of Contents
• 1. Introduction to Stochastic Processes
• 2. The Poisson Process
• 3. Renewal Theory
• 4. Markov Chains
• 5. Semi-Markov, Markov Renewal and Regenerative Processes
• 6. Markov Decision Processes
• 7. Semi-Markov Decision Processes
• 8. Inventory Theory
• 9. Brownian Motion and Continuous Time Optimization Models
• Appendices
• Index