Sei sulla pagina 1di 6

A FIRST COURSE IN

STOCHASTIC

SECOND EDITION

SAMUEL KARLIN

STANFORD UNIVERSITY AND THE WEIZMANN INSTITUTE OF SCIENCE

PROCESSES

HOWARD

M.

CORNELL UNIVERSITY

TAYLOR

ACADEMIC PRESS

New York

San Francisco

London

A Subsidiary of Harcourt Brace Jovanovich, Publishers

CONTENTS

Preface

Preface

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

xi

to First Edition

 

.

.

.

.

.

.

.

.

xv

Chapter 1

 

ELEMENTS OF

STOCHASTIC

PROCESSES

 

1. Review of Basic Terminology and Properties of Random Variables

 
 

and Distribution Functions

.

.

.

.

.

.

.

.

.

.

1

2. Two Simple Examples of Stochastic Processes

.

.

.

.

.

.

20

3. Classification of General Stochastic Processes

.

.

.

.

.

.

26

4. Denning a Stochastic Process

Chapter 2

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

32

Elementary Problems

.

.

.

.

.

.

.

.

.

33

Problems Notes References

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

36

44

44

MARKOV

CHAINS

 

1.

Definitions

.

.

.

.

.

.

.

.

.

.

.

.

45

2.

3.

. Transition Probability Matrices of a Markov Chain

Examples of Markov Chains

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

47

58

vi

CONTENTS

4. Classification of States of a Markov Chain

.

.

.

.

.

.

.

.

59

5. Recurrence

.

.

.

.

.

.

.

.

.

.

62

6. Examples of Recurrent Markov Chains

.

.

.

.

.

.

.

.

67

7. More on Recurrence

 

.

.

.

.

.

.

.

.

.

.

72

Elementary Problems

.

.

.

.

.

.

.

.

73

Problems

.

.

.

.

.

.

.

.

.

.

.

.

77

Notes

.

.

.

.

.

.

.

.

.

.

.

.

79

References

.

.

.

.

.

.

.

.

.

.

.

.

80

Chapter 3

THE BASIC LIMIT APPLICATIONS

 

THEOREM

OF MARKOV CHAINS AND

 

1. Discrete

Renewal Equation

.

.

.

.

.

.

.

.

.

.

81

2. Proof of

Theorem 1.1

 

.

.

.

.

.

.

.

.

.

.

87

3. Absorption

Probabilities

.

.

.

.

.

.

.

.

.

.

89

4. Criteria for

Recurrence

 

.

.

.

.

.

.

.

.

.

.

94

5. A Queueing Example

 

.

.

.

.

.

.

.

.

.

.

96

6. Another Queueing Model

.

.

.

.

.

.

.

.

.

.

102

7. Random Walk

.

.

.

.

.

.

.

.

.

.

.

.

106

Elementary Problems

 

.

.

.

.

.

.

.

.

.

.

108

Problems

.

.

.

.

.

.

.

.

.

.

.

.

112

Notes

.

.

.

.

.

.

.

.

.

.

.

.

.

.

116

Reference

.

.

.

.

.

.

.

.

.

.

.

.

116

Chapter 4

CLASSICAL EXAMPLES CHAINS

O F CONTINUOUS TIME

MARKOV

 

1. General Pure Birth Processes and Poisson Processes

.

.

.

.

.

.

117

2. More about Poisson Processes

.

.

.

.

.

.

.

.

.

.

123

3. A Counter Model

.

.

.

.

.

.

.

.

.

.

.

.

128

4. Birth and Death Processes

.

.

.

.

.

.

.

.

.

.

131

5. Differential Equations of Birth and Death Processes

.

.

.

.

.

.

135

6. of Birth and Death Processes

Examples

 

.

.

.

.

.

.

137

7. Death Processes with Absorbing States

Birth and

.

.

.

.

.

.

145

8. Finite State Continuous Time Markov Chains

 

.

.

.

.

.

.

150

Elementary Problems

 

.

.

.

.

.

.

.

.

.

.

152

Problems

.

.

.

.

.

.

.

.

.

.

.

.

158

Notes

.

.

.

.

.

.

.

.

.

.

.

.

165

References

.

.

.

.

.

.

.

.

.

.

.

.

166

CONTENTS

~

vii

Chapter 5

RENEWAL

PROCESSES

 

1.

Definition of a Renewal Process and Related Concepts

 

.

.

.

.

167

2. Examples of Renewal Processes

Some

.

.

.

.

.

.

.

.

170

3. on Some Special Renewal Processes

More

.

.

.

.

.

.

.

.

173

4. Renewal Equations and the Elementary Renewal Theorem

.

.

.

.

181

5. The Renewal Theorem

 

.

.

.

.

.

.

.

.

.

.

189

6. the Renewal Theorem

Applications of

 

.

.

.

.

.

.

192

7. and Variations on Renewal Processes

Generalizations

.

.

.

.

.

.

197

8. More Elaborate Applications of Renewal Theory

.

.

.

.

.

.

212

9. Superposition of Renewal Processes

.

.

.

.

.

.

.

.

221

Elementary Problems

.

.

.

.

.

.

.

.

.

.

228

Problems

.

.

.

.

.

.

.

.

.

.

.

.

230

Reference

.

.

.

.

.

.

.

.

.

.

.

.

237

Chapter 6

MARTINGALES

1. Preliminary Definitions and Examples

.

.

.

.

.

.

.

.

238

2. Supermartingales and Submartingales

.

.

.

.

.

.

.

.

248

3. The Optional Sampling Theorem

 

.

.

.

.

.

.

.

,

253

4. Some Applications of the Optional Sampling Theorem

.

.

.

.

.

.

263

5. Martingale Convergence Theorems

 

.

.

.

.

.

.

.

.

278

6. Extensions of the Martingale Convergence Theorems

Applications and

.

.

287

7. Respect to <r-Fields .

8.

Martingales with

Other Martingales

.

.

.

.

.

.

.

.

.

.

.

. .

.

.

.

.

297

313

Elementary Problems

.

.

.

.

.

.

.

.

.

.

325

Problems

.

.

.

.

.

.

.

.

.

.

.

.

330

Notes

.

.

. x .

.

 

.

.

339

References

.

.

.

.

.

.

.

.

.

.

.

339

Chapter 7 BROWNIAN MOTION

 

1. Background Material

.

.

.

.

.

.

.

.

.

.

340

2. Joint Probabilities for Brownian Motion

.

.

.

.

.

.

.

.

343

3. Continuity of Paths and the Maximum Variables

.

.

.

.

.

.

345

4. Variations and Extensions

.

 

.

.

.

.

.

.

.

.

.

351

5. Computing Some Functionals of Brownian Motion by Martingale Methods

.

.

357

6. Brownian Motion

Multidimensional

.

.

.

.

.

.

.

.

365

7. Brownian Paths

.

.

.

.

.

.

.

.

371

viii

CONTENTS

 

Elementary Problems

.

.

.

.

.

.

.

.

383

Problems

.

.

.

.

.

.

.

.

.

.

386

Notes

.

.

.

.

.

.

.

.

.

391

References

.

.

.

.

.

.

.

391

Chapter 8

 

BRANCHING

PROCESSES

 

1. Discrete Time Branching Processes

 

.

.

.

.

.

.

.

.

392

2. Generating Function Relations for Branching Processes

 

.

.

.

.

394

3. Extinction Probabilities

 

.

.

.

.

.

.

.

.

.

.

396

4.

Examples

.

.

.

.

.

.

.

.

.

.

.

.

400

5. Two-Type Branching Processes

 

.

.

.

.

.

.

.

.

.

.

404

6. Branching Processes

Multi-Type

 

.

.

.

.

.

.

.

.

411

7. Time Branching Processes

Continuous

.

.

.

.

.

.

.

.

412

8. Extinction Probabilities for Continuous Time

Branching

Processes

.

.

416

9. Limit Theorems for Continuous Time Branching Processes

 

.

.

.

.

419

10.

Two-Type

Continuous Time Branching Process

 

.

.

.

.

.

.

424

11.

Branching

Processes with General Variable Lifetime

.

.

.

.

.

.

431

Elementary Problems

.

.

.

.

.

.

.

.

.

.

436

Problems

43

8

Notes

.

.

.

.

.

.

.

.

.

.

.

.

.

.

442

Reference

.

.

.

.

.

.

.

.

.

.

.

.

442

Chapter 9

 

STATIONARY

PROCESSES

 

1. Definitions and Examples

 

.

.

.

.

.

.

.

.

.

.

443

2. Mean

Square

Distance

 

.

.

.

.

.

.

.

.

• •

 

451

3.

Mean Square

 

461

4. Prediction of

Error Prediction Covariance Stationary Processes

.

.

.

.

.

.

470

5.

Ergodic Theory and Stationary Processes

.

.

.

.

.

.

.

.

474

6.

Applications of Ergodic Theory

.

.

.

.

.

.

. •

489

7. Spectral Analysis of Covariance

8.

Gaussian Systems

.

Stationary Processes

.

.

.

.

.

.

.

.

.

.

.

.

*•

.

. •

.

502

510

9.

. Stationary Point Processes

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

516

10.

The Level-Crossing Problem

.

.

.

.

.

.

.

.

.

.

519

Elementary Problems

.

.

.

.

.

.

.

.

.

.

524

Problems Notes References

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

• •

.

.

.

.

• •

527

534

535

CONTENTS

~

ix

Appendix

REVIEW

OF MATRIX

ANALYSIS

 

1. Spectral Theorem

The

 

.

.

.

.

.

.

.

.

.

.

536

2. Frobenius Theory of Positive Matrices .

The

.

.

.

.

.

.

.

542

Index

.

.

.

.

.

.

.

.

. •

.•

• •

553