INTRODUCTION TO STOCHASTIC PROCESSES:

Loosely speaking, a stochastic or random process is a sequence of events occurring over

time, where each step follows from the last after a random choice. More precisely, stochastic
processes are essentially mathematical models used to model time(or space) dependence
of a random phenomenon usually referred to as a system. The complete evolution of the
system is modelled by assigning a random variable to each point in time. The collection
of all these random variables is called a stochastic process.
When the random variables assigned are independent, the task of studying a stochastic
process is relatively easy. But assuming independence is not always possible as it is very
often does not reflect realism of systems being modeled. In addition, dependence is some-
times a desirable property as it helps in predicting the future evolution of the system from
its past evolution. To deal with the dependence of the random variables, we usually need
to make some simplifying assumptions about the nature of dependence.
The most common assumptions made are:

(i) That the future evolution of the system is dependent on the past only through the
present state of the system: this is called the Markov property.

(ii) That within a certain time duration, the system will behave in the future similarly as
it has behaved in the past within same time duration: this is the notion of stationarity.

(iii) That the evolution of the processes within non overlapping time intervals are inde-
pendent or identically distributed: this is the notion of independent and stationary
increments

A more formal and mathematical definition of a stochastic process is:

Definition: A stochastic process is a family of random variables defined over a given

probability space; and the variables are indexed by a parameter(usually time) which varies
over a suitable index set.
For example we define a stochastic process by {X(t) : tT } where X(t) is the random
variable, t is the parameter and T is the index set. The same process can be defined as
{Xt : tT }. The possible values X(t) can take are referred to as states or realizations and
the set of all states of the process is known as the state space. The random variables may
be defined on an n dimension; n ≥ 1.

1
 If the state space is discrete then the process is said to be a chain or discrete space
process; while if it is continuous then it is a continuous state space process. Similarly, if
T , the index set is discrete then the process is said to be discrete parameter process while
it T is continuous, then the process is continuous parameter process. This classification
leads to four kinds of stochastic processes

• Discrete state-disrete parameter process: e.g Branching process, Markov chains etc

• Discrete state-continuous parameter process: e.g Poisson process, birth-death pro-

cesses.

• Continuous state-discrete parameter process: e.g A time series

• continuous space-continuous parameter process e.g Diffusion process

Examples of stochastic processes :

(i) Let {Xt : tT } be a stochastic process that models the state of health of policyholders
of a life insurance company. The company classifies its policyholders as either healthy,
sick or dead. Then tha state space usually denoted by S is S = {Healthy, sick, Dead}.
If the health states of policyholders are observed continuously, then index set T is
T = [0, ∞) and the process is discrete state-continuous time process.
However if the health states of policyholders are observed daily, the index set T is T
= {0, 1, 2 . . .} and the process is discrete state-discrete time process.

(ii) Consider a model {Xt : tT } for the daily maximal temperatures observed in Nairobi;
Xt is the temperature reading observed on t − th day. The index set T is T is T = {0,
1, 2 . . .} is discrete and the state space is continuous since temperature can assume
any real value. Therefore the model is continuous state-discrete time process.

(iii) Let {Xt : tT } be a stochastic process that models the stock price of a company.
Then the state space S = <+ if we assume that the stock price can assume any
positive real number. The time space T can either be discrete or continuous. This
process can either be a continuous state-continuous time process or a continuous
state-discrete time process.

2
Classical Types of stochastic processes

Process with independent increments

A continuous time process is said to have independent increments if for parameters ti; i=
0, 1, .., n such that t0 < t1 < t2 < ... < tn the random variables

X(t1 ) − X(t0), X(t2 ) − X(t1), ..., X(tn) − X(tn−1 )

are independent of each other.

Process with stationary increments

A continuous time process is said to have stationary increments if the random variables
X(t + s) − X(t) ∀t have the same distribution; i.e distribution of the increments depend
only the length of time interval s.

Stationary process

If n variables X(t1 , X(t2), ..., X(tn) have the same distribution as X(t1 +h), X(t2+h), ..., X(tn+
h); h > 0; then the distribution of X(t) is the same for all t.

Probability generating functions

(1) The probability generating function (p.g.f) of a probability distribution function

P (Y = k) = pk , k = 0, 1, 2, .... is given by
X X
G(s) = E[S Y ] = sk P (Y = k) = sk pk |s| < 1
k k

(2) Let X and Y be two random variables with joint probability distribution function
Pr(X = x, Y = y) = f (x, y). The probability generating function of this bivariate
function is given by
XX
G(s, t) = E[S X T Y ] = f (x, y)sxty |s, t| < 1
x y

3
 Let X1 , X2 , ..., XN be independent and identically random variables and G(s) be the
P
probability generating function of Xi . Also let SN = N i=1 Xi ;

(i) if N is a fixed number, p.g.f of SN is [G(s)]N . The mean and variance of SN are
given by
E[SN ] = NE[X] and V ar(SN ) = NV ar(X)

(ii) if N is a random variable with p.g.f P(s); then p.g.f of SN is P [G(s)]. The mean and
variance of SN are given by

E[SN ] = E[X]E[N ] and V ar(SN ) = V ar(N )E[X]2 + E[N ]V ar(X)