Renewal theory

From Wikipedia, the free encyclopedia

Renewal theory is a branch of probability theory with an interesting and varied range of applications. Applications include calculating the expected time for a monkey who is randomly tapping at a keyboard to type the word Macbeth and comparing the longterm benefits of different insurance policies.

1 Renewal processes - an introduction
2 Formal definition
- 2.1 Interpretation
3 Renewal-reward processes
- 3.1 Interpretation
4 Properties of renewal processes and renewal-reward processes
5 Example applications
- 5.1 Example 1 - use of the strong law of large numbers
  - 5.1.1 Solution
6 See also

[edit] Renewal processes - an introduction

A renewal process is a generalisation of the Poisson process. In essence, the Poisson process is a continuous-time Markov process on the positive integers (usually starting at zero) which has independent identically distributed holding times at each integer $i$ (exponentially distributed) before advancing (with probability 1) to the next integer: $i + 1$ . In the same informal spirit, we may define a renewal process to be the same thing, except that the holding times take on a more general distribution. (Note however that the IID property of the holding times is retained).

[edit] Formal definition

Sample evolution of a renewal process with holding times S_i and jump times J_n.

Let $S_1 , S_2 , \ldots$ be a sequence of independent identically distributed random variables such that

$0 < \mathbb{E}[S_i] < \infty.$

We refer to the random variable $S i$ as the " $i$ th" holding time.

Define for each n > 0 :

$J_n = \sum_{i=1}^n S_i,$

each $J n$ referred to as the " $n$ th" jump time and the intervals

[J n, J n + 1]

being called renewal intervals.

Then the random variable $(X_t)_{t\geq0}$ given by

$X_t = \max \left\{\, n: J_n \leq t\, \right\}$

is called a renewal process.

[edit] Interpretation

One may choose to think of the holding times $\{ S_i : i \geq 1 \}$ as the time elapsed before a machine breaks for the " $i$ th" time since the last time it broke. (Note this assumes that the machine is immediately fixed and we restart the clock immediately.) Under this interpretation, the jump times $\{ J_n : n \geq 1 \}$ record the successive times at which the machine breaks and the renewal process $X t$ records the number of times the machine has so far had to be repaired at any given time $t$ .

However it is more helpful to understand the renewal process in its abstract form, since it may be used to model a great number of practical situations of interest which do not relate very closely to the operation of machines.

[edit] Renewal-reward processes

Sample evolution of a renewal-reward process with holding times S_i , jump times J_n and rewards W_i

Let $W_1, W_2, \ldots$ be a sequence of IID random variables (rewards) satisfying

$\mathbb{E}|W_i| < \infty$ .

Then the random variable

$Y_t = \sum_{i=1}^{X_t}W_i$

is called a renewal-reward process. Note that unlike the $S i$ , each $W i$ may take negative values as well as positive values.

[edit] Interpretation

In the context of the above interpretation of the holding times as the time between successive malfunctions of a machine, the "rewards" $W_1,W_2,\ldots$ (which in this case happen to be negative) may be viewed as the successive repair costs incurred as a result of the successive malfunctions.

An alternative analogy is that we have a magic goose which lays eggs at intervals (holding times) distributed as $S i$ . Sometimes it lays golden eggs of random weight, and sometimes it lays toxic eggs (also of random weight) which require responsible (and costly) disposal. The "rewards" $W i$ are the successive (random) financial losses/gains resulting from successive eggs (i = 1,2,3,...) and $Y t$ records the total financial "reward" at time t.

[edit] Properties of renewal processes and renewal-reward processes

We define the renewal function:

$m(t) = \mathbb{E}[X_t].\,$

[edit] The elementary renewal theorem

The renewal function satisfies

$\lim_{t \to \infty} \frac{1}{t}m(t) = 1/\mathbb{E}[S_1].$

The proof of this statement is non-trivial and therefore omitted.

[edit] The renewal equation

The renewal function satisfies

$m(t) = F_S(t) + \int_0^t m(t-s) f_S(s)\, ds$

where $F S$ is the cumulative distribution function of $S 1$ and $f S$ is the corresponding probability density function.

[edit] Proof of the renewal equation

We may iterate the expectation about the first holding time:

$m(t) = \mathbb{E}[X_t] = \mathbb{E}[\mathbb{E}(X_t \mid S_1)].$

But by the Markov property

$\mathbb{E}(X_t \mid S_1=s) = \mathbb{I}_{\{t \geq s\}} \left( 1 + \mathbb{E}[X_{t-s}] ) \right).$

$m(t) = \mathbb{E}[X_t]$

$= \mathbb{E}[\mathbb{E}(X_t \mid S_1)]$

$= \int_0^\infty \mathbb{E}(X_t \mid S_1=s) f_S(s)\, ds$

$= \int_0^\infty \mathbb{I}_{\{t \geq s\}} \left( 1 + \mathbb{E}[X_{t-s}] \right) f_S(s)\, ds$

$= \int_0^t \left( 1 + m(t-s) \right) f_S(s)\, ds$

$= F_S(t) + \int_0^t m(t-s) f_S(s)\, ds,$

as required.

[edit] Asymptotic properties

$(X_t)_{t\geq0}$ and $(Y_t)_{t\geq0}$ satisfy

$\lim_{t \to \infty} \frac{1}{t} X_t = \frac{1}{\mathbb{E}S_1}$ (strong law of large numbers for renewal processes)

$\lim_{t \to \infty} \frac{1}{t} Y_t = \frac{1}{\mathbb{E}S_1} \mathbb{E}W_1$ (strong law of large numbers for renewal-reward processes)

almost surely.

[edit] Proof

First consider $(X_t)_{t\geq0}$ . By definition we have:

$J_{X_t} \leq t \leq J_{X_t+1}$

for all $t \geq 0$ and so

$\frac{J_{X_t}}{X_t} \leq \frac{t}{X_t} \leq \frac{J_{X_t+1}}{X_t}$

for all t ≥ 0.

Now since $0< \mathbb{E}S_i < \infty$ we have:

$X_t \to \infty$

as $t \to \infty$ almost surely (with probability 1). Hence:

$\frac{J_{X_t}}{X_t} = \frac{J_n}{n} = \frac{1}{n}\sum_{i=1}^n S_i \to \mathbb{E}S_1$

almost surely (using the strong law of large numbers); similarly:

$\frac{J_{X_t+1}}{X_t} = \frac{J_{X_t+1}}{X_t+1}\frac{X_t+1}{X_t} = \frac{J_{n+1}}{n+1}\frac{n+1}{n} \to \mathbb{E}S_1\cdot 1$

almost surely.

Thus (since

t / X t

is sandwiched between the two terms)

$\frac{1}{t} X_t \to \frac{1}{\mathbb{E}S_1}$

almost surely.

Next consider $(Y_t)_{t\geq0}$ . We have

$\frac{1}{t}Y_t = \frac{X_t}{t} \frac{1}{X_t} Y_t \to \frac{1}{\mathbb{E}S_1}\cdot\mathbb{E}W_1$

almost surely (using the first result and using the law of large numbers on

Y t

[edit] The inspection paradox

A curious feature of renewal processes is that if we wait some predetermined time t and then observe how large the renewal interval containing t is, we should expect it to be typically larger than a renewal interval of average size.

Mathematically the inspection paradox states: for any t > 0 the renewal interval containing t is stochastically larger than the first renewal interval. That is, for all x > 0 and for all t > 0:

$\mathbb{P}(S_{X_t+1} > x) \geq \mathbb{P}(S_1>x) = 1-F_S(x)$

where F_S is the cumulative distribution function of the IID holding times S_i.

[edit] Proof of the inspection paradox

The renewal interval determined by the random point t (shown in red) is stochastically larger than the first renewal interval.

Observe that the last jump-time before t is $J_{X_t}$ ; and that the renewal interval containing t is $S_{X_t+1}$ . Then

$\mathbb{P}(S_{X_t+1}>x)=\int_0^\infty \mathbb{P}(S_{X_t+1}>x \mid J_{X_t} = s) f_S(s) \, ds$

$= \int_0^\infty \mathbb{P}(S_{X_t+1}>x | S_{X_t+1}>t-s) f_S(s)\, ds$

$= \int_0^\infty \frac{\mathbb{P}(S_{X_t+1}>x \, , \, S_{X_t+1}>t-s)}{\mathbb{P}(S_{X_t+1}>t-s)} f_S(s) \, ds$

$= \int_0^\infty \frac{ 1-F(\max \{ x,t-s \}) }{1-F(t-s)} f_S(s) \, ds$

$= \int_0^\infty \min \left\{\frac{ 1-F(x) }{1-F(t-s)},\frac{ 1-F(t-s) }{1-F(t-s)}\right\} f_S(s) \, ds$

$= \int_0^\infty \min \left\{\frac{ 1-F(x) }{1-F(t-s)},1\right\} f_S(s) \, ds$

$\geq 1-F(x)$

$= \mathbb{P}(S_1>x)$

as required.

[edit] Example applications

[edit] Example 1 - use of the strong law of large numbers

Eric the entrepreneur has n machines, each having an operational lifetime uniformly distributed between zero and two years. Eric may let each machine run until it fails with replacement cost £2600; alternatively he may replace a machine at any time while it is still functional at a cost of £200.

What is his optimal replacement policy?

[edit] Solution

We may model the lifetime of the n machines as n independent concurrent renewal-reward processes, so it is sufficient to consider the case n=1. Denote this process by $(Y_t)_{t \geq 0}$ . The successive lifetimes S of the replacement machines are independent and identically distributed, so the optimal policy is the same for all replacement machines in the process.

If Eric decides at the start of a machine's life to replace it at time 0 < t < 2 but the machine happends to fail before that time then the lifetime S of the machine is uniformly distributed on [0, t] and thus has expectation 0.5t. So the overall expected lifetime of the machine is:

$\begin{matrix} \mathbb{E}S & = & \mathbb{E}[S \mid \mbox{machine fails before } t] \cdot \mathbb{P}[\mbox{machine fails before } t] \\ \\ & & \qquad + \quad \mathbb{E}[S \mid \mbox{machine does not fail before } t] \cdot \mathbb{P}[\mbox{machine does not fail before } t] \end{matrix}$

$= \frac{t}{2}\left(0.5t\right) + \frac{2-t}{2}\left( t \right)$

and the expected cost W per machine is:

$\begin{matrix} \mathbb{E}W & = & \mathbb{E}(W \mid \mbox{machine fails before } t) \cdot \mathbb{P}(\mbox{machine fails before } t) \\ \\ & & + \quad \mathbb{E}[W \mid \mbox{machine does not fails before } t).\mathbb{P}(\mbox{machine does not fail before } t) \end{matrix}$

$= \frac{t}{2}( 2600 ) + \frac{2-t}{2} ( 200 ) = 1200t + 200.\,$

So by the strong law of large numbers, his longterm average cost per unit time is:

$\lim_{t \to \infty} \frac{1}{t} Y_t = \frac{\mathbb{E}W}{\mathbb{E}S} = \frac{ 4(1200t + 200) }{ t^2 + 4t - 2t^2 }$

then differentiating with respect to t:

$\frac{\partial}{\partial t} 4\frac{ 4(1200t + 200) }{ t^2 + 4t - 2t^2 } = 4\frac{ (4t - t^2)(1200) - (4 - 2t)(1200t + 200) }{ (t^2 + 4t - 2t^2)^2 },$

this implies that the turning points satisfy:

0 = (4 t - t 2)(1200) - (4 - 2 t)(1200 t + 200) = 4800 t - 1200 t 2 - 4800 t - 800 + 2400 t 2 + 400 t

= - 800 + 400 t + 1200 t 2,

and thus

0 = 3 t 2 + t - 2 = (3 t - 2)(t + 1).

We take the only solution t in [0, 2]: t = 2/3. This is indeed a minimum (and not a maximum) since the cost per unit time tends to infinity as t tends to zero, meaning that the cost is decreasing as t increases, until the point 2/3 where it starts to increase.

Category: Probability theory

Renewal theory

From Wikipedia, the free encyclopedia

Contents

[edit] Renewal processes - an introduction

[edit] Formal definition

[edit] Interpretation

[edit] Renewal-reward processes

[edit] Interpretation

[edit] Properties of renewal processes and renewal-reward processes

[edit] The elementary renewal theorem

[edit] The renewal equation

[edit] Proof of the renewal equation

[edit] Asymptotic properties

[edit] Proof

[edit] The inspection paradox

[edit] Proof of the inspection paradox

[edit] Example applications

[edit] Example 1 - use of the strong law of large numbers

[edit] Solution

[edit] See also

Views

Navigation

interaction

Search

In other languages