Dynamics in Survival Analysis: Conditional Gaussian Property versus Cameron-Martin Formula

NOT FOR QUOTATION WITHOUT PERMISSION OF

THE

AUTHOR

DYNAMICS

IN S'UWNAL

~ Y S I S :

CONDITIONAL

GAUSIAM

PROPKRTY VERSUS CAMERON-- FDRMULA

AI. Yashin

December 1984 FG-84-107

Working Rxpers are interim reports on work of the International Institute for Applied Systems Analysis and have received only limited review. Views or opinions expressed herein do not necessarily represent those of t h e Institute or of its National Member Organizations.

INTERNATIONAL INSTJTUTl3 FOR APPLIED SYSTEMS ANALYSIS 2361 Laxenburg, Austria

This paper describes the stochastic process model for mortality rates of the popu- lation. The key question is the relationship between conditional and unconditional survival functions. The Cameron and Martin solution to t h e problem is compared to the solution based on the Conditional Gaussian Approach. The advantages of the Gaus- sian approach are discussed. The proof of the main formula for averaging uses the martingale specification of the random hazard rate.

DYNAMICS IN

SUl7VIKAL

ANALYSIS CONDITIONAL GAUSSIAN _PROPERIY VERSUS CAMERON-MARTIN

FOFMJLA

kl.

Yashin

1.

Introduction

The well-known Carneron and Martin formula

[

1,2,3 ] gives a way of calculating the mathematical expectation of the exponent which is the functional of a Wiener process. More precisely, let ( R,H.P) be the basic probability space,

H = ( H , ) , ~

be the nondecreasing right-continuous family of o-algebras, and

Ho

is completed by the events of

P-

probability zero from

H = H, .

Denote by

W,

n-dimensional H- adapted Wiener process and

Q ( u )

a symmetric non-negative definite matrix whose elements

g i j ( u ) . i , j =

1,2,

...,

ⁿ satisfy for some t the con- dition

The following result is known as a Cameron-Martin formula.

Theorem 1.

Let

( I )

be

h u e .

Then

w h e r e ( W u , Q ( u ) W u )

is

the scalar p r o d z ~ t equal t o w ~ , Q ( u ) W ,

,

a n d

r(u)

^is

a s y m m e t r i c nonpositive definite m a t r i z , being a u n i q u e solu- t i o n of the f i a t t i mat* e q u u t w n

r(t

)

= 0

.is

a zero m a t r i z .

The proof of this formula in [ 3

]

uses t h e property of likeli- ^' hood ratio for diffusion type processes. The idea of using this approach comes from Novikov [4

1.

Using this idea Myers in [ 5

]

developed this approach and found the formula for averaging t h e exponent when, instead of a Wiener process, there is a process satisfying a linear sto- chastic differential equation driven by a Wiener process. His result may be formulated a s follows.

Theorem 2.

Let Y ( t

)

be a n

m

-dinzenswnnl d i f j k s i o n process of the f o m

with d e t e r m i n i s t i c

inM

condition Y ( 0 ) . Assume t h a t m a w Q(u )

hm

t h e properties described above. m e n the nezt f o n n u l a is is:

w h e r e r(u)

^is

^the solution of m a t r i z Aicatti equation

with the terminal c o 4 d i t i o n r ( t )

=

^0.

These results have direct implementation t o survival analysis: any exponent on t h e left-hand sides of ( 2 ) and (4) c a n be considered a s a con- ditional survival function in some life cycle problem [ 5 , 6 , 7

].

The sto- chastic process in t h e exponent is i n t e r p r e t e d in t e r m s of spontaneously changing factors t h a t influence mortality o r failure rate.

Such interpretation was used in some biomedical models. The qua- dratic dependence of risk from some risk factors was confirmed by t h e r e s u l t s of numerous physiological and medical studies [ 6

1.

The r e s u l t s a r e also applicable t o the reliability analysis.

The way of proving t h e Cameron-Martin formula and i t s generaliza- tions given in [I, 2 . 3

]

does not use a n interpretation and unfortunately does not provide any physical o r demogaphical sense t o t h e variables

r(u)

t h a t appear on t h e right-hand side of t h e formulas ( 2 ) a n d (4).

Moreover, t h e form of t h e boundary conditions for equation (3) a n d ( 5 ) on t h e right-hand side complicate t h e computing of t h e Cameron-Martin formula when one needs t o calculate i t on-line for many time moments

t .

These difficulties grow when t h e r e a r e some additional on-line obser- vations correlated with t h e influential factors.

Fortunately t h e r e is t h e straightforward method t h a t allows avoidance of these complications. The approach uses t h e innovative transformations random intensities or compensators of a point process.

Usage of this "martingale" techniques allows t o g e t a more g e n e r a l

formula for averaging exponents which m i g h t be a m o r e complex func- tional of a random process from a wider class.

If t h e functional is of a q u a d r a t i c form one can g e t a n o t h e r con- s t r u c t i v e way of averaging t h e exponent using t h e conditional Gaussian property. The goal of t h i s paper i s t o illustrate t h i s approach.

2.

Results Formulation

We shall s t a r t from t h e following general s t a t e m e n t .

Theorem 3. Let Y ( u ) be an a r b i t r a r y H-adapted r a n d o m process a n d h ( Y , u ) is s o m e n o n - n e g a t i v e HY-adaptive f u n c t i o n s u c h that f o r s o m e t

SO

w h e r e

T

i s the s t o p p i n g t i m e a s s o c i a t e d with the process Y ( U ) us 101- Lovrs:

and

w = n u j ~ ( v ) . v

S u

1

^is u - d g e b r a g e n e r a t e d b y the h i s t o r y of

the

u >t

p r o c e s s

Y ( U )

up to t i m e t ,

W = (w)tao ^.

The

proof of this s t a t e m e n t based on t h e idea of "innovation", widely u s e d in martingale approach t o filtration a n d stochastic control prob- l e m s [ 3 , 8 , 9

]

is given in t h e Appendix.

Another Eorrn of this idea appeared and was explored in t h e demo- graphical studies of population heterogeneity dynamics [7.10,11

].

Differences among t h e individuals or units in t h e s e s t u d i e s were described in t e r m s of a random heterogeneity factor called "frailty". This factor is responsible for individuals' susceptibility t o death and c a n change over t i m e in accordance with t h e changes of s o m e e x t e r n a l vari- ables, influencing t h e individuals' c h a n c e s t o die (or t o have failure for some u n i t if one deals with t h e reliability studies).

When t h e influence of t h e external factors on t h e failure r a t e may be r e p r e s e n t e d in t e r m s of a function which is a quadratic form of t h e diffusion type Gaussian process, t h e result of Theorem 3 . m a y be developed a s follows:

Theorem

4 . Let the m - d i m e n s i o n a l H-adapted p r o c e s s Y(u) s d i s f y the l i n e a r s t o c h a s t i c d i f f e r e n t i a l equafion

w h e r e

Yo

.is the Gaussian r a n d o m v a r i a b l e w i t h m e a n

mo

a n d v a r i a n c e yo

.

Denote b y ^Q(U) a symmeCric non-negative d e f i n i t e

mat*

w h o s e e l e m e n t s s a t i s f y c o n d i t i o n (1). m e n the n e z t f o r m d a is h e

The p r o c e s s e s m, a n d y,, are the s 0 1 u t i o m of the following o r d i n a r y d i f f e r e n t i a l e g u a t i a n s :

with the initial conditions m o and yo , r e s p e c t i v e l y .

The proof of t h i s theorem is based on t h e Gaussian property of t h e conditional distribution function

P

(Y(t )

<

z

1 T > t

)

.

This situation recalls t h e well-known generalization of t h e Kalman filter s c h e m e [3,12,13,14

]

(see Appendix).

Note t h a t a similar approach t o t h e averaging of t h e survival func- tion was studied i n [6

]

under t h e assumption t h a t t h e conditional Gaus- sian property t a k e place. The mortality r a t e in this paper was assumed t o be influenced by t h e values of some randomly evolving physiological fac- t o r s such a s blood p r e s s u r e or s e r u m cholesterol level.

We will illustrate t h e results a n d ideas on several examples.

3.1.

Failure Rate

^{as a}

Function

of a

Random Variable

Let (Q,H.P) be t h e basic probability space,

Y ( o )

a n d T(w) be two random variables, s u c h t h a t T(o)

>

⁰ with a probability o n e a n d h a s a continuous distribution function. ~ ( w ) a n d T(w) will be i n t e r p r e t e d a s external environmental factor and t e r m i n a t i o n ( d e a t h ) t i m e , respec- tively.

Assume t h a t t h e external factor influences failure r a t e by m e a n s of random variable

Z =

y 2 . Let o(Z) be a o-algebra in Q g e n e r a t e d by t h e random variable Z

.

Denote by ~ (

.z)

t

= P

( T ^S t

I

^{U(Z) )} t h e u(z)- conditional distribution function of termination t i m e

T .

Assume t h a t

F ( t .Z) h a s t h e form

where A(t) , t r 0 , is deterministic function of t t h a t may be inter- preted as t h e age-specific mortality r a t e for an average (standard) indivi- dual [11

1.

Let F ( t ) denote the unconditional distribution function for

T ( w )

,

and x ( t ) is determined by t h e equality

This function was called "observed mortality rate i n [I1

]

since i t represents mortality approximated by empirical death r a t e s which are evaluated without taking population heterogeneity into account. It can be easily shown [11

]

t h a t

where

is the conditional mathematical expectation of Z given t h e event t T > t j .

The form of t h e x ( t ) a s a function of time is determined by t h e t h e conditional distribution of frailty Z and h ( t )

.

It turns out t h a t if the frailty Z i s generated by Gaussian random variable

Y .

t h e analytical form for a t ) might be easily found. Moreover, this conditional distribu-

tion of Y i s Gaussian, as shown in t h e following theorem.

Proposition 1.

Let

Z

=

y2 , w h e r e

Y

is a Gaussian r a n d o m variable with m e a n

a

and v a r i a n c e

.

Then the conditional d i s t r i b u t i o n of

Y

g i v e n the e v e n t

1 T >

t j isalso Gaussian, with a m e a n ml a n d v a r i a n c e

yt that

safisfy

t h e equations

The r e s u l t of this s t a t e m e n t follows from Theorem 4. I t can also b e proved independently using Bayes' rule. According t o t h i s rule t h e condi- tional density of random variable Y m a y b e r e p r e s e n t e d in t h e form:

where (from t h e definitions of Z a n d T )

and

Substituting t h e formulas for h ( 2 ) a n d P ( T

> t

) into t h e equation for

P ( z

(

T >

t ) leads t o

where

and f ( t ) is some function that does not depend on z and acts as a nor- malizing factor. It is evident t h a t this form of t h e conditional density

g ( z , t ) corresponds t o a Gaussian distribution with a 2a2A(t )

+

¹ and

8

a s mean and variance, respectively. Substituting these 2 u 2 ~ ( t )

+

¹

values for mt and yt , i t is not difficult t o check t h a t they satisfy the equations given in the theorem.

Remark.

Note t h a t results of this theorem may be represented by t h e following averaging formula

which has t h e form similar to the Cameron-Martin result.

3.2. Mortality in

a

Structurized Population

Assume t h a t some population may be represented as a collection of several groups of individuals (men and women, ethnic groups, etc.).

Introduce a random variable Z taking a finite n u m b e r of 'possible values (1.2

,.... K)

with a p i o n probabilities

p I . p 2

,,...

p ~

Let t h e age-specific

.

mortality r a t e of the average individual depend on t h e value of the ran- dom variable Z ;

-

this will be associated with a particular social group.

Assume t h a t the survival probability of a person from group j with a history

f l

of environmental or physiological characteristics up to time

t

may be written a s follows:

where Y ( t ) is the process described in formulation of t h e Theorc-; 4.

If the observer takes into account the differences between t h e peo- ple belonging to different social groups he should produce K different patterns of age-specific mortality rates i ( i , t ) , i

=

- 1 . K .

Proposition

2.

7he m o r t a l i t y r a t e s corresponding t o the conditional survival probabilities

are g i v e n b y the forrnuLas

where

K

d i f f e r e n t estimations m, ( i ) , yt ( i ) are the soLutions of the fol- lowing equations:

If evolution of t h e environmental or physiological factors also depends on random variable Z , t h e r e a r e K different processes influencing t h e mortality rates in each of t h e

K

population's group respectively.

d Y , ( t )

=

a o ( i , t )

+

a l ( i , t ) Y , ( t ) dt

+

b ( i , t ) d

W,,! .

Y i ( 0 )

= &,,

where t h e Y,,o a r e Gaussian random variables with m e a n s m o ( i ) and

variances

yo(i)

, a n d t h e W , a r e independent H-adapted 'Wiener processes. The formula for

X(i,t)

will be t h e s a m e a s before, b u t t h e equations for

( i )

a n d

7 ; ( i )

will contain different p a r a m e t e r s

ao(i,t

), a l ( i , t ), b ( i , t ) :

dm;

( i )

-

dt = a o ( i , t ) + a l ( i , t ) mt

(i)

-

2

mt ( i ) 7; ( i ) A(i,t)

,

rno(i), i =

1,K

If t h e observer does not differentiate between people from different groups t h e observed age-specific mortality r a t e

x ( t )

will depend on t h e

-

proportion

n i ( t ) ,

i

=

1 , K , of individuals in t h e different groups. These proportions coincide with t h e conditional probabilities of t h e e v e n t s [ Z

= i j

,

i = -

1 . K , given f

T > t 1

, a n d can be shown t o satisfy t h e fol-

lowing equations

where 9(0)

= pj .

In t h i s case

i ( t )

m a y be r e p r e s e n t e d a s follows:

3.3.

Evaluation of Mortality Rate in Multistate Demography

Assume t h a t

&

is a finite s t a t e continuous t i m e Markov process

I

with v e c t o r initial probabilities

p , , . . .

% and intensity m a t r i x

with bounded e l e m e n t s for a n y

t r

0 . The process

&

^can be i n t e r - p r e t e d as a formal description of t h e individuals' transitions from o n e

s t a t e to another in the multistate population model. Denote

Hf = 01L,u ^s

t j. The following s t a t e m e n t is the direct corollary of Theorem 3.

Proposition

3. Let the p r o c e s s

&

be a s s o c i a t e d w-dh t h e d e a t h t i m e

T as

follows:

Then the n e z t f o r m u l a .Is t r u e :

w h e r e the n i ( t ) are t h e solutions of the following s y s t e m of the o r d i n a r y d i f f e r e n t i a l e q u a t i o n s :

urith

?(o)

= p j

.

The variables s r j ( t ) , j

=

- 1.K can be interpreted a s the proportions of the individuals in different groups a t time t .

3.4.

Proof

of

Theorem

3

Let

H =

(Ht)tM be a nondecreasing right-continuous family of o- algebras in

R

and let Ho be completed by sets of P-zero m e a s u r e from

H = H , .

Denote by Y(t ),

t

² 0 , t h e continuous time H-adapted process defined on (D,H,P) t h a t describes t h e evolution of t h e s e factors. Denote by

W

t h e family of o-algebras in

R

generated by t h e values of t h e ran- dom process Y(u) :

Assume t h a t w-conditional distribution function of d e a t h time T may be represented by t h e formula

where A ( Y , u ) was introduced before.

Using t h e terminology of t h e martingale theory [3,15 ] a n d t h e r e c e n t compensator representation results [16 ] one can say t h a t t h e process

is an F - p r e d i c t a b l e compensator of t h e life cycle process

where

HZY = (w)trD, ^{= V} w, ^HiZ =

cr1;51u.u : t j. This m e a n s t h a t t h e process

is a n I F - a d a p t e d martingale. If t h e t e r m i n a t i o n time T is viewed a s t h e time of death, t h e process A( Y , u ) , 0

s

u

<

t , m a y be regarded a s t h e age-specific mortality r a t e for a n individual with history

rfi

= { Y ( u ) , O l u ~ t

1.

Let

HI ₌ (x)t,D.

Denote by A ( t ) t h e HI-predictable compensator of t h e life cycle process

Xt .

According t o t h e definition of t h e compensa- t o r and t h e compensator representation r e s u l t s [3,1? ] one can write

The formula for

h ( u )

is t h e r e s u l t of the following statement:

1. Let Y ( t ) and

T

are related ^(LF it isdescribed b y the for- m u l a (17). Then

- h ( t ) = E [ A ( Y . t )

1

T r t

]

Proof. Note t h a t the process

is Hz- adapted martingale t h a t can be represented in the form

where

The process

Nt

seems to be Hz-predictable martingale. To prove t h a t , i t is enough to check the martingale property

t h a t easily follonvs from t h e equality

a n d t h e process

is IF-adapted martingale. Note further t h a t o-algebra has t h e atom

tT >

u j [18

]

and consequently

The non-decreasing process on t h e right-hand side of t h i s equality is Hs-adapted and continuous and, consequently, i t is HZ-predictable. The uniqueness of Hz-predictable compensator implies t h e formula

and consequently

x ( t ) = E [ A ( Y , ~ ) I T > t ]

In particular cases when

h ( Y , u ) = Y * ( u ) ~ ( u ) Y ( u )

where Q ( u ) is t h e m a t r i x with property ( I ) , formula for

h(t

) will be

-

1 = m;Q(t

)mt +

* ^{( Q ( t IT*)*}

where

rn, = F . [ ~ ( t ) l T > t ] 7: = a ( Y ( t ) - m t ) ( Y ( t )

- m t ) *

I T > t I .

and

3.5. Proof of

Theorem

4

I ~ o a u c e t h e conditional c h a r a c t e r i s t i c Function f t(a) defined as follows:

f r (a)

= E

( e'"*Y(')

I T > t ) , t

^{2 0 .}

According t o Bayes' rule, t h i s can be approximated by f t (a)

=

^I&' ( e ' ~ * Y ( t )

~ ( t )

where

and denotes t h e m a t h e m a t i c a l expectation with respect t o marginal probability m e a s u r e corresponding t o t h e trajectories of t h e Wiener pro- c e s s

W,,

O s u

I t

, and

Y . ( d Q ( u ) Y ( u ) = E ( Y e ( u ) Q ( u ) Y ( u ) I ^T >

u ).

Using Ito's differential rule one can r e p r e s e n t t h e product

e i a * Y ( ' ) p ( t )

a s

Taking t h e m a t h e m a t i c a l expectation of both sides of this equality leads t o

Notice t h a t f ,(a) h a s t h e form:

This particular form a n d t h e equation for f ( a ) generate t h e idea t h a t o n e should s e a r c h for a n f ; ( a ) i n the s a m e form:

where mc a n d yt satisfy some ordinary differential equations

(We assume t h a t t h e equations for m, and y, have unique solutions.)

The vector function g ( t ) and matrix ~ ( t ) can be found from the equa- tion for f ,

( a ) .

In order t o do this note that the following equalities hold:

where f i t and f denote t h e vector of t h e first order derivatives and t h e matrix of the second order derivatives respectively, of t h e function

f t (a) with respect to a

.

Applying these formulas t o the equation for f ( a ) we obtain (omit- ting the dependence of f t ( a ) on a for simplicity):

Derivatives f

ht

^{and f}

kt

may be calculated from equation ( k 2 ) :

Substituting these derivatives into the equation for

f

; ( a ) , differentiating with respect to

t

and using equations ( k 3 ) and ( k 4 ) for mt and 7; we obtain:

Taking t h e real and imaginary parts of t h i s equality yields

which lead to t h e equations for mt and yt described in t h e theorem.

Notice t h a t t h e form of t h e f,(a) n o t e d above corresponds t o t h e Gaussian law for conditional distribution of t h e

Y ( t )

given t h e e v e n t

[T > t 1.

It is left to shonr now t h a t equation

( A 4 )

with

G(t

) given by (A.6) has a unique solution. One can easily do this implementing t h e approach developed i n [ 3

]

, chapter 12.

1. R.H. Cameron and W.T. Martin, "The Wiener Measure of Hilbert Neighborhoods in t h e Space of Real Continuous Functions," Journal o

j

M a t h e m a t i c a l PAysics

23,

pp. 195-209 ( 1 9 4 4 ) .

2. R.H. Cameron and W.T. Martin, "Transformation of Wiener Integrals by Nonlinear Transformation," 7Samac t i o n s o

j

A m e r i c a n M a t h e m a t - i c a l S o c i e t y 66, pp.253-203 ( 1 9 4 9 ) .

3. R.S. Liptzer and AN. Shiryaev, S t a t i s t i c s o j R a n d o m P r o c e s s e s , Springer-Verlag, Berlin and New York ( 1 9 7 8 ) .

4. A.A. Novikov, "On Parameters Estimation of Diffusion Processes,"

ShLdia S c i e n c e M a t h e m a t i c s ( 7 ) , pp.201-209 ( 1 9 7 2 ) .

5. L.E. Myers, "Survival Functions Induced By Stochastic Covariate Processes," Journal of Applied Probability 18, pp.523-529 ( 1 9 6 1 ) . 6 . M.A Woodbury and

K.G.

Manton ,

"A

Random Walk Model of Human

Mortality and Aging ," ?heoretical P o p u l a t i o n Biology 1 1 ( 1 N ) , pp.37- 48 ( 1 9 7 7 ).

7 . A.I. Yashin, Chances of S u m i v a l in

a

Chaotic E n v i r o n m e n t . WP-83- 100, International Institute for Applied Systems Analysis, Laxenburg, Austria (1983).

8. k l . Yashin, "Filtering of Jumping Processes," A u t o m a t i c a n d R e m o t e Control 5, pp.52-58 ( 1 9 7 0 ) .

9.

P.

Bremaud, P o i n t P r o c e s s e s a n d Q u e u e s , Springer-Verlag, New York, Heidelberg, and Berlin ( 1 9 8 0 ) .

J.W. Vaupel, K. Manton, and E. Stallard, "The lmpact of Heterogeneity in Individual Frailty on t h e Dynamics of Mortality," D e m o g r a p h y 16, pp.439-454 (1979a).

J.W. Vaupel and k l . Yashin, lRe Deviant D y n a m i c s of Death in H e t e r o g e n e o u s P o p u l a t i o n s , RR-83-1, lnternational lnstitute for Applied Systems Analysis, Laxenburg, Austria (1983).

R.S. Liptzer, "Gaussian Martingales and Generalization of t h e Kalman-Bucy FilLer," Theory of Probability a n d A p p l i c a t i o n s ( in R u s s i a n ) 20(2), pp.34-45 (1975).

A.1. Yashin, "Conditional Gaussian Estimation of Characteristics of t h e Dynamic Stochastic Systems," A u t o m a t i c a n d R e m o t e Con- f r o l ( 5 ) , pp.57-67 (1980).

A.I. Yashin, "A New Proof and New Results in Conditional Gaussian Estimation Procedures ," R o c e e d i n g s of t h e 6th E u r o p e a n Meeting o n C y b e r n e t i c s a n d a s t e r n &search, April 2982, pp.205-207, North- Holland Publishing Company (1982).

J. Jacod, C a l c u l e s Stochastzque e t Problerne d e M a r t i n g a l e s . Lec- t u r e N o t e s in M a t h e m a t i c s , Vol. 714, Springer-Verlag, Heidelberg (1979).

A.1. Y ashin, H a z a r d h t e s and R o b a b i l i t y

Distributions:

R e p e s e n t a -

tion

of R a n d o m h t e n s i t i e s . W P - 8 4 2 1 , lnternational lnstitute for Applied Systems Analysis, Laxenburg, Austria (1984).

J. Jacod, "Multivariate Point Processes: Predictable Projection, Radon-Nicodim Derivatives, Representation of Martingales,"

Z e i l s c h r i f t f u r Wohrscheinlichkeitstheorie u n d Verw. Gebiete 31,

pp.235-253 (1975).

18. C. Dellacherie,

Capacities e t P r o c e s s u s S o c h a s t i q u e s (Capacities

and

S t o c h a s t i c Processes),

Springer-Verlag, Berlin and New York (1972).