Hamburg University of Technology, Telecommunications Dept., Eissendorfer Str. 40, D-21071 Hamburg, Germany, e-mail: J.Wohlers@tu-harburg.d400.de

(1)

USING HIDDEN MARKOV MODELS

Jens Wohlers, Alfred Mertins, and Norbert Fliege

Hamburg University of Technology, Telecommunications Dept., Eissendorfer Str. 40, D-21071 Hamburg, Germany, e-mail: J.Wohlers@tu-harburg.d400.de

ABSTRACT

In this paper a new time-delay estimation algorithm for compound point-processes is presented. Compound point- processes, a generalization of temporal point-processes, de- scribe processes with discrete events, where each occurrence time is associated with certain features. It is shown that, al- though the events are not observable, the time delays from events at one location to the same events at a second location can be estimated using hidden Markov models based on the associated features. We demonstrate the performance of this time-delay estimation algorithm with an application to the estimation of section-related trac data in road traf- c monitoring and control systems.

1. INTRODUCTION

While the time-delay estimation for continuous processes and the estimation of time-of-arrival for determined events or signals have been studied intensively in the past, so- lutions for the time-delay estimation for compound point- processes give only estimations for the average time-delay of point clusters between two locations using correlation methods [1, 2].

We propose a new solution to the estimation problem which is based on the re-identication of single events at the output location with the aid of the feature vectors from the input location. In general, both, input and output feature vectors, are disturbed by noise.

As we will show, a hidden Markov model [3]{[5] can be used for the re-identication. In the model, the feature vectors of the events are combined to a parametric random process similar to speech recognition problems [6], and the sta- tistical properties of the mixture of the unobservable events between input and output location are taken into account.

The ecient determination of the most likely sequence of events is based on the Viterbi algorithm.

The algorithm is implemented to estimate section- related trac data in trac monitoring and control systems on freeways. Section-related trac data including the travel time of vehicles in a road section improves the estimation of trac states and the incident detection in trac control systems. The vehicles on the road can be regarded as points of a point process and the trac ow process can be modeled as a compound point-process. The estimation is based on vehicle signals measured with inductive loops.

It will be shown via simulations based on measured traf-

t time

t

t t t t t t t

u u₂u₃ u₄ u_i u u_i+1 _i+2 u_i+3

i+3 i+2 i+1 i 4 3 2 1 1

t₀

Figure 1: Compound point-process in time c data that the new approach gives a good estimation of individual and average travel times of vehicles.

Compound point-processes and the estimation problem are described in Section 2. In Section 3, hidden Markov models are introduced. A hidden Markov model for the re-identication of events is presented in Section 4. The choice of the model parameters and the derivation of the model probability distributions will be discussed. The application of the algorithm is explained in Section 5. Finally, simulation results will be shown.

2. PROBLEM FORMULATION

A random point process is a mathematical model for a phys- ical phenomenon characterized by highly localized events distributed randomly in a continuum. Each event is repre- sented in the model by an idealized point to be conceived of as identifying the position of the event. The space of the process is usually a semi-innite real line representing time, a subset of Euclidean space representing a spatial region, or a combination of these.

Compound point-processes are obtained from temporal point processes by associating an auxiliary random vari- able, called a mark, with each point occurrence. Each occurrence time^tⁱis associated with a mark^uⁱhaving values in a specied space^U. A marked point-process is called a compound Poisson-process, if the marks are independent from the occurrence-time sequence, and if the occurrence- time sequence is an inhomogeneous Poisson-process [7, 8].

Figure 1 shows a marked point-process and its notation.

We consider a point process with discrete events in time and space as shown in Figure 2. The points are in motion with dierent velocities under the restriction

x2(^t2)^>^x1(^t1) with ^t2^>^t1. Each point only occurs once at the locations^x¹ and^x².

The point processes at the locations^x1 and^x2 can be regarded as temporal Poisson-processes. Each occurrence time of an event^t^i;1and^t^i;2at^x¹and^x²is associated with

(2)

x₁ x₂

t time

4,1 t

t t_{5,1 6,1}t t_7,1 u

u_1,2 _2,2 u_4,2 u_3,2u_6,2 u_5,2 u_7,2

x location

u_i,2

i,1 t_i+1,1 t_i+2,1 t_i+3,1 t1,2 t_2,2 t_4,2 t_3,2 t_6,2 t_5,2 t_7,2 t_i,2

u4,1 u_5,1u_6,1u_7,1 u_i,1 u_i+1,1u_i+2,1 u_i+3,1

Figure 2: Compound point-process in time and space a mark ^ui;1 and ^ui;2, respectively. In our approach the marks are assumed to be feature vectors of the events de- tected at location^x1or^x2in a region of a nite-dimensional Euclidean space. The resulting processes are compound Poisson-processes.

The problem is to estimate the time delay of one pointⁱ with occurrence time^ti;1at location^x1 and^ti;2at location

x

2:

i=^t^i;2 ^t^i;1^: (1) Since the events are not observable, the feature vectors at the input and the output location have to be combined to a parametric random process to assign the feature vector of an event at^x2 to a feature vector of the same event at

x

1. The assignment can be realized using hidden Markov models. After the assignment, the individual time delay of the event can be determined.

3. HIDDENMARKOVMODELS

A hidden Markov model is a doubly stochastic process with an underlying stochastic process that is not directly observable, but can be observed through another set of stochastic processes that produce the sequence of observed symbols.

The symbols can be countably or continuously distributed, they can be scalars or vectors.

The hidden Markov model is characterized by the num- ber^N of states in the model. The individual states are labeled as^f1^;2^;^:^:^:^;^N^gand the state at time^tis labeled as

q

t. Based upon the state-transition probability distribution

A=^fa^ij^g, the new state^q^t+1at^t+ 1 is entered, where

aij =^P(^qt+1=^jjqt =ⁱ)^; 1^i;^j^N: (2) With the initial state distribution = ^fig, ⁱ =

P(^q1 =ⁱ), 1ⁱ^N, the probability of a state sequence

q= (^q¹^q²^:^:^:^q^N) can be written as

P(^q) =^q¹^Y^T

t=2 aq

t 1aq

t

; (3)

where ^T is the length of the observation sequence Ô = (ô¹ô² ^:^:^:ô^T).

After each transition, an observation output symbol is produced according to a probability distribution which de- pends on the current state. The observation symbol probability distribution^B=^fbj(^k)^g, in which

b

j(^k) =^P(^o^t=^v^k^jq^t=^j)^; 1^k^M; (4) denes the symbol distribution in state^j, ^j= 1^;2^;^:^:^:^;^N.

v=^fv¹^;^v²^;^:^:^:^;^v^M^gis the set of^M possible discrete symbol observations. For observations of continuous symbols or continuous vectors, the probability density^bj(^k) is replaced by the continuous density,^b^j(^o), 1^j^N. A special form for^bj(^o) is the Gaussian M-component mixture density

b

j(^o) =^X^M

k =1 c

jk

N[^o;^jk^;^R^jk]^; (5) where ^c^jk is the mixture weight, ^N is the normal density and^jkand^Rjkare the mean vector and covariance matrix associated with state^jand mixture^k.

It can be seen that a complete specication of an HMM requires the specication of two model parameters,^N and

M, the specication of observation symbols^v, and the specication of the three sets of probability densities^A,^Band

. To indicate the complete parameter set of the model, the compact notation

= (^A;^B;) (6) is used [3].

The probability of the observation sequence^Ogiven the state sequence^qis

P(^Ojq) =^Y^T

t=1

P(ôtjqt) =^bq¹(ô1)^bq²(ô2)^:^:^:^bq^T(ôT)^: (7) The joint probability ofÔand^qis

P(Ô;^q) =^P(Ôjq)^P(^q)^: (8) Given the observation sequence Ô and the model , the probability of the observation sequenceÔ is obtained by summing the joint probability over all possible state se- quences^q:

P(^Oj) =^X all^q

P(^Ojq;)^P(^qj)

=^X

q

1

;:::;q

T q

1 bq

1(ô1)âq¹^q²^bq²(ô2)^:^:^:aq^T ¹^q^T^bq^T(ôT)^:(9) The model parameters= (Â;^B;) that maximize the probability^P(Ôj) can be adjusted with a set of training samples, e.g. by an iterative procedure called the Baum- Welch method [9].

Given an observation sequenceÔ= (ô1^:^:^:ôT), one has to nd the optimal state sequence^q= (^q¹^:^:^:^q^T), that is, to maximize the probability^P(^qjO). A formal technique for nding this single best state sequence is the Viterbi algorithm [10].

(3)

P R O B L E M S O L U T I O N

T h e es ti m ati o n of t h e ti m e d el a y i of a n e v e nt i wi t h o c

c u rr e n c e ti m es ti k a t l o c ati o ns x k k is b as e d o n

t h e ass o ci a t e d f e at ur e v e c t ors u i k T h e f e at ur e v e c t ors of a

p oi nt gr o u p of si z e T c a n b e i nt e r p r et e d a s t h e o b s er v ati o n

ve c t ors O o o o T of a hi d d e n M ar k o v m o d el T h e

hi d d e n st at e s e q u e n c e q q q q T of t h e m o d el c or r e

s p o n ds t o t h e u n k n o w n p o si ti o ns of t h e p oi nts at x T h e

p ositi o n at l o c ati o n x of a n e ve nt o c c urri n g a t ti m e ti a t

x r e pr es e nts t h e a c t u al st at e q t of t h e m o d el

Gi v e n t h e p o siti o n s at x e a c h e v e nt c a n b e cl as si e d

a n d t h e ti m e d el a y o f t h e e v e nt c a n b e d e t er mi n e d T h e

a ct u al n u m b er of p oi nts b et we e n l o c ati o n s x a n d x gi v e s

t h e n u m b e r N of s t at e s of t h e hi d d e n M a r k o v m o d el

T h e st at e t r a n siti o n pr o b a bili t y di stri b uti o n f or t h e

tr a nsiti o n fr o m st at e i at ti m e t t o st at e j at ti m e t

a ij P q t j jq t i i j N d e p e n ds o n t h e

mi x u p of t h e p oi nt s b e t w e e n x a n d x T his l e a ds t o a

n o n er g o di c m o d el wit h f o r bi d d e n s t a t e tr a nsiti o ns Si n c e

m ulti pl e a ssi g ns of o n e e v e nt a r e f or bi d d e n t h e t r a n si ti o n

fr o m s t at e i t o st a t e j i is i m p ossi bl e i e a ij f or

i j

Si n c e t h e st atis ti c al d e p e n d e n c e of t h e p oi nt p o si ti o n s

h a v e t o b e i n d e p e n d e nt f r o m t h e d et e c t e d p oi nt at ti m e

t t h e s t at e t r a n si ti o n m at ri x A f a ij g r es ult s i n a b a n d

m a tri x wi t h z e r o s o n t h e m ai n di a g o n al

T h e c o nti n u o u s o bs e r v a ti o n p r o b a bili t y d e n sit y f u n cti o n

i n st a t e j b j o p o jq t j is b as e d o n t h e c o m p a ri s o n

of t h e f e at ur e v e ct or s u i i T wit h t h e f e at ur e

ve c t ors u j j N

Gi v e n t h e hi d d e n M ar k o v m o d el f or t h e r e i d e nti c ati o n

a n d t h e o bs e r v a ti o n s e q u e n c e of t h e f e a t u r e v e ct o rs

u u T fr o m l o c ati o n x o n e h a s t o n d t h e m os t li k el y

p oi nt gr o u p of t h e N p oi nt s a ss o ci a t e d wi t h t h e f e at ur e

ve c t ors u u N fr o m l o c ati o n x t h at i s t o n d t h e

o p ti m al s e q u e n c e q q q T f or a gi v e n o b s er v ati o n

s e q u e n c e O o o T

Af t er r e i d e nti c a ti o n t h e i n di vi d u al ti m e d el a ys of t h e

e v e nt s c a n b e o b t ai n e dA P P L I C A T I O N

D u e t o o pti m al us e of e xis ti n g r o a d n e t w or k s a n o pti mi z a

ti o n of tr a c o w wit h tr a c m o nit ori n g a n d c o nt r ol s y s

t e m s i s e ss e nti al T h e d e sir e d i nf or m a ti o n f or t h e s e s y s

t e m s c a n b e e xt r a ct e d f r o m m o d el b as e d t r a c st at e e sti

m a ti o n pr o c e d ur e s P r es e nt t r a c d a t a a c q ui si ti o n s y st e ms

ar e us u all y b as e d o n l o c al tr a c d at a li k e t h e s p e e d or o w

r a t e of p assi n g v e hi cl es A n i m p r o v e m e nt of t h e t r a c s t at e

esti m ati o n o n f r e e w a y s c a n b e o b t ai n e d wi t h s e cti o n r el at e d

tr a c d a t a i n cl u di n g t h e t r a v el ti m e of ve hi cl es i n a r o a d

s e cti o n

T h e v e hi cl es o n a f r e e w a y c a n b e r e pr e s e nt e d a s p oi nt s

a n d i n c as e of f r e e t r a c o w t h e m oti o n of t h e v e hi cl es

o n t h e r o a d c a n b e m o d el e d as a P ois s o n pr o c e ss

A bl o c k di a gr a m of t h e s e c ti o n r el at e d tr a c d at a e sti

m a ti o n i s s h o w n i n Fi g ur e T h e es ti m ati o n is b a s e d o n

ve hi cl e si g n al s m e as ur e d wi t h i n d u cti v e l o o ps a t t h e e ntr y

a n d t h e e xi t of a f r e e w a y s e cti o n s o t h e r e s ul ti n g pr o c e ss i s

a c o m p o u n d P oi ss o n pr o c es s

Fr e e w a y s e cti o n

Pr e- pr o c essi n g

F e at ur e e xtr a cti o n

Pr e- pr o c essi n g

F e at ur e e xtr a cti o n

Ti m e- d el a y esti m ati o n

ui, 2

M e m or y u j= 1,..., Nj, 1

Fi g ur e S e c ti o n r el at e d t r a c d at a es ti m ati o n

A hi d d e n M ar k o v m o d el f o r t h e s e cti o n r el at e d t r a c

d a t a es ti m a ti o n is c h ar a c t eri z e d b y t h e f oll o wi n g el e m e nts

T h e p o si ti o n of t h e v e hi cl es e n t eri n g t h e r o a d s e cti o n i s

d es c ri b e d b y t h e st at e s e q u e n c e Q f q q q T g

T h e p ositi o n of t h e v e hi cl e l e a vi n g t h e r o a d s e c ti o n i s

d e n o t e d as t h e di s cr e t e o bs er v a ti o n ti m e t

T h e u n k n o w n p ositi o n of t h e ve hi cl e t at t h e e nt r y of

t h e r o a d s e c ti o n is d e n ot e d a s q t

T h e a ct u al n u m b er of v e hi cl es i n t h e r o a d s e cti o n is t h e

n u m b e r of s t at e s N i n t h e m o d el T h e n u m b er of st a t es

d e p e n d s o n t h e o bs e r v a ti o n ti m e t

T h e r a n d o m p r o c es s of t h e f e a t ur e v e c t ors of t h e v e hi cl es

fr o m t h e s e c o n d m e as ur e m e nt p oi nt c a n b e i nt e r p r et e d a s

a n o bs er v ati o n s e q u e n c e O f o o o T g

T h e si z e of t h e v e hi cl e gr o u p l e a vi n g t h e s e cti o n c or r e

s p o n ds t o t h e n u m b er of o bj e ct s T of t h e hi d d e n M ar k o v

m o d el

T h e st at e t r a n siti o n pr o b a bilit y di st ri b uti o n f or t h e

tr a n siti o n f r o m st at e i at ti m e t t o s t at e j a t ti m e t a ij

d e p e n d s o n t h e mi x t u r e of t h e v e hi cl es i n t h e r o a d s e cti o n

Fi g ur e s h o w s t h e st a t e tr a nsiti o n pr o b a bili t y dist ri b u ti o n

a ij esti m at e d wit h a s a m pl e of v e hi cl es i n a k m

r o a d s e c ti o n

- 4 0 - 2 0 0 2 0 4 0

0 0. 0 1 0. 0 2 0. 0 3 0. 0 4 0. 0 5 0. 0 6 0. 0 7

j-i → aij=P(qt+1=j|qt=i) →

Fi g ur e Esti m at e d s t at e t r a n si ti o n pr o b a bilit y dis tri b u

ti o n a ij f or t h e tr a nsiti o ns f r o m st a t e i a t ti m e t t o s t at e j