The model











p11 . . . p1K

... . .. ... ... . .. ... pK1 . . . pKK













and

K

P

k=1

p_jk = 1 for all j∈ {1, . . . , K}.

The long-term behavior of a homogeneous Markov chain is determined by them-step transition probabilitiesp_jk(m) :=P(Yi+m =k|Y_i =j) (in contrast to the one-step transition probabilities given by P). The matrix P(m) which contains the m-step transition probabilities can be calculated as them-th power of P (e.g., Grimmett and Stirzaker, 2001).

πstart describes the probability distribution of the initial state. Using this initial distribution, the distribution of the state at timem may be computed as

(P(Ym= 1),P(Ym = 2), . . . ,P(Ym=K)) =πstartP^m−1.

π_startP^m−1 converges to a fixed vector (which we term π) if the Markov chain is homogeneous and irreducible, i.e., all states are accessible from each other state (e.g., Norris, 1998). This so-calledstationary distribution π can be determined by solving

π=πP subject toπ1^T = 1.

A proof can be found in Seneta (1981).

9.2 The model

In this section, we give a brief introduction to HMMs and their basic properties. In a HMM the hidden state of an underlying Markov chain determines the distribution that generates an observation. This framework provides flexible models for univariate and multivariate time series such as discrete or continuous valued series or categorical series. Hidden Markov Models

9. A Hidden Markov Model

are widely used, e.g, for temporal pattern recognition such as speech, gesture or handwriting recognition (e.g., Rabiner, 1989; Wilson and Bobick, 1999). Other fields of application involve computational linguistics (Och and Ney, 2003), bioinformatics (Yoon, 2009), finance (Ryd´en et al., 1998) or robotics (Fox et al., 2005). For further reading, see, e.g., Elliott et al. (1995);

MacDonald and Zucchini (1997) or Ephraim and Merhav (2002). In the following, we also discuss parameter interpretation.

9.2.1 Definition

We define the HMM formally. The sequence of observations is denoted byd:= (di)i∈{1,2,...,n}

modeled as realizations of random variablesD:= (D_i)i∈{1,2,...,n}. Furthermore,y:= (y_i)i∈{1,2,...,n}

describes the realization of a Markov chainY := (Y_i)i∈{1,2,...,n} on the state space{1, . . . , K} with initial distribution πstart = (πstart,1, πstart,2, . . . , πstart,K). To ease notation, we write dⁱ_i¹

0 :={d_i0, . . . , d_i1}where i₀ < i₁ and similarly forDⁱ_i¹

0, y_iⁱ¹

0, Y_iⁱ¹

0 . Definition 9.1. Hidden Markov Model

LetY be an underlying discrete-time hidden process fulfilling the Markov property (9.1)and Dbe a state-dependent observation process for which the conditional independence property

P(D_i=d_i|D₁ⁱ⁻¹=dⁱ⁻¹₁ , Y₁ⁱ =yⁱ₁) =P(D_i =d_i|Y_i=y_i) (9.2) holds for all1≤i≤n.

The pair of stochastic processes(Y_i, D_i)1≤i≤n is called Hidden Markov Model.

Equation (9.2) means that, ifY_i is known, D_i depends only on this current state and not on any previous states or observations. The probabilities for the stateY_i+1 depend only on the previous stateYi. A visualization of a HMM is given in Figure 9.3.

Y₁ D₁

Y₂ D₂

Y₃ D₃

Y₄ D₄ observations

Markov chain (unobserved)

Figure 9.3: Basic structure of a Hidden Markov Model. The unobserved/hidden Markov chainY determines the density of the visible observation Di at time i∈N.

72

9. A Hidden Markov Model

Here, we intend to model response data from bistable perception experiments. We reduce data analysis to the dominance times (di)i∈1,2,...,n, i.e., the times between reported changes of a percept. In the unimodal continuous case, we assume the d_i to be the realizations of i.i.d. distributed random variables (D_i)i∈1,2,...,n(Figure 9.4 A), where the Gamma or the inverse Gaussian distribution are suitable two-parametric distributions (Wilson, 2007; Gigante et al., 2009; Gershman et al., 2012; Cao et al., 2016). This framework may also be interpreted as one-state HMM. For the intermittent case, the observed dominance times are modeled as the outcomes of a two-state Hidden Markov Model with a Markov chain Y on{S, U} and continuous observations. In the stable stateS long dominance times are emitted, and in the unstable stateU perception changes quickly. To describe the dominance time distribution in S, we use an inverse Gaussian or a Gamma distribution which allows us to fit the mean µS of the dominance duration as well as its standard deviationσ_S. The unstable states are modeled either by another inverse Gaussian distribution with meanµ_U and standard deviationσ_U or – in the case of Gamma-distributed dominance times in S – by an Exponential distribution with parameter 1/µU. Furthermore, we denote the transition probabilities between S and U byp_SU = 1−p_SS and between U and S by p_{U S} = 1−p_{U U}, respectively (see Figure 9.4). We assume that not both ofpSS and pU U equal one.

continuous presentation f_{µ, σ}^IG

A

● ●

intermittent presentation

U

^1−pUU

S

1−p_SS

pSS

pUU

f_µ^IG_{S, σS} f_µ^IG_{U, σU}

B

Figure 9.4: A simple HMM for bistable perception producing inverse Gaussian distributed dominance times. (A) One state describes a unimodal distribution of dom-inance times under continuous presentation. (B) Two states (stable, S, and unstable, U)

produce long and short dominance times under intermittent presentation.

Note that independence of dominance times is assumed here in continuous presentation. This assumption enables straightforward parameter estimation (Section 9.3) and is in agreement with the observation that serial correlations of dominance times are typically not reported (e.g., Walker, 1975; Lehky, 1995). However, weak long-term dependencies of dominance times reported under continuous presentation (Pastukhov and Braun, 2011) cannot be reproduced in the HMM. As such long-term dependence was not observed in the majority of cases in the present data set, also showing no group differences (Section 14), we use here the simple assumption of independence.

9.2.2 Discussion of model parameters

We discuss how the different parameters of the HMM influence the visible response pattern.

The interpretations are trivial for the one-state HMM describing the response patterns to continuous presentation: A larger meanµincreases stability, and a larger standard deviation σ increases irregularity.

9. A Hidden Markov Model

We turn to the two-state HMM for intermittent stimulation. It is obvious that increasing µS increases the length of stable dominance times and increasing σS increases the variance of stable dominance times. Similar arguments hold for increasing µ_U, σ_U. Increasing the probabilities p_SS or p_{U U} implies that a state transition gets less likely. In case of a rather small difference betweenµS and µU or a large variance of at least one of the two distributions the stable and unstable distributions of dominance times are not separated clearly. This is visualized in Figure 9.5.

0 150 300

0.00.1

x

density

A

f_{7, 6}^IG

f_{180, 30}^IG

0 150 300

0.00.1

x

B

f_{60, 30}^IG f_{7, 6}^IG

0 150 300

0.00.1

x

C

f_{180, 180}^IG f_{7, 8}^IG

Figure 9.5: Comparison of densities in the stable (light blue) and the unstable (blue) state. The parameters are indicated in the graph. In panel A the stable and unstable distribution are strictly separated, whereas in B a small µ_S and in C a large σ_S and large σ_U cause a stronger mixing of the distributions.

Visually one may use criteria like regularity (Are the lengths of stable dominance times widespread?) and stability (Which types of dominance times occur? Long and/or short ones?) to distinguish between different response patterns. In the HMM the parametersp_SS and µ_S answer the question how stable the response pattern is. pSS = 1 (together withπstart,S = 1) indicates that only stable dominance times occur. In contrast, a smallp_SS or a small mean of stable dominance timesµ_S are an indicator of a response pattern with rather short dominance times. The regularity is judged by the coefficient of variation CVS = σS/µS of the stable dominance times. A large CV_S implies irregular distributed stable dominance times (which are mainly responsible for the visual impression of regularity), whereas a small CV_S is typical for more regular response patterns.

Figure 9.6 illustrates the impact of different two-state HMM parameters on the response patterns. In panel E a stable response pattern withp_SS = 1 is shown. In contrast, panel B presents a rather unstable HMM with a small µS. The CV of the stable dominance times in panel D is large (CVS = 1) leading to an irregular response pattern. Panel C shows the impact of increasingµ_U, and in panel F an example of a HMM with a large p_{U U} is printed.

74

9. A Hidden Markov Model

Figure 9.6: Impact of the two-state HMM parameter values on the response patterns. Examples of simulated response patterns are shown for different values of the HMM parameters. In panel A we useµ_S = 180,σ_S= 30, µ_U = 7,σ_U = 6, p_SS = 0.67, p_{U U} = 0.95.

In panels B-F one of these parameters is changed as follows where all others remain unchanged to panel A: µS = 60 (B), µU = 10 (C), σS = 180 (D), pSS = 1 (E) and pU U = 0.98 (F).

Changingσ_U does not have clearly visible effects.

Im Dokument Stochastic models for variability changes in neuronal point processes (Seite 79-83)