The presented approach should certainly be applied to network models that are known to accurately describe aspects of real cortical networks. The comparison of the dynamical entropy production in such networks to the actual sensory information content would yield fundamental insight for the understanding of such networks. An example for such a network model is the ring model [124]. A model accurately describing the rat barrel cortex would also be highly interesting because of the parallel conclusion drawn here of a rapid information loss in theta neuron networks and that from experimentally measured information loss in the rat barrel cortex [93]. Finding an appropriate model of the rat barrel cortex, however, will be a challenging initial task. Once this model is established, the application of our approach can be applied to compare the dynamical entropy production rate to the real information content in such networks.
Further analytical steps towards a better understanding of the network dynamics should also be taken into account. This is quite a challenging task for random graphs with an asynchronous ac-tivity. Further analysis of, e.g., the stability of the asynchronous splay state in all-to-all coupled networks could potentially explain the qualitatively different dynamical regimes of neural networks
6.1 Outlook with different AP onset rapidness (see Appendix D). In order to address the network dynamics of random networks in the asynchronous balanced state, we have taken a semianalytic approach. The single neuron model was solved analytically leading to the exact single spike Jacobian. Then, we numerically calculated the Lyapunov spectra of such networks. It was also possible to derive a random matrix approximation of the mean Lyapunov exponent analytically. A similar approach or a generalization of the dynamical mean-field theory studied in Ref. [46] might be used to de-rive an upper bound of the largest Lyapunov exponent. With such a result the dynamics in the thermodynamic limit could be rigorously analyzed, complementing our numerical findings. The understanding of the scaling of the largest Lyapunov exponent with the number of neurons in rapid theta networks or the numerically observed logarithmic increase in correlated leaky integrate and fire networks is not yet exhausted, and would strongly benefit from an analytic derivation.
Other important aspects in terms of nonlinear dynamics are a extension of the analysis of the covariant Lyapunov vectors. Here, we started to investigate the hyperbolicity of the studied systems and introduced the characteristics of temporal network chaos in random theta neuron networks.
This should be further examined together with the temporal and spatial correlation of the covariant Lyapunov vectors and local Lyapunov exponents.
Small theta neuron networks also provide an interesting type of conservative chaos. For example three neuron motifs can exhibit stable, quasiperiodic or chaotic dynamics depending on the topol-ogy and coupling strength. This should be analyzed in more detail, numerically and analytically.
The route to chaos in such three neuron motifs but also in random networks of rapid theta neuron at the edge of chaos (see Fig. 3.24) should be characterized as well.
A demonstration of how large neural networks with a chaotic dynamics would perform under real conditions is another fundamentally important aspect that should be addressed in future re-search. Other work has already made considerable progress in this respect, such as the FORCE algorithm [125] and the promising research of reservoir computing [108,109]. Besides quantifying the dynamical entropy production and analyzing the information flow in such networks, taking into account the flux tube picture in networks of neurons with instantaneous AP initiation may strongly benefit these applications.
The edge of chaos is a recurrent theme in neural computation. A similarly famous aspect is chaos control. Whether realistic computations can be achieved on short time scales on existing stable modes in a stable or chaotic state, or unstable modes are dynamically stabilized in a chaotic state remains an open and fascinating field of research.
A final point is the computational neuroscience aspect. The calculation of the Lyapunov spectra requires formidable computational power because the computation time of the necessary matrix orthogonalizations scales proportional to the cube of the system size. We have developed a highly performant parallel algorithm that allows for the calculation of the Lyapunov spectra of moderately large networks. This algorithm appears to be an ideal candidate to be used on graphics processing units (GPU). Such an implementation would enable a large community of scientists to use the proposed approach to accurately characterize the network dynamics without the need of a high-performance cluster.
Appendix A
Integrate and Fire Neuron Models
The membrane of a neuron can be thought of as capacitor with voltage-dependent membrane cur-rents. The membrane potential, or voltage, is then described by the differential equation
CdV
dt =Im+Is (A.1a)
with the membrane capacitanceC, the voltage-dependent membrane currentIm and the synaptic input currentIs. The family of integrate and fire neuron models differs from standard conductance-based neuron models, in that they do not describe the dynamics of different ion channels in detail but try to capture the essential dynamics of the membrane potential in a one-variable model.
There are four different integrate and fire models:the perfect integrator (PIF) where the mem-brane current is zero, the leaky (also linear) integrate and fire model (LIF) where only a leak current is considered, the quadratic integrate and fire model (QIF) where the membrane current is a quadratic function of the voltage, and the exponential integrate and fire model (EIF) where a leak term and an exponential function constitute the membrane current. The voltage-dependent membrane currents of the integrate and fire models are [88]
ImPIF(V) = 0 (A.1b)
ImLIF(V) = −gL(V−VL) (A.1c)
ImQIF(V) = gL 24T
(V−VT)2−IT (A.1d)
ImEIF(V) = −gL(V−VL) +gL4Texp
V−VT 4T
, (A.1e)
where gL is the leak conductance, andVL the leak potential. The rheobase current, the minimal input current for the neuron start firing is denotedIT, the voltage at the minimum of the membrane current isVT =VL+4T with the spike slope factor4T. (Fig. A.1). The passive membrane time constant isτm=C/gL. These models are complemented with reset and threshold conditions. At a certain threshold, the neuron is said to fire an action potential. Afterwards, the membrane potential is reset toVR, possibly after a refractory period τr. Setting the reset and threshold potential to
±∞in the QIF model leads to the theta neuron. The above equations have physiologically correct dimensions.
Figure A.1–Integrate and fire neuron models. (Figures from Ref. [88]) A) Voltage traces for WB, EIF, QIF, and LIF neurons for the same realization of the noisy input current, B) shows a higher resolution for a short time interval in which a spike has been generated in all models. The subthreshold traces are similar for all models; however, the dynamics of the spike are different on a ms time scale. When the fluctuation leads to a spike in all models, the LIF neuron spikes first. The EIF neuron spikes almost exactly at the spike onset of the WB. The QIF neuron fires much later, C)IV curve of the EIF (solid line) and WB (dotted line) neurons. The thresholdVT is defined as the minimum of the curve. The spike slope factor4Tis proportional to the radius of the curvature of theIV curve at its minimum.
The stable fixed point of the LIF model is atVS=VL. It does not incorporate an action potential generating unstable fixed point. For the neuron to fire, the stable fixed point needs to be pushed by the rheobase current above the threshold.
The QIF model has two fixed points dV
dt =0 = gL 24T
(V∗−VT)2−IT
V∗ = VT± s
IT24T gL . The fixed point at VS =VT −p
IT24T/gL is stable and the one at VU =VT +p
IT24T/gL is unstable. The derivative of the voltage is set to−1/τmas in the LIF model, which sets the rheobase currentIT in the QIF model:
d dV
dVS
dt =− 1
τm = gL
C4T(VS−VT)
= − 1 τm4T
s IT24T
gL IT = gL4T
2 , and sets the stable and unstable fixed pointVS,U=VT∓ 4T.
The above equations are in physiologically correct dimensions. The transformation to a dimen-sionless voltage representation is obtained by
I¯ = I
24TgL (A.2)
V¯ = V−VT 24T
(A.3) This transformation withτm=C/gL leads to
τm
d ¯V
dt = I¯m+I¯s (A.4a)
I¯mLIF = −V¯ −1
2 (A.4b)
I¯mQIF = V¯2−I¯T with ¯IT = 1
4 (A.4c)
I¯mEIF = −V¯ −1 2+1
2exp(2 ¯V). (A.4d)
This corresponds to the stable and unstable fixed point in the QIF model atVS,U =∓1/2. In the LIF model, the stable fixed point is alsoVS=−1/2 and the threshold would be set to+1/2. In the main part of the thesis, this was for simplicity shifted to 0 and 1, respectively.
Another commonly used representation is obtained by a transformation of the currents only:
I˜= I C = I
τmgL. (A.5)
This leads to a representation in which the voltage is still measured in mV but the currents are measured in mV/ms:
dV
dt = I˜m(V) +I˜ I˜mLIF(V) = − 1
τm
(V−VL) I˜mQIF(V) = 1
24Tτm
(V−VT)2−I˜T I˜mEIF(V) = − 1
τm(V−VL) +4T
τm exp
V−VT 4T
.