A Survey of Replicator Equations

NOT

FOR

QUOTATION

WITHOUT

PERMISSION

OF THE

AUTHOR

A

SURVET

OF

REPLICATOR EQUATIONS

Karl Sigmund

July 1964 WP-84-57

Working Papers a r e interim reports on work of the International Institute for Applied Systems Analysis and have received only limited review. Views or opinions expressed herein do not necessarily represent those of the Institute or of its National Member Organizations.

INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS 2361 Laxenburg, Austria

This survey of t h e "state of the art" of replicator dynamics covers recent developments in the theory of the difference and differential equations which describe the evolution of population frequencies under the influence of selection. Mathematical models of this type play a cen- tral role in population genetics, ecology, prebiotic evolution a n d ethol- ogy. They introduce a dynamic element into the theory of normal form games and may also be applied to models of learning and economic evo- lution. The mathematical aspects considered include fixed-point analysis, t h e notions of permanence and exclusion, the gradient systems obtained by t h e introduction of certain Riemann metrics, Hopf bifurca- tions, and relations with game-theoretical concepts.

This research was undertaken as part of t h e Feasibility Study on t h e Dynamics of Macrosystems in t h e System and Decision Sciences Pro- gram.

ANDRBJ WIERZBICKl

Qurirman

System and Decision Sciences

A SURVET OF REPLTCATOR EQUATIONS

KmZ

S i g m u n d

Institute of Mathematics, University of Vienna, Vienna, Austria

What are t h e units of natural selection? This question has aroused consid- erable debate in theoretical biology. Suggestions range from pieces of polynu- cleotides, genes or gene complexes to individuals, groups or species. It could turn out, however, t h a t different answers a r e correct in different contexts, depending on t h e scale on which selection acts most decisively. This is some- what analogous t o physics, where t h e dominant force may be gravitational, electromagnetic, or strong or weak inter-particle attractions. depending on t h e problem.

I t is therefore convenient t o consider an abstract unit of natural selection in theoretical investigations, which can be replaced by t h e appropriate real unit (genes, individuals or species) in specific circumstances. This abstract unit is termed a r e p l i c a t o r in Dawkins' book ?'he E z t e n d e d PBRnotllpe (Dawkins, 1982). The t e r m describes any entity which (a) can give rise to a n unlimited (at least in principle) sequence of copies and (b) occurs in variants whose proper- ties may influence the number of copies.

Biomathematical arguments support the usefulness of t h i s concept.

Indeed, t h e remarkable similarity of dynamical systems describing the action of selection in t h e most diverse fields lends weight t o the notion of a common mechanism underlying these different observations. The t e r m r e p l i c a f u r

-its

has been applied to this mechanism (see Schuster and Sigmund, 1983). In the case of continuous time (generations blending into each other), the dynamics can be described by an ordinary differential equation

x = fix)

of the type

while for discrete t i e (separate generations) the dynamics a r e given by a difference equation

x

+

T x

with

In both cases, t h e t e r m @ is defined by

and ensures t h a t t h e state

x

of the system remains on the unit simplex n

2$, = ⁼

( Z ~ , . . . . I ~ ) clRn :

Czi =

^1, zi 2 O for all ij

.

_{( 1.4)}

i =l

The functions fi(x) describe t h e interaction of t h e different variants of the underlying replicator, and are specified by an appropriate biological model.

In particular. first-order interaction terms, i.e., linear functions f i(x)

= AX)^

defined by a matrix A

=

(%). wkere

lead t o dynamics which have been investigated independently in (i) population genetics, (ii) population ecology, (iii) t h e theory of prebiotic evolution of self- replicating polymers and (iv) sociobiological studies of evolutionarily stable t r a i t s of animal behavior. Within these contexts, the dynamics describe the effects of selection upon (i) allele frequencies in a gene pool, (ii) relative fre- quencies of interacting species, (iii) concentrations of polynucleotides in a dialysis reactor and (iv) distributions of behavioral phenotypes in a given species.

Alter a brief summary of the biological background in Section 2, we present a survey of the mathematical aspects of continuous- and discrete-time replica- tor equations. There are many interesting results, in particular for t h e Brst- order case, due to t h e work of Akin, Hofbauer, Zeeman and others. Section 3 is concerned with some general properties of replicator equations. and in Section 4 we discuss t h e existence and stability of equilibria and present some theorems on time averages and exclusion properties. Results concerning the permanence of the biological components of t h e system are presented in Sec- tion 5. Gradient systems for replicator equations are described in Section 8, and Section 7 gives an overview of t h e classification of low-dimensional phase portraits. Fmally, Section €3 summarizes t h e relationships between game theory and first-order replicator equations.

2.1. Population genetics

Genes a r e the quintessential replicators. It is therefore quite appropriate t h a t t h e first systematic study of a class of replicator equations was in popula- tion genetics: t h e classical work of Fisher, Haldane a n d Wright on the eflects of n a t u r a l selection upon t h e frequencies of alleles a t a single locus of a diploid.

randomly mating population.

Briefly, i f A,

,... ,&

denote t h e possible alleles a n d zi ,..A, t h e i r frequencies within t h e adult population, t h e n random fusion of gametes yields zygotes of genotype

4%

with frequency 2zizj for i # j a n d z: for i

=

j. (This is t h e Hardy-Weinberg law). Let

qj

denote the fifness of genotype

45,

which in t h i s context i s t h e probability of i t s survival from zygote t o adulthood. The geno- types

4%

a n d

$4

a r e identical (it does not m a t t e r which p a r e n t contributes which allele) a n d hence

=

a j i . Since t h e heterozygous genotype

4%

( i # j) carries one gene A, while t h e homozygous genotype A,A, carries two s u c h genes, the frequency ( T x ) ~ of allele

A,

in t h e adult stage of the new generation is proportional t o

a n d h e n c e t o z,

AX)^.

Thus

under t h e obvious assumption t h a t @ (which can be interpreted as the average fitness of t h e population) is n o t equal t o zero.

The corresponding continuous-time selection equation

=

^zi ((A x),

-

i P ) with

qj =

aj, (2.2)

h a s been known since t h e thirties. I t is considerably easier t o handle t h a n i t s discrete counterpart (2.1), b u t its derivation is less clear. I t is usually obtained under t h e assumption t h a t the population is always in Hardy-Weinberg equili- brium, a n assumption which is not strictly valid in general (see Ewens, 1979).

Thus first-order replicator equations with symmetric matrices occur in population genetics.

In t h e model considered here, selection acts through the different viabili- ties of t h e genotypes. Differential fecundities (where the number of offspring depends on t h e mating pair) lead t o equations for t h e genotype frequencies which a r e not of replicator type (see Pollak, 1979). Except in some special cases (e.g., multiplicative fecundity), these equations behave rather differently from (2.1) or (2.2) (see Bomze e t al., 1903). The effects of mutations and (for models with several genetic loci) recombination are also not described by repli- cator equations.

On t h e other hand, frequency-dependent fitness coe5cients fall within the general framework of replicator equations. Models for haploid organisms lead to equations of t h e type

where z, is the frequency of chromosome Gi and

a,

denotes its fitness. Equa- tions of this type are almost trivial if the coe5cients

q

a r e constant. If they are frequency dependent, however, (e-g., if they are linear Functions of 2,) then interesting replicator dynamics occur.

2.2. Prebiotic evolution

Equations of type (2.3) were f i s t studied (initially within the framework of chemical kinetics) in an important series of papers by Eigen (1971) and Eigen and Schuster (1979) on prebiotic evolution. In this context the zi a r e the con- centrations of self-replicating polynucleotides (RNA or DNA) in a well-stirred dialysis reactor with a dilution flow

9

regulated in such a way that t h e total con- centration z1

+ . . . +

^z,, remains constant (without loss of generality we can set this concentration equal to 1). In the absence of mutations this leads to continuous-time replicator equations (generation effects do not play any part even if the initial population of molecules reproduces in some synchronized way

1.

Independent replication of t h e polymers leads to (2.3) with constant repro- duction rates %. This implies (except in the case of kinetic degeneracy) t h a t all but one of t h e molecular species will vanish, with the loss of the correspond- ing encoded information. In their search for ways of preserving the initial amount of molecular information, Eigen and Schuster were led t o study net- works of catalytically interacting polynucleotides. Such interactions (and the corresponding replication rates) a r e usually quite complicated, but neverthe- less some r a t h e r general results have been obtained. In addition, certain spe- cial cases of linear catalytic (or inhibiting) interactions, yielding first-order replicator equations

have been studied as approximations of more realistic chemical kinetics.

The hypercycle (a closed feedback loop in which each molecular species is catalysed by its predecessor) has attracted particular attention (see Schuster e t al., 1979, 1980; Hofbauer e t al., 1980). Both the cooperation of the com- ponents within a hypercycle and the strict competition between individual hypercycles suggest t h a t such networks may have been involved in some phases of early prebiotic evolution. The hypercycle equation is given by

where t h e indices a r e taken on modulo n and the functions

4 ( x )

are strictly positive on

q.

If t h e

4

are constants kt, the above equation reduces to a spe- cial case of the first-order replicator equations:

obtained if matrix A

=

( q j ) in (2.5) is a permutation matrix:

2.3. Animal behavior

Taylor and Jonker (1978) were t h e first t o introduce first-order replicator equations into models of t h e evolution of animal behavior. This approach was based on Maynard Smith's use of game theory in t h e study of animal conflicts within a species, equating "strategies" with behavioral phenotypes a n d "payoffs"

with increments of individual fitness.

These investigations were initially centered on t h e notion of evolutionary stability (see Maynard Smith, 1974), which m a y be i n t e r p r e t e d a s game- theoretic equilibria which a r e proof against t h e invasion of behavioral mutants.

This static approach assumed certain implicit dynamics which were soon made explicit in t h e form of equations, once again of replicator type.

Let El. ....E, denote the behavioral phenotypes within a population. z

...

z, t h e frequencies with which they occur, and

q,

( 1 r i, j .E n ) t h e expected payoff for a n Ei-strategist in a contest against a n &"-strategist. Then, assuming random encounters, we obtain

AX)^

as the average payoff for a n &-strategist within' a population in s t a t e x, and

a s t h e mean payoff. In t h e case of asexual reproduction, the r a t e of increase i,/zi of phenotype Ei is given by the difference

AX)^ -

^x-Ax, which once again yields (2.5) (or, in t h e discrete-time case,

where C is a positive constant).

The assumption of asexual reproduction a t first seems r a t h e r unnatural. I t can be shown, however, t h a t in many important examples t h e essential features of the dynamical model a r e preserved in the more complicated case of sexual reproduction (see Maynard Smith, 1981; Hofbauer e t al., 1982; Hines, 1980;

Bomze e t al., 1983; Eshel, 1982). Rather than introducing some sort of Men- delian machinery which, given t h e present s t a t e of knowledge of the genetic basis of behavior. is bound to be highly speculative, i t s e e m s reasonable to stick t o t h e more robust and manageable asexual model (see Schuster a n d Sigmund, 1984).

The corresponding replicator equations are examples of frequency- dependent sexual o r asexual selection equations. Many specific types of conflicts (e.g., t h e Hawk-Dove-Bully-Retaliator game, t h e War of Attrition game and t h e Rock-Scissors-Paper game) have been examined within this framework (see Zeernan, 1981; Bishop and Cannings,l978; S c h u s t e r e t al., 1981).

The game-dynamical aspects of t h e linear replicator equation (2.5) may be expected to lead t o applications in fields such a s psychology a n d economics (see Zeeman, 1981). A justification of viewing strategies as replicators i s given by Dawkins (1982).

2.4. Population ecology

Equations used t o model ecological systems are usually of t h e form

where t h e yi a r e t h e densities of different populations interacting through com- petition, symbiosis, host-parasi t e or predator-prey relationships. Such equa- tions "live" on

Rt

a n d a r e usually n o t of replicator type. However, taking rela- tive densities yields replicator equations. In particular, Hofbauer (19Bla) has shown t h a t t h e classical (n

-

1)-species Loth-Volterra equation

is equivalent to t h e f i s t - o r d e r replicator equation (2.5) on

S,,\ f x :

zn

=

⁰¹ with

%j

=

bij

-

^b*,

and yn

=

1. The barycentric transformation (2.12), together with a change in velocity, maps t h e orbits of (2.11) into the orbits of (2.5). Which of these equa- tions is more convenient will depend on t h e problem considered. Similar results hold for interactions of order higher than linear.

Sexually reproducing organisms a r e not replicators in the s t r i c t sense of t h e t e r m , but within ecological considerations and disregarding genotypes they may be viewed a s such.

3. GENERALPROPEUrIES

The t e r m cP in (1.3) guarantees t h a t the continuous-time replicator equa- tion (1.1) "lives" on

%,

since

( C z i ) . ⁼

^{0 on}

^%.

Thus the simplex and all its faces (which consist of subsimplices characterized by zi

=

⁰ for all i in some non-trivial subset I of 11

,...,

n j ) a r e invariant. In particular, t h e "corners"

e,

a r e equilibria. The solutions of ( 1 . 1 ) in $, are defined for all t EIR.

For the discrete-time replicator equation ( 1 . 2 ) t o have any meaning, the t e r m 9 m u s t be non-vanishing on

&.

It always h a s the same sign, a n d we shall assume t h a t t h e f i ( x ) a r e also of this sign, say positive. In this case t h e sim- plex a n d all of i t s faces a r e once again invariant.

If a continuous- or discrete-time replicator equation is restricted t o a face of

S ,

t h e resulting equation is again of replicator type.

We shall say t h a t two vector fields f and g on

5;,

^are equwnlenf if t h e r e exists a function c : S,,

+IR

such t h a t f i ( x )

-

g i ( x )

=

^c ( x ) holds on

&

for all i.

If f and g a r e equivalent t h e n t h e restrictions

ii =

z i C f i ( x )

-

9 ) and zi

=

z i ( g i ( x )

-

9 ) coincide on

%.

In t h e s a m e way, if t h e r e exists a function c :

& +IR+

such t h a t f i ( x )

=

c ( x ) g i ( x ) holds on S, for all i , then t h e difference equations x -+ Tx with ( T X ) ~

=

zi f i ( x ) ~ - l and ( T X ) ~

=

z i g i ( x ) @ - l coincide on Sj,.

In particular, we say t h a t n x n matrices A a n d B a r e equivalent if t h e vec- tor fields A x and B x a r e equivalent in the sense described above. This i s the c a s e iff t h e r e exist constants c j such t h a t

9 -

^bij

=

c j for all i a n d j.

Equivalent matrices l i a d t o identical &st-order replicator equations. Thus, without loss of generality, we may consider only matrices with zeros in the diagonal, for example, or matrices whose first row vanishes.

Another useful property is t h e quotient rule

or, in t h e discrete case.

for z,

>

0.

Losert a n d Akin (1983) have shown t h a t t h e discrete-time k i t - o r d e r repli- cator equation induces a diffeomorphism from i n t o itself. This result is important because i t excludes the "chaotic" behavior caused by the non- injectivity of mappings such as z ⁺ a z ( 1 - z ) . However, t h e discrete case i s still f a r less well-understood t h a n t h e continuous one, a n d may behave in quite a different way.

4. EQUILlBRIA

AND

THHR SI'ABILITT

The k e d points of (1.1) or (1.2) in t h e interior of

Sn

a r e t h e strictly posi- tive solutions of

If (4.1) holds, t h e common value is 9. Similarly, t h e equilibria in the interior of a face defined by zi

=

D for some i E [ l

,....

^nj a r e t h e strictly positive solutions of the analogous equations.

In particular, t h e i n n e r equilibria of first-order replicator equations are t h e strictly positive solutions of the linear equations (4.2) a n d

These solution form a linear manifold. Generically, t h e r e is e i t h e r one or no interior equilibrium. In fact t h e r e is an open dense subset of n x n matrices such t h a t the corresponding replicator equations a d m i t a t m o s t one fixed point in the interior of Si, a n d i n t h e interior of each face (Zeeman, 1980).

In many cases it is easy to perform a local analysis around a fixed point p by computing t h e eigenvalues of its Jacobian. One s u c h eigenvalue is 9(p); this corresponds to an eigenvector p which is not in t h e t a n g e n t space. Since we a r e studying t h e restriction of (1.1) to

%,

this eigenvalue (or more precisely.

one of its multiplicities) is irrelevant. Thus, for example, t h e relevant eigen- values of a c o r n e r e, a r e the n

-

^{1 values of a,,}

- ^q,

(j # i).

For t h e hypercycle (2.7) t h e r e is always a unique Axed point p i n i n t

S,,,

which is given by

and t h e eigenvalues of the Jacobian a t p a r e ( u p t o a positive factor) t h e n - t h roots of unity. except for 1 itself (see Schuster e t al.. 1980). I t follows t h a t p is asymptotically stable for n s 3, and unstable for n r 5. In fact, using a s a Ljapunov function, i t can be shown t h a t p i s globally stable for n 4. For n

r

5, numerical computations show t h a t a p e r i o d c a t t r a c t o r exists, although this h a s not been proved rigorously.

Linearization around t h e inner equilibrium of (1.1) allows the use of the Hopf bifurcation technique. Zeeman (1980) has shown t h a t for n

=

3 such bifur- cations a r e degenerate and do n o t lead t o periodic attractors. In fact, the equivalence of (2.5), for n

=

3, with t h e two-dimensional Lotka-Volterra equa- tion (2.1 l ) implies t h a t it admits no isolated periodic orbit. For n r 4, however, t h e r e exist nondegenerate Hopf bifurcations, t h e simplest of which i s given by

which, for p

=

^0, reduces t o t h e hypercycle equation with globally stable inte- rior equilibrium (see Hofbauer e t al., 1980). De Carvalho (1984) refined this by showing t h a t for small p

>

0, t h e periodic orbit is globally attracting in i n t

.!&,

except for t h e s t a b l e manifold of the i n n e r equilibrium.

If t h e r e is n o fixed interior point, t h e n there exists a c c l R n with C c ,

=

0 such t h a t t h e function (which is defined on i n t S ) increases along the orbits of (2.5) (Hofbauer, 1981b). It follows from Ljapunov's theorem t h a t each orbit x ( t ) in t h e interior of

%

has its w limit

D(X)

.=

f p E

$,

:

3

tn ^{-D +m} with x(tn) ^-D ~ r ]

contained in t h e boundary of

%.

This implies t h a t t h e r e are no periodic, or r e c u r r e n t , or even non-wandering points in int

&

if there is no fixed inner point. However, this does not mean t h a t limt+,z,(t)

=

0 for some i. Akin and Hofbauer (1982) give a n example, with n

=

4. where t h e w limit of every interior orbit i s a "cycle" consisting of t h e corners el, e2, %, e4 and t h e edges joining them.

Conversely, if the orbit x ( t ) is periodic in int S, or, more generally, has its o limit in int 5;, then t h e time averages of this orbit

lim -fzi(t)dt, 1 i

=

1, ..., n T++-

T

exist and correspond t o an interior equilibrium of (2.5) (see Schuster e t al., 1980). It frequently happens t h a t an interior equilibrium i s unstable, and hence physically unattainable, but is nevertheless still empirically relevant as a time average.

I t is often very difficult t o derive a full description of t h e a t t r a c t o r s of replicator equations. (Recall t h a t strange attractors have been observed numerically (Arneodo e t al., 1980), and t h a t there is still no proof of the elristence of a unique limit cycle for the hypercycle (2.7) with n 15). More modest results may be obtained in such situations by considering only whether the attractors a r e in t h e interior or on the boundary.

In particular, we shall say t h a t the replicator equation (1.1) is permanent if there is a compact s e t in int

S'

which contains the o limits of all orbits starting in int S, (or, equivalently, if there is a 6

>

0 such t h a t limt,+, inf zi (t ) 2 6 for all i , whenever z i ( 0 )

>

0 for all i ) . Such systems a r e robust in a sense which is obviously of great practical importance in ecology, genetics or chemical kinet- ics. On the one hand. t h e s t a t e remains bounded a t some distance from the boundary even if i t oscillates in some regular or irregular fashion: therefore a population (or component) within this system cannot be wiped o u t by small fluctuations. On t h e other hand, if the system starts on the boundary, i-e., with one or more components missing, then mutations introducing these com- ponents (even if only in tiny quantities) will spread, with the result t h a t the sys- t e m will soon be safely cushioned away from the faces of t h e simplex.

We should make two remarks here. Firstly, permanence is not a structur- ally stable property (in the same way t h a t the asymptotic stability of a fixed point is not necessarily structurally stable). Secondly, a non-permanent sys- tem does not always lead to the exclusion of some components. Zeeman (1980) has shown t h a t there is a specific case of (2.5) which has an a t t r a c t o r on the boundary and one in t h e interior. It can also happen t h a t each interior orbit

remains bounded away from t h e faces, but by a threshold which depends on t h e orbit; for permanence, t h e threshold m u s t be uniform.

The most useful sufficient condition for permanence is t h e e~cistence of a function P defined on S,, with P(x)

>

0 for

x

E int Sn and P(x)

=

0 for

x

^E bd S,, s u c h t h a t

P =

P+, where

+

is a continuous function with t h e property that, for all

x

^E bd S,, there is some T

>

⁰ such t h a t

We shall describe

P

a s an average I j a p m o v f u n c t i o n . Near t h e boundary, P increases "on average", so t h a t t h e orbits move away from t h e boundary (Hofbauer, 1981 b).

I t h a s been shown by Schuster e t al. (1981) and by Hofbauer (1981b) t h a t the general hypercycle equation (2.6) has P(x)

=

z ,z,.

.

.zn a s an average Ljapunov function and is therefore permanent. This is of g r e a t importance in t h e realistic design of catalytic hypercycles, whose dynamics a r e too complex t o be represented by (2.7).

Brouwer's fixed point t h e o r e m implies t h a t a necessary condition for per- m a n e n c e is the existence of a k e d point in int .Sn (Hutson a n d Vickers, 1983).

For p e r m a n e n t first-order replicator equations (2.5). such an equilibrium is necessarily unique. Another very useful condition for t h e permanence of (2.5) is t h a t t h e t r a c e of t h e Jacobian at this fixed point m u s t be strictly negative (Amann and Hofbauer, 1984).

Amann a n d Hofbauer obtained a remarkable characterization of per- m a n e n c e for systems (2.5) with matrices A of the form

where

+

m e a n s t h a t the corresponding element is strictly positive and - m e a n s t h a t i t is negative or zero. The following conditions a r e equivalent for equa- tions of this type:

1 The s y s t e m is permanent.

2. There is a unique inner equilibrium p and +(p) is strictly positive.

3. There is a vector z EIR,, with zi

>

0 for all i, s u c h t h a t all components of ZA a r e strictly positive.

4. The m a t r i x C obtained from A by setting cU

=

(taking indices of modulo n ) , i-e., by moving the first row to t h e bottom, is s u c h t h a t its d e t e r m i n a n t a n d all i t s principal minors a r e strictly positive.

Note t h a t -@(p) is just t h e trace of t h e Jacobian a t p, and t h a t matrices such a s C. wbich have diagonal t e r m s strictly positive a n d all other t e r m s non- positive, play an important role in mathematical economics.

As a special c a s e we find t h a t t h e hypercycle equation (2.7) is always per- m a n e n t . Anotker special case has been obtained by Zeeman (1980): t h e replica- tor equation (2.5) with n

=

3 a n d A of t h e form

is p e r m a n e n t iff d e t A

>

⁰ (in this case the i n n e r equilibrium is a global a t t r a c - tor). In addition, Arnann and Hofbauer (1984) have used t h e general t h e o r e m t o c h a r a c t e r i z e permanence in special types of reaction networks, s u c h a s hyper- cycles of autocatalysts:

o r superpositions of counter-rotating hypercycles:

(%.bi

>

^0). Hofbauer ( l 9 8 l b ) has also proved t h a t inhomogeneous hypercycles

with %

>

0, a r e permanent if they have an interior equilibrium. This was done using a s a n average Ljapunov function. More generally. Hofbauer conjec- t u r e s t h a t (2.5) is permanent iff for some p with pi

>

0, the function

&P'

^{is an}

average Ljapunov function or, equivalently, ifl for such a p t h e inequality p-Ax

>

x-Ax holds for all fixed points

x

in bd

&.

This was proved by Arnann (1984) for t h e case n

=

4.

It can be s h o r n t h a t a necessary condition for t h e permanence of first- order replicator equations with

\

r 0 is t h a t an irreducible graph is obtained on drawing an arrow from j to i wherever Q

>

0, i.e., t h a t any two vertices can be joined by an oriented graph (see Sigmund and S c h u s t e r , 1984). It would be interesting t o h o w if s u c h a graph is necessarily Harniltonian, i.e.. contains a closed oriented path visiting each vertex exactly once. (This has been shown by Arnann (1984) for t h e case n 5 4 and ^qi

=

0.)

An interesting class of examples is provided by models describing the com- petition between several hypercycles. If these hypercycles a r e disjoint then the equation i s of t h e form

zi = z ~ ( ~ ~ z ~ ( ~ )

-Q)

,

where n is a permutation of indices containing several cycles. Such systems are not irreducible and hence not permanent. If t h e cycles a r e all of length less than 4. then one of them "wins out" and t h e o t h e r s vanish (see Schuster e t al., 1980). This is probably also t r u e for larger cycles, but h a s n o t yet been pro- ven.

Once again, t h e situation is much less clear i n t h e case of discrete-time replicator equations. A sufficient condition analogous t o the existence of ar, average Ljapunov function has been given by Hutson a n d Moran (1982).

Hofbauer (1984) h a s shown t h a t the discrete hypercycle

(with ki

>

⁰⁾ is p e r m a n e n t iff C

>

0.

6. GRADIENT S Y S K M S

OF

REPUCAlVR

TYPE

The evolutionary dynamics deAned by the gradients of certain potential functions a r e of great i n t e r e s t because they correspond t o popular notions of adaptive genotypic o r phenotypic landscapes and yield biological models with extremum principles of a type familiar in theoretical physics. The action of selection in such situations drives the s t a t e uphill along the path of steepest ascent.

Gradients depend on metrics. Shahshahani (1979) provided a geometric framework for population dynamics by using a Riemann m e t r i c instead of t h e more usual Euclidean metric on S,. Replicator equations which are gradients with respect t o t h i s metric a r e of considerable interest (see Akin, 1979).

Shahshahani defines t h e inner product of two vectors x and y in t h e tangent space

TpS,

(where p E i n t Sn) in the following way:

This introduces a notion of orthogonality which depends oh p , and a definition of distance which differs from t h e Euclidean distance by attaching more weight t o changes occurring near t h e boundary of

S,,.

If V is a differentiable function de6ned in a neighborhood of p , t h e n the Shahshahani gradient Grad V(p) is defined by

for all y E

Tp&,

where DV(P) i s t h e derivative of V a t p. The more usual Euclidean gradient grad V(p) is defined by

Using t h e f a c t t h a t y E

TpS;,

iff y EIR, satisfies

xyi ⁼

^0. i t can be s h o r n t h a t t h e replicator equation (1.1) is a Shahshahani gradient of V iff f is equivalent t o grad V, in the s e n s e outlined in Section 3.

The case where V is a homogeneous function of degree s is of particular interest, since this implies t h a t 9(x) =.sV(x), from Euler's theorem. The aver- age fitness ih t h e n grows a t the largest possible r a t e a n d the orbits are "orthogo- nal" (in the Shahshahani sense) t o t h e constant level s e t s of QI.

In particular, if we have

then the Shahshahani gradient i s zi(%

-

QI), i.e., (2.4). If, however, we have

where

qj =

a*, t h e n t h e Shahshahani gradient is t h e selection equation (2.2).

The corresponding e x t r e m u m principles, which give conditions for t h e average fitness

9

to increase a t t h e largest possible r a t e , have been s t a t e d by Kiippers (1979) a n d Kimura (1958). respectively. However, they did not specify the appropriate m e t r i c . The fact t h a t @ increases along t h e orbits of (2.2) is Fisher's Fundamental Theorem of Natural Selection.

An immediate consequence of Fisher's theorem is t h a t t h e orbits of (2.2) converge t o t h e s e t of equilibria. In addition, each orbit converges t o some equilibrium. This has been proved by Akin and Hofbauer (1982), who once again used a Ljapunov function of type

h?.

Analogous r e s u l t s also hold for discrete-time selection equations, but are considerable h a r d e r t o establish

-

they have been proven by a n der Heiden (1975) for t h e c a s e n

=

3 and by Losert a n d Akin (1983) in t h e general case. It would be interesting t o know whether this convergence holds whenever f is the Euclidean gradient of a homogeneous function.

Rrst-order replicator equations (2.5) a r e Shahshahani gradients iff

ai,. +

a,,

+

^ah

=

a-. _F

+

^Q

+

^akj _(6.5)

holds for all indices i, j and k (Sigmund, 1984). This is the case iff t h e matrix A is equivalent (in t h e sense described in Section 3) t o a symmetric matrix, o r equivalently, iff t h e r e a r e constants ci such t h a t

% -

aji

- -

^ci - c j h o l d s f o r a l l i and j .

Equations of t h e type

a r e obviously Shahshahani gradients. If t h e functions g, a r e monotonically decreasing, they model competition between replicators which inhibit their own growth but a r e otherwise independent. In this case it can be shown t h a t t h e r e exists a unique global attractor. More precisely, we can assume without loss of generality t h a t g ,(0) I g 2(0) ² . . . 2 g, ( 0)

>

0, in which case t h e r e exists a number K a n d a p ^E 5; s u c h t h a t

where m is t h e largest integer j with gj(0)

>

K. The point p i s t h e limit, as t approaches t m , of all orbits x ( t ) for which zi(0)

>

0, i

=

1,

...,

m . A variant of this model shows t h a t if t h e total concentration xzi is kept a t a constant value c (not necessarily equal t o 1) by replacing !$ by 9 / c , t h e n t h e number of species t h a t can coexist increases with increasing c (see Hofbauer e t al., 1981).

The special cases

gi(zi)

=

^U,

-

bizi a n d gi(zi)

=

1

ci

+ 4zi

have been studied by Epstein (1979).

Except in low-dimensional cases, t h e r e is little hope of obtaining a corn- plete classification of first-order replicator equations (2.5) u p t o topological equivalence. Two such equations a r e said t o be topologically equivalent if t h e r e exists a homeomorphism from onto itself which maps t h e orbits of one equa- tion onto t h e orbits of t h e other equation in s u c h a way t h a t orientation i s preserved. Two n x n m a t r i c e s a r e described as R-equivalent if t h e correspond- ing replicator equations a r e topologically equivalent.

Zeeman (1980) proposed a method for the classification of stable cases. By analogy to t h e definition of s t r u c t u r a l stability, an n x n matrix A i s said t o be stable if its R-equivalence class is a neighborhood of A. Thus small perturba- tions of A do not change t h e topological s t r u c t u r e of t h e corresponding replica- tor equation. Zeeman conjectured t h a t t h e stable matrices form an open dense s e t in the space of n x n matrices and a r e divided into a finite n u m b e r of R- equivalence classes for each n . He proved this for n

=

2 and 3, a n d classified all corresponding stable replicator equations. (For n

=

2 and 3 t h e r e a r e 2 and 19 stable classes, respectively, u p to time reversal.)

A basic requirement for t h e classification of (2.5) for n

=

3 is t h a t t h e r e are no limit cycles. This is a consequence of the corresponding result for two- dimensional Lotka-Volterra equations (see, e.g., Coppel, 1966) a n d of the equivalence between s u c h equations and brst-order replicator equations (Hofbauer, 1980a). Bomze (1983) extended Zeeman's classification t o cover unstable cases, obtaining 102 types of phase portraits up t o time reversal.

Little is known about stable matrices for higher dimensions, apart from the f a c t t h a t stability implies t h a t all fixed points of (2.5) a r e hyperbolic (the real

parts of the eigenvalues of their Jacobians do not vanish). This was proved by de Carvalho (1984).

Recall t h a t , without loss of generality, the diagonal of a matrix may be assumed t o contain only zeros. Let

&

denote t h e class of such matrices with non-zero off-diagonal terms. Two matrices A and B in

&

are said t o be sign equivalent if t h e corresponding off-diagonal terms have t h e same sign, and cornbinatorially eguivaleht if A can be made sign equivalent to B by permuting t h e indices. Zeeman (1980) showed t h a t A and B are combinatorially equivalent iff t h e equations obtained by restricting t h e corresponding replica- t o r equations t o t h e edges of

&

a r e topologically equivalent. Within

&.

R- equivalence classes are r e h e m e n t s of the combinatorial classes. There a r e 10 such combinatorial classes for n

=

2 and 114 for n

=

3 up t o sign reversal (Zee- man, 1900). De Carvalho (1984) has studied 19 combinatorial classes without inner equilibria as a first s t e p towards a classification of R-stable matrices for n

=

4. Another step in this direction was taken by Amann (1984), who charac- terized all 4 x 4 matrices which lead to permanent replicator equations.

Another interesting (although highly degenerate) class of examples is pro- vided by circulant matrices (%j

= q+lj+l

for all a and j , counting indices modulo n ) . A partial analysis of this class is given in Hofbauer e t al. (1980). I t is shown t h a t the center of

&,

i-e., t h e point

m,

where mi

=

1/n. is always an equilibrium; i t is not hard t o compute the eigenvalues of its Jacobian. If m i s a sink, then

m

is a global attractor; if

m

is a source, then all orbits converge to t h e boundary. Non-degenerate Hopf bifurcations occur for n 2 4.

I t has often been remarked t h a t game theory is essentially static. How- ever, the replicator equations (2.5) and (2.9) offer dynamic models for normal form games which a r e symmetric in t h e sense t h a t both players have t h e same strategies and the same payoff matrix A. In fact, the dynamic extension is already implicit in the notion of an evolutionarily stable state (Maynard Smith, 1974, 1982), which is a r e h e m e n t of t h e concept of a Nash equilibrium.

A point p ^E

S,

is said to be evolutionarily stable if i t satisfies t h e following two conditions:

1. Equilibrium condition:

P A P > x - ~ p for all x E

~ i ,

_(8.1)

2. Stability condition:

if p A p

=

x-Ap for x ^{# p} t h e n p - A x > x-Ax . (6.2) A game can have zero, one or several evolutionarily stable points. As shown by Selten (1984). t h e notion is not structurally stable: some m a t r i c e s which yield evolutionarily stable points can be perturbed into m a t r i c e s which do not. In this context we also refer the reader t o Bomze (1984) for a thorough analysis of the relation of evolutionary stability t o t h e multitude of equilibrium concepts used in g a m e theory.

I t can be shown t h a t t h e following f o u r conditions a r e equivalent (see Hofbauer e t al., 1979; Zeeman, 1980):

(i) p is evolutionarily stable

(ii) for all q ^E 5; with q # p, we have

provided t h a t E

>

0 is sufficiently small

(iii) for all x ^# p in some neighborhood of p, we have

(ir) t h e function

r'h?

i s a s t r i c t local Ljapunov function a t p for t h e replicator equation (2.5). i.e., strictly increasing along all orbits in a neighborhood of

P.

Condition (ii) is probably the most intuitively obvious in a biological con- text: if t h e s t a t e of t h e population is p, then a fluctuation introducing a small subpopulation in s t a t e q eventually gets wiped out, since t h e p population fares better than t h e q population against the "mixture" ( 1

-

^c)p

+

^cq.

I t

follows from t h e equivalence of (i) and (iv) t h a t any evolutionarily stable point p i s a n asymptotically stable fixed point of (2.5). However, t h e converse is not true. In particular, (iii) implies t h a t if p E i n t

%

is evolutionarily stable, then i t is a n a t t r a c t o r for all orbits in i n t

S,,

and h e n c e t h e unique evolu- tionarily stable point in

&;

however, Zeeman (1980) has shown t h a t t h e r e exist

3 x 3 games with two asymptotically stable fixed points, one in the interior and t h e other on t h e boundary of

S,.

Akin (1980) h a s shown t h a t (2.5) has no fixed point in i n t

5i,

iff there exist two strategies

x

a n d y in S, such t h a t

x

dominates y in t h e sense t h a t

for all z E i n t

5;,.

This result is supplemented by precise statements concern- ing t h e support of strategies

x

and y and t h e Form of global Ljapunov functions or invariants of motion for (2.5).

The results obtained using the time averages (4.4) described in Section 4 suggest a computational method for finding equilibria (and hence solutions) of normal form garnes. These results, which can easily be extended t o asym- metric games (i.e., games in which the players have different payoff matrices).

should be compared with t h e classical methods (involving differential equa- tions) for finding t h e solutions of garnes (see, e.g.. Luce and Raiffa. 1957, p.

438).

The discrete analogues of such methods involve iterative procedures. It t u r n s out, however, t h a t discrete-time replicator equations of type (2.9) do not seem t o lend themselves very well t o game dynamics; in particular, an evolu- tionarily stable point need n o t be asymptotically stable for (2.9) (see, e.g., Schuster a n d Sigmund. 1984).

The behavior of (2.5) and (2.9) for zero-sum games (a,,

=

-a,,) is analyzed in Akin and Losert (1984). If a n interior equilibrium exists, t h e n the continuous model (2.5) has an invariant of motion. The equilibrium is stable, b u t not asymptotically stable, and all non-equilibrium orbits of (2.5) in int

S,

have o limits in i n t

5i,

b u t do not converge t o an equilibrium. By contrast, if the discrete t i m e model (2.9) has an interior equilibrium then i t is unstable and all non-equilibrium orbits converge t o t h e boundary. If t h e r e is no inner equili- brium, then all orbits converge to the boundary in both discrete a n d continu- ous cases. In t h e discrete case all possible attractors may be described using the notion of chain recurrence.

Akin, E. (1979). The geometry of population genetics. Lecture Notes h 5 o m a t h e m a t i c s 31. Springer-Verlag, Berlin, Heidelberg and New York

Akin, E. (1980). Domination or equilibrium. M d h . &sciences 50: 239-250.

Akin, E. and J. Hofbauer (1982). Recurrence of t h e unfit. Math. Biosciences 61:

5 1-63.

Akin, E. and V. Losert (1984). Evolutionary dynamics of zero-sum games. To appear in J. Math, Biology.

Amann,

E.

(1984). Permanence for catalytic networks. Dissertation, University of Vienna.

Amann,

E.

and J. Hofbauer (1984). Permanence in Lotka-Volterra and replicator equations. Forthcoming.

Ameodo, A, P. Coullet and C. Tressor (1980). Occurrence of strange attractors in three-dimensional Volterra equations. Baysics Letters 79A: 259-263.

Bishop. T. and C. Cannings (1978). A generalized war of attrition.

J.

Zheor. h l - o g y 70: 85-124.

Bomze, 1. (1983). Lotka-Volterra equations and replicator dynamics: a two- dimensional classification. Biol. Cybernetics 48: 201-211.

Bomze. I. (1984). Non-cooperative two-person games in biology: symmetric con- tests. Forthcoming.

Bomze, I., P. Schuster and K. Sigmund (1983). The role of Mendelian genetics in strategic models on animal behavior.

J.

7heor. Biology 101: 19-38.

Coppel, W. (1966). A survey of quadratic systems. J.

Dig.

Eqns. 2: 293-304.

Dawkins, R. (1982). 7he R t e n d e d m e n o t y p e . Freeman. Oxford and San Fran- cisco.

De Carvalho, M. (1984). Dynamical systems and game theory. Ph.D. Thesis, University of Warwick.

Eigen, M. (1971). Self-organization of matter and t h e evolution of biological macromolecules. Die Natunvissenschaften 58: 465-523.

Eigen, M. and P. Schuster (1979). The Hypercycle: A Principle of h t u ~ u l 3 1 1 - Organization. Springer-Verlag. Berlin a n d Heidelberg.

Epstein. M. (1979). Competitive coexistence of self-reproducing macro- molecules.

J.

7heor. Biology 78: 271-298.

Eshel, 1. (1982). Evolutionarily stable strategies and natural selection in Men- delian populations. m e o r . Pop. f f i l , 21: 204--217.

Ewen s, W. J. (1979). Mathemafical Population Genetics. Springer-Verlag, Berlin, Heidelberg and New York.

a n der Heiden,

U.

(1975). On manifolds of equilibria in t h e selection model for multiple alleles.

J.

Math. Biol. 1: 321-330.

Hines,

G.

(1980). An evolutionarily stable strategy model for randomly mating diploid populations.

J.

m e o r . Biology 87: 379-384.

Hofbauer, J. (1981a). On t h e occurrence of limit cycles i n t h e Volterra-Lotka equation. Nonlinear Analysis IMA 5: 1003-1007.

Hofbauer, J. (1981b). A general cooperation theorem for hypercycles. Monatsh.

Mcrfh. 9 1: 233-240.

Hofbauer, J. (1984). A difference equation model for t h e hypercycle. SIAM

J.

&l. Math., forthcoming.

Hofbauer, J., P. Schuster and K. Sigmund (1979). A note on evolutionarily stable strategies and game dynamics. J. m e o r . 3 b l o g y 81: 609-612.

Hofbauer, J., P. Schuster and

K.

Sigmund (1981). Competition and cooperation in catalytic self-replication.

J.

Math. Biol. 11: 155-168.

Hofbauer, J., P. Schuster and

K.

Sigmund (1982). Came dynamics for Mendelian populations. Biol. Cybern. 43: 5 1-57.

Hofbauer, J., P. Schuster,

K.

Sigmund and R. Wolff (1980). Dynamical systems

under constant organization. Part 2: Homogeneous growth function; of degree 2. SIAM J. Appl. Math. 38: 282-304.

Hutson, V. and W. Moran (1982). Persistence of species obeying difference equa- tions.

J.

Mafh. Biol. 15: 203-213.

Hutson, V. and C.T. Vickers (1983). A criterion for permanent coexistence of species, with an application to a two-prey/one-predator system. M d h . Biosci. 63: 253-269.

Kimura,

M.

(1958). On t h e change of population fitness by natural selection.

Heredity 12: 145-167.

Kiippers, B.O. (1979). Some remarks on the dynamics of molecular self- organization. &11. Mafh. Biol. 41: 803-809.

Losert, V. and E. Akin (1983). Dynamics of games and genes: discrete versus continuous time. J. Math. Biol. 17: 241-251.

Luce. R and H. Raiffa (1957). Gumes a n d Decisions. John Wiley, New York.

Maynard Smith, J. (1974). The theory of games and t h e evolution of animal conflicts.

J.

l h m . &logy 47: 209-221.

Maynard Smith, J. (1981). Will a sexual population evolve to an ESS? h e r . M t u r a l i s t 177: 1015-1018.

Maynard Smith, J. (1982). E v o l u t i o n q Game l h e o r y . Cambridge University Press, Cambridge.

Pollak, E. (1979). Some models of genetic selection. E h m e t r i c s 35: 119-137.

Schuster. P. and K. Sigmund (1983). Replicator dynamics. J. %or. Biology 100:

535-538.

Schuster. P. and K. Sigmund (1984). Towards a dynamics of social behavior:

strategic and genetic models for t h e evolution of animal conflicts.

J.

S c . Biol. S r u c t u r e s , forthcoming.

Schuster. P.,

K.

Sigmund, J. Hofbauer and R. Wolff (1981). Self-regulation of behavior in animal societies. Part 1: Symmetric contests. Biol. Cybern. 40:

1-8.

Schuster. P.. K. Sigmund and R Wolff (1979). Dynarnical systems under con- stant organization. Part 3: Cooperative and competitive behavior of hyper- cycles. J,

m.

^Eqns. 32: 357-368.

Schuster. P., K. Sigmund and R. Wolff (1980). Mass action Ldnetics of self- replication in bow reactors. J, Math. Anal. Appl. 78: 88-112.

Shahshahani, S. (1979). A new mathematical framework for the study of linkage and selection. Memoirs AMS 211.

Selten,

R.

(1984). Evolutionary stability in extensive two-person games. M d h . Sbc. a i e n b e s , forthcoming.

Sigmund., K. (1984). The maximum principle for replicator equations. Working Paper WP-84-56, international Institute for Applied Systems Analysis, Laxen- burg, Austria.

Sigmund, K. and P. Schuster (1984). Permanence and uninvadability for deter- ministic population models. In P. Schuster (Ed.), Stochastic PRenomena m d

C k o t i c Behavior in Complez * s t e m s . Springer-Verlag, Berlin, Heidelberg and New York

Taylor, P. and L. Jonker (1978). Evolutionarily stable strategies and game dynamics. Math. Biosciences 40: 145-156.

Zeeman, E.C. (1980). Population dynamics from game theory. In C l o b d Z h o q of D p a m i c a l S y s t e m s , Lecture Notes in Mathematics 819. Springer-Verlag, Berlin, Heidelberg and New York.

Zeeman, E.C. (1981). Dynamics of t h e evolution of animal conbicts. J. 77aeor.

Biology 89: 249-270.