On Information and Complexity

NOT FOR QUOTATIOX h7THOUT PERMISSION OF THE AUTHOR

ON INFORMATION AND COYrPLeMTY

Robert Rosen

April 1985 CP-85-19

ColLabotattue Papers r e p o r t work which has not been performed solely at t h e International Institute for Applied Systems Analysis and which has received only limited review. V i e w s o r opinions expressed herein do not necessarily r e p r e s e n t those of the Insti- tute, its National Member Organizations, o r o t h e r organizations supporting t h e work.

IWTERKATIONAL INSTITLITE FOR APPLIED SYSTEMS ANALYSIS 2361 Laxenburg, Austria

This paper represents the written version of a Lecture given a t IIASA in Sep- tember 1984 under the auspices of the Science & Technology and the Regional Issues projects. In its current form i t will appear as a chapter in the forthcoming IIASA book, CompLeAty, Language a n d Life: Mathematical Approaches, J . Casti and A. Karlqvist, eds.

Boris S e g e r s ' d Leader Science & Technology Prograrr.

Arguing by analogy with Aristotle's four distinct categories of causation (material, formal, efficient and final), this paper argues that there a r e correspondingly distinct categories of information, and that the same mathematical language cannot be used to describe each of them. This fact leads to the conclu- sion that our mathematical language is somehow deficient, and that it must be sup- plemented by new structures. These considerations lead to a formalization of the ideas of a complex system and anticipatory control.

Robert Rosen is a Professor i n the Department of PhysioLogy and Biophysics at Dalhousie U n i v e r s i t y , HaLiJax, Nova Scotia, k n a d a .

-

^vii

-

On Information and Complexity

R o b e d R o s e n

Introduction

We introduce t h e r a t h e r wide-rangizg considerations which follow with a dis- cussion of t h e concept of information and its role in scientific discourse. Ever since Shannon began t o talk of information t h e o r y (by which h e meant a proba- bilistic analysis of t h e deleterious e f f e c t s of propagating signals through chan- nels; cf. Shannon and Weaver, 1949), t h e concept has been relentlessly analyzed and reanalyzed. The time and e f f o r t expended on t h e s e analyses must surely rank zs one of t h e most unprofitable investments in mode.rn scientific history; not only has t h e r e been no profit, but also t h e c u r r e n c y itself has been debased t o worth- lessness. Yet, in biology, f o r example, t h e terminology of information intrudes itself insistently a t e v e r y level; code, signal, computation, recognition. It may be t h a t t h e s e informational terms a r e simply not scientific a t dl; t h a t t h e y are a tem- porary anthropomorphic expedient; a &on de parler which merely r e f l e c t s t h e immaturity of biology as a science, t o be replaced a t t h e earliest opportunity by t h e more rigorous terminology of force, energy, and potential which m e t h e pro- vince of more mature sciences (i.e. physics), in which information is never men- tioned. O r , i t may b e t h a t t h e informational terminology wkiich seems t o force itself upon us bespeaks something fundamental; something t h a t is missing from physics as we now understand it. We t a k e this latter viewpoint, and see where it leads us.

In human terms, information is easy t o define; it is anything t h a t is o r can be t h e answer t o a question. Therefore, we preface our more formal considerations with a brief ciiscussiori of t h e s t a t u s of interrogatives, in logic and in science.

The amazing f a c t is t h a t interrogation is not e v e r z part of formal logic, including mathematics. The symbol "?" is not a logical symbol, as, for instance, =e

"v", " A " ,

"3",

^{o r "V1; nor is it} a mathematical symbol. I t belongs entirely t o I ~ f o r m a l discourse and, as f a r as I k ~ o w , t h e purely logical o r formal c h z r a c t e r of interrogation has not been investigated. Thus, if information is indeed connected in a n intimate fashion with interrogation, it is not surprising t h a t i t hzs not been

formally characterized in any real sense. There is simply no existing basis on which t o do so.

I do not intend. t o go deeply h e r e L ~ t o t h e problem of extending f ormzl logrc (always including mathematics in this domain) so as t o include interrogatives. What I want t o suggest h e r e is a relation between our inf o d notions of interrogation and t h e familiar logical operation "-"; t h e conciitional, o r t h e implication. opera- tion. Colloquially, this operatior, can b e r e n d e r e d in t h e form "If A , then B". My argument involves two s t e p s . First, t h a t e v e r y interrogative car? b e put into a kind of conditional f orm:

I f A , then B ?

(where B can be an indefinite pronoun like who, what, etc., as w e l l as a definite proposition); and second, and most important, t h a t every ~ ~ t e r r o g a t i v e can be expressed in a more special conditional form, which can b e described as foUows.

Suppose I know t h a t some proposition of t h e form

I f A , t h e n B

is t r u e . Suppose I now change o r vary A ; t h a t is, replace A by a new expression.

dA

.

The result is ul interrogative, which I can e x p r e s s as If dA , then

6B

?

Roughly, I am treating t h e t r u e proposition "If A , t h e n B ", a s a r e f e r e n c e , 2nd I zm asking what happens t o this proposition if I replace t h e r e f e r e n c e expression A by t h e new expression 6 A . I could, of course, do t h e same thing with B In t h e r e f e r e n c e proposition; replace it by a new proposition d B and ask what happens t o A . I assert t h a t e v e r y interrogative can b e e x p r e s s e d this way, in whzt I call a varicrtional f i r m .

The importance of t h e s e notions for us lies in t h e i r relation t o t h e e x t e r n z l world; most particularly in t h e i r relation to t h e concept of measurement, and. to t h e notions of causality to which t h e y become connected when a formal o r logical system is employed t o r e p r e s e n t whzt is happening in t h e e x t e r n a l world; t h a t is, t o describe some physical o r biological system o r situation.

Before discussing this, I want t o motivate t h e two assertions made above, regarding t h e expression of a r b i t r a r y interrogatives in a kind of conditional form.

I do this b y considering a f e w typical examples, and leave t h e rest t o t h e r e a d e r for t h e moment

.

Suppose I consider t h e question

"Did i t rain yesterday?"

First, I write i t as

"If (yesterday), t h e n (rain)?"

which is t h e first kind of conditional form described above. To find t h e variational form, I presume I know t h a t some proposition like

"If (today), t h e n (sunny)"

On I n f i r m a t i o n a n d Complezity

is true. The general variational form of this proposition is

"If d(todzy), then b(sunny)?"

Then, i f I put

b(today)

=

(yesterday), 5(sunny)

=

^(rain)

I have, indeed, expressed my original question in t h e variation& form. A Little experimentation with interrogatives of various kinds taken from informal discourse (of great interest a r e questions of classification, including existence and universality) should serve to make manifest t h e generality of the relation between interrogation and t h e impLicative forms described above; of course, this cannot be proved in any logical sense since, as noted above, interrogation remains outside logic.

It is clear t h a t t h e notions of observation and experiment a r e closely related to the concept of interrogation. That is why the results of observation and experi- ment (i-e. data) a r e so generally regarded as being information. In a formal sense, simple observation can be regarded as a special case of experimentation; intui- tively, an observer simply determines what is, while an experimenter systemati- cally perturbs what is, and then observes the effects of his o r h e r petturbation.

In t h e conditional form, an observer is asking a question which can generally be expressed as

"If (initial conditions), then (meter readings)?"

In the variational f o m , this question may be formulated as follows: assuming t h e proposition

"If (initial conditions

=

0), then (meter readings

=

0) "

is true (this establishes t h e reference, and. corresponds to calibrating t h e meters), we ask

"If &initial conditions

=

O), then &meter readings

=

O)? "

where, simply

&initial conditions

=

0)

=

(initial conditions) and

&meter readings

=

0)

=

(meter readings).

The experimentalist, essentially, takes t h e results of observation as t h e refer- ence and asks, in variational f o m , simply

"If b(initial conditions), then 6(meter readings)?"

The theoretical scientist, on the other hand, deals with a different class of question; namely, those t h a t arise from assuming a 6B (which may be B itself) and asking for the corresponding 6A. These a r e questions t h a t an experimentalist can- not approach directly, not even in principle. I t is t h e difference between t h e two kinds of questions which distinguishes between experiment and theory, as well as the difference between t h e explanatory and predictive roles of theory itself;

clearly, if we give bA and ask for t h e consequent 6B, w e a r e predicting, wherees if we assume 5B and ask for t h e antecedent 6A , w e a r e explaining.

4 R. Rosen

It should. b e noted t h a t exactly t h e same duality arises in mathematics and logic themselves; t h a t is, in purely formal systems. Thus, a mathematician can ask (informally): If (I make c e r t a i n assumptions), t h e n (what follows)? O r , t h e mathematician can s t a r t with a conjecture, and ask: If (Termat's Last Theorem is true), t h e n (what initial conditions must I assume to construct explicitly a proof)?

The former is analogous t o prediction, t h e l a t t e r t o explanation.

When formal systems (i.e. logic and mathemztics) are used t o construct images of what occurs in t h e world, t h e n interrogations and implications become associ- a t e d with ideas of causality. Indeed, t h e whole concept of natural law depends precisely on t h e idea t h a t causal processes in natural systems can be made to correspond with implication in some appropriate, descriptive inferential system

(e.g. Rosen, 1984, where this theme is developed a t g r e a t length).

But t h e concept of causality is itself a complicated one; a f a c t largely over- looked in modern scientific discourse, t o i t s cost. That causality is complicated has already been pointed out by Aristotle, f o r whom aLI science was animated b y a specific interrogative: Why? He said explicitly t h a t t h e business of science was t o concern itself with "the why of things". In our language, t h e s e me just t h e ques- tions of theor-eticzl science: If

(B),

t h e n (what A ) ? and h e n c e w e can say B because A. O r , in t h e variational form,

6B

because 6 A .

However, Aristotle argued t h a t t h e r e were four distinct categories of causa- tion; four ways of answering t h e question why. These categories, which h e called mcztertul muse, fornral cause, efSLnent crruse, and final cause, are not inter- changeable. If this is s o (and I argue below t h a t , indeed, it is), t h e n t h e r e are correspondingly d t m r e n t kinds of inf o m t i o n , associated with different causal categories. These different kinds of information have been confused, mainly because we are in t h e habit of using t h e same mathematical language t o describe each of them; i t is from t h e s e i n h e r e n t confusions t h a t much of t h e ambiguity and murkiness of t h e concept of information ultimately arises. Indeed, we can say more than this: t h e very f a c t t h a t t h e same mathematical language does not (in f a c t , cannot) distinguish between essentially distinct categories of causation means t h a t t h e mathematical language w e have been using is, in itself, somehow fundamen- tally deficient, and t h a t it must b e extended by means of supplementary struc- t u r e s t o eliminate those deficiencies.

The _Paradigm

of

Mechanics

The appearance of Newton's m n c i p i a toward t h e end of t h e seventeenth c e n t u r y was surely an epochal event. Though nominally t h e theory of physical sys- tems of mass points, i t was much more. In practical t e m , by showing how t h e mysteries of t h e heavens could b e understood on t h e basis of a f e w simple, univer- sal Laws, i t set t h e s t a n d a r d s for explanation a n d . p r e d i c t i o n which have been a c c e p t e d e v e r since. I t unleashed a feeling of optimism almost unimaginable today;

i t was t h e culmination of t h e e n t i r e Renaissance. More than that: in addition t o provi&ing a universal explanation for specific physical events, i t also provided a language and a way of thinking about systems which has persisted, essentially unchanged, to t h e p r e s e n t time; what has changed has only been t h e technical manifestation of t h e language and its interpretation. In this lznguage, t h e word information does not a p p e a r in any formal, technical sense; we have only words like energy, force, potential, work, and t h e like.

On I n f o r m a t i o n a n d CompLerity 5

It is important t o recognize t h e twin roles played by Newtonian mechanics in science: as a reductionistic ultimate and as a paradigm for representing systems not yet reduced t o arrangements of interacting particles. The essential feature of this paradigm is the employment of a. mathematical language with an inherent dual- ity, which w e may express as t h e distinction between i n t e r n a l s t a t e s and dynamicaL Laws. In Newtonian mechanics, the internal s t a t e s a r e represented by points in some appropriate manifold of phases, and t h e dynamical laws r e p r e s e n t t h e internal o r impressed forces. The resulting mathematical image is thus what is called nowadays a d y l a m i c a l s y s t e m . However, t h e dynamical systems arising in mechanics are mathematically r a t h e r special ones, because of t h e way phases are defined (they possess a symplectic structure). Through t h e work of people like PoincarC, Birkhoff , Lotka, and many others over t h e years, however, this dynami- cal system paradigm, o r its numerous variants, has come t o be regarded as t h e universal vehicle for t h e representation of systems which could not, technically, be described in terms of mechanics; systems of interacting chemicals, organisms, ecosystems, and many others. Even t h e most radical changes occurring within phy- sics itself, Like relativity and quantum theory, manifest this framework; in quantum theory, for instance, there was t h e most fundamental modification of what consti- tutes a s t a t e , and how it is connected to what we can observe and measure; but otnerwise, t h e basic partition between s t a t e s and dynamical laws is relentlessly maintained. Roughly, this partition embodies a distinction between what is inside or intrinsic (the states) and what is outside ( t h e dynamical laws, which are formal generalizations of t h e mechanical concept of impressed force).

This, then, is our inherited mechanical p a r a d i g m , which in its many techni- cal variants or interpretations has been regarded as a universal language for describing systems and their effects. The variants take many forms; automata theory, control theory, and t h e Like, but t h e y all conform to tne same basic frame- work first exhibited in t h e Prineipia.

Among o t h e r things, this framework is regarded as epitomizing t h e concept of causality. W e examine this closely h e r e , because it is important when we consider t h e concept of information within this framework.

Mathematically, a dynamical system can be regarded simply as a vector field on a manifold of s t a t e s ; t o each s t a t e , t h e r e is an assigned velocity vector (in mechanics it is, in fact, an acceleration vector). A given s t a t e (representing what t h e system is intrinsically like a t an i n s t m i ) together with its associated tangent vector (which r e p r e s e n t s what t h e effect of t h e external world on t h e system is like a t an instant) uniquely determine how t h e system will change s t a t e , o r move in time. This t m s l n t i o n of environmental effects into a unique t m g e n t vector is already a causal statement, in some sense; it translates into a more perspicuous form through a process of i n t e g r a t i o n , which amounts t o solving t h e equations of motion. More precisely, if a dynamical system is expressed in the familiar form

in which tine does not generally appear as an explicit variable (but only @plicitly through its differential o r derivation, d t ) , t h e process of integration manifests t h e explicit dependence of t h e s t a t e variables zi

=

^zt (t ) on tine,

6 R. Rosen

This is z more traditional kind of causal statement, in which t h e s t a t e a t time t is t r e a t e d as an effect, and t h e right-hand side of equation (7.2) contzins t h e c a u s e s on wnich this e f f e c t depends.

Before going f u r t h e r , l e t us t a k e a look a t t h e integrands in equation (7.2), which are t h e velocities o r r a t e s of change of t h e s t a t e variables. The mathemati- cal c h a r a c t e r of t h e e n t i r e system is determine2 solely b y t h e f o r m of t h e s e func- tions. Hence, w e can ask: What is it t h a t expresses this form (i.e. what determines whether our functions are polynomial?;, o r exponentials, o r of some o t h e r form)?

And given t h e general form (polynomial, say), what is it t h a t picks out a specific function and distinguishes it from aIl o t h e r s of t h a t form?

The answer, in a nutshell, is p a r a m e t e r s . As I have written t h e s y s t e m (7.1) above, no such parameters are explicitly visible, b u t t h e y are a t least tacit in t h e very writing of t h e symbol f f

.

Mathematically, t h e s e parameters serve as coordi- nates for function spaces; just as any o t h e r coordinate, t h e y label o r identify t h e individual members of such spaces. They thus play a very different role t o t h e s t a t e M ~ l e s , which constitute t h e arguments o r domains of t h e functions t h a t t h e y identify.

Here we find t h e first blurring. For t h e parameters which specify t h e form of t h e functions f f can, mathematically, b e thrown in as arguments of t h e func- tions f themselves; thus, w e could (and in f a c t always do) write

where at are p a r a m e t e r s . W e could even e x t e n d t h e dynamical equations (7.1) b y ^. writing d a f / d t

=

0 (if t h e at are indeed independent of time); thus, mathemati- cally we can entirely e r a d i c a t e any distinction between t h e p a n m e t e r s and t h e s t a t e variables.

There is still one f u r t h e r distinction t o b e made. We pointe2 out above t h a t t h e parameters a* r e p r e s e n t t h e effects of the outside world on t h e intrinsic sys- tem s t a t e s . These effects involve both t h e system and t h e outside world. Thus, some of t h e parameters must b e i n t e r p r e t e d as intrinsic too ( t h e so-called c o n s t i - t u t i v e parameters), w h i l e o t h e r s describe t h e state of t h e outside world. These latter obey t h e i r own laws, not incorporated in equation (7.1), s o t h e y a r e , from t h e standpoint of equation (7.1), simply regarded as & n c t i o n s of time and must b e posited independently. They constitute what a r e variously called inputs, con- t r o l s , o r f o r c i n g s . Indeed. if we regard t h e states [zi (t)], o r any mathematical functions of them, as corresponding o u t p u t s (that is, output as a function of input r a t h e r than just of time) w e pass directly to t h e world of control theory.

So l e t us review our position. Dividing t h e world into s t a t e variables plus dynamical laws amounts t o dividing t h e world into s t a t e variables plus parameters.

where t h e role of t h e parameters is t o determine t h e f o r m of t h e functions, which in t u r n define t h e dynamical laws. The s t a t e variables are t h e arguments of t h e s e functions, while t h e parameters are coordinates in function spaces. F u r t h e r , we must partition t h e parameters themselves into two classes; those which a r e i n t r i n s i c ( t h e constitutive parameters) and those which are e z t r i n s i c ; t h a t is, which r e f l e c t t h e nature of t h e environment. The intrinsic parameters are intui- tively closely connected with t h e system i d e n t i t y ; t h a t is, with t h e specific nature o r c h a r a c t e r of t h e system itself. The values t h e y assume might, for exam- ple, tell us whether w e are dealing with oxygen, c=Son dioxide, o r any o t h e r chemical species, and, t h e r e f o r e , cannot change without our perceiving t h a t a

On I n f i n n a t i o n a n d ConrpLeztty 7

change of s p e c i e s has occurred. The environmental p a n m e t e r s , as well as the state variables, however, can change without affecting the species of the system.

These distinctions cannot be accommodated. with the simple lznguage of vec- tor fields on manifolds; that language is too abstract. We can only recapture these distinctions by (a) superimposing an informal layer of i n t e r p r e t a t i o n on the for- mal language, as we have done above, or (b) changing the language itself, to render it less abstract. Let us examine how this can be done.

In order to have names for t h e various concepts involved, I call the constitu- tive parameters, which specify the f i m s of t h e dynamical laws, and hence the species of system with which we a r e dealing, the system genome; the remaining parameters, which reflect t h e nature of t h e external world, I call the system e n v i r o n m e n t , and t h e s t a t e variables themselves I call p h e n o t y p e s . This rather provocative terminology is chosen to deliberately reflect corresponding biological situations; in particular, I have argued (cf. Rosen, 1978) that, viewed in this light.

t h e genotype-phenotype dualism which is regarded as so characteristically bio- logical has actually a far more universal currency.

The mathematical structure appropriate to reflect t h e distinctions w e have made is that of genome-parameterized mappings from a space of environments to a space of phenotypes; t h a t is, mappings of the form

specified in such a way that given any initial phenotype, environment-plus-genome determines a corresponding trajectory. Thus, w e have no longer a simple manifold of states, but r a t h e r a fiber-space structure in which t h e basic distinctions between genome, environment, and phenotype a r e embodied from the beginning.

Some of the consequences of this scenario are examined in Rosen (1978, 1983); we cannot pause to explore them here.

Now we a r e in a position to discuss the actual relation between t h e Newtonian paradigm and the categories of causation described earlier. In brief, if we regard.

t h e phenotype of the system a t time t as e m c t , then (1) Initial phenotype i s material cause.

(2) Genome g i s formal cause.

(3) ^f

,

^(a), as an operator on the initial phenotype, i s efficient cause.

Thus, t h e distinctions w e have made between genome, environment, and phenotype a r e directly related to t h e old Aristotelian categories of causation. As we shall soon discover, that is why these distinctions a r e so important.

Note that one of the Aristotelian categories is missing from the above; there

is no f i n a l c a u s e . Ultimately, this is t h e reason why final cause has been ban- ished from science; t h e Newtonian paradigm simply has no room for it. Indeed, it is

evident that any attempt to superinpose a category of final causation upon t h e Newtonian world would effectively destroy the other categories within it.

In a deep sense, the Newtonian paradigm has led us to the notion t h a t we may effectively segregate the categories of c a u s a t i o n in our system descriptions.

Indeed, the very concept of system state segregates t h e notion of material cause from other categories of causation, and tells us t h a t it is correct to deal with all aspects of material causation independent of other categories: likewise with t h e concepts of genome and environment. I, in fact, claim that this v e r y s e g r e g a t i o n

i n t o i n d e p e n d e n t categories of c a u s a t i o n i s the h e a r t of the Newtonian p a r a - digm. When stated in this way, however, t h e universality of t h e paradigm perhaps no longer appears so self -evident.

Information

We said above t h a t information is, or can be, t h e answer t o a question, and that a question can generally be put in the variational form: If b A , then

dB?.

This serves as t h e connecting bridge between information and t h e Newtonian paradigm.

In f a c t , i t has played an essentiaI role in t h e historical development of Newtonian mechanics and its variants, under the rubric of v i r t u a l d i s p l a c e m e n t s .

In mechanics, a virtual displacement is a small, imaginary change imposed on t h e configurettion of a mechanical s y s t e m , while t h e impressed forces a r e kept fixed. The animating question is: If such a virtural displacement is made under given circumstances, then what happens? The answer, in mechanics, is t h e well- known m n c i p l e of R r t u a l Work: if a mechanical s y s t e m is in equilibrium. then the virtual work done by t h e impressed forces as a result of the virtual displace- ment must vanish. This is a static (equilibrium) principle, but i t can readily be extended f r o m statics t o dynamics, where i t is known as D'dkmbert's P r i n c t p k . In the dynamical case, i t leads directly to the differential equations of motion of a mechanical s y s t e m when the impressed forces a r e known. Details can be found in any text on classical mechanics.

In what follows, w e explore t h e effect of such virtual displacements on the apparently more general class of dynamical systems of t h e form

There is, however, a close relationship between t h e general dynamical systems (7.4) and those of Newtonian mechanics; indeed, t h e former systems can be regarded as arising out of t h e Latter by t h e imposition of a sufficient number of nonholonomic constraints. [l]

As we have already noted. t h e language of dynamical systems. like that of Newtonian mechanics, does not include t h e word information; the study of such systems revolves around t h e various concepts of s t a b i l i t y . However, in one of his analyses of oscillations in chemical systems, Higgins (1967) drew attention to the quantities

These quantities, which he called cross-couplings if i # j and self-couplings if i

=

j , arise fundamentdy from t h e conditions which govern t h e existence of oscillatory solutions to equations (7.4). I t turns out that i t is not so much t h e mag- nitudes as t h e signs of these quantities that a r e important. In order t o have a con- venient expression for t h e signs of these quantities, h e proposed that we call t h e j t h s t a t e variable, z, , an a c t i v a t o r of t h e i t h , in t h e s t a t e (z:

...

^I,), 0 ^whenever

t h e quantity

On Information and C~~llpLez%ty and an i n h i b i t o r whenever

U t j ( 2 ;

,...,

^2;)

<

0

.

Now, activation and inhibition are i n f o r m a t i o n a l terms. Thus, Higgins' ter- minology provides an initial hint as to how dynamical language might be related to informational Language, through t h e Rosetta stone of stability.

Now l e t us examine what Higgins' terminology implies. If z j activates zt in a particular s t a t e , then a (virtual) increase in z j increases t h e r a t e of change of z , or, alternatively, a (virtual) decrease of z j decreases t h e rate of change of z , . I t is, intuitively, eminently reasonable t h a t this is t h e role of an activator. Con- versely, if z j inhibits z , , i t means that an increase in z j decreases t h e r a t e of change of z , , e t c .

Thus, t h e n Z functions. ^uU ( z l

....

^{z n ) ; i , j}

=

¹

...

n , constitute a form of informational description for t h e dynamical system (7.4), which I have elsewhere (Rosen, 1979) called an a c t i v a t i o n - i n h i b i t i o n pattern. As we have noted, such a p a t t e r n concisely represents t h e answers to t h e variational questions: If we make a virtual change in z j , what happens to t h e r a t e of production of z t ? .

There is no reason to consider only t h e quantities u t j . We can, for instance, go one s t e p f u r t h e r , and consider t h e quantities

If we s t a r t from the dynamical equations (7.4), then nothing new is learned from these circumlocutions beyond, perhaps, a deeper insight into t h e relations between dynamical and informational ideas. Indeed, given any layer of informz- tional structure, w e can proceed to succeeckg layers by mere differentiation. and to antecedent layers by mere integration. Thus, knowledge of any layer in this infinite a r m y of layers determines dl of them and, in p a r t i c u l u , the dynzmical equations themselves.

If

we know, for instance, the activation-inhibition pattern Intuitively, these quantities measure t h e effect of a (virtual) change in z k on the e z t e n t to which z j activates o r inhibits z , . If such a quantity is positive in any particular s t a t e , it is reasonable to call z k an a g o n i s t of z j with respect to zt ; if negative, an a n t a g o n i s t . That is, if u t j k is positive, a (virtual) increase in zk increases or facilitates t h e activation of zt by z j , etc. The quantities ui j k thus define another layer of informational interaction, which we may call an agonist-antagonist p a t t e r n .

We can i t e r a t e this process, in fact to infinity, t o produce a t each s t a t e T a family of n functions, u y ( z l,.... z n ) Each layer in this increasing sequence describes how a (virtual) ch;;bge of a variable a t t h a t level modulates t h e proper- ties of t h e preceding level.

So far we have considered only t h e effects of virtual changes in s t a t e MI-i- ables, z ^{j ,} on t h e velocities, dz, / d t , a t various informational levels. W e could simi- larly consider t h e effects of virtual &placements a t these various levels on t h e second derivatives, d2zC / d t 2 (i.e. on the a c c e k m t i o n s of z t ) , t h e third deriva- tives d3zt / dt3. and SO on. Thus. we have a doubly infinite web of informational interactions, defined by t h e functions

rn

a

utjk...r ( Z 1

...

^2,)

=

^{' ' '}

-

azj

f

dz,"

-

^{' ' '}

d t r n

: ^I

10 R. Rosen ut (Z z n ) , w e can r e c o ~ s t r ~ c t t h e dynamic& equations (7.4) through t h e rela- tiomhip

(note in particV& t h a t t h e differential form on t h e right-hand side resembles a generalized w r k ) , and t h e n set t h e function f (z

.

^{zn )} so determined equal t o t h e r a t e of change, dzt / dt , of t h e i t h s t a t e variable.

However, o u r a b i l i t y to do all t h i s d e p e n d s f u n d a m e n t a l l y o n the ezact- n e s s of the di,#erent.inl fonns w h i c h a r i s e a t e v e r y Level of o u r web of infur- m t w n a l i n t e r a c t i o n , and which relate each level t o its neighbors.

If

t h e forms in equation (7.5) are not exact, t h e r e are no functions f (z z,) whose dif- ferentials are given by it, and hence no r a t e e q v a t i o n s of the f o n n V.4). In such a situation, t h e simple relationship between t h e levels in our web breaks down completely; t h e levels become independent of each o t h e r , and must be posited separately. So two systems could have t h e same activation-inhibition patt erns, but vastly different agonist-antagonist p a t t e r n s , and hence manifest entirely dif- f e r e n t behaviors.

To establish firmly t h e s e ideas. let us examine what is implied by t h e require- ment t h a t t h e differential forms

defined by t h e activation-inhibition p a t t e r n b e exact. The familiar, necessary conditions for exactness h e r e t a k e t h e form

for all i , j , k

= I, ...,

n

.

Intuitively, these conditions mean t h a t the r e l a t i o n s of agonism a n d a c t i v a t i o n a r e e n t i r e l y symmetrical (commutative); t h a t zk as an agonist of t h e activator z j is exactly t h e same as z j as an agonist of t h e activator zk ; and similarly for all o t h e r levels.

Clearly, such situations are extremely degenerate in informational t e r m . They a r e so because t h e requirement of exactness is highly nongeneric for dif- f erential forms. Thus, t h e s e very simple considerations suggest a most radical con- clusion: t h a t the Newtonian p a r a d i g m , w i t h its e m p h a s i s o n d y h a m i c a l laws.

r e s t r i c t s u s ~ V o m the o u t s e t to a n extremely special c l a s s of s y s t e m s , a n d t h a t the most e l e m e n t a r y i n f i n n a t i o n a l c o n s i d e r a t i o n s f o r c e us ou't of that class.

We explore some of t h e implications of this situation in t h e following section.

Meanwhile, let us consider some of t h e ramifications of t h e s e informational ideas t h a t hold even within t h e confines of t h e Newtonian paradigm. These con- cern t h e distinctions made in t h e preceding section between environment, pheno- type, and genome; t h e relations of these distinctions t o different categories of causation; and t h e correspondingly different categories of information which these causal categories determine.

First, Let us r e c d t h a t according t o t h e Newtonian paradigm, every relation between physical magnitudes (i.e. every equation of s t a t e ) can b e r e p r e s e n t e d as a genome-parameterized family of mappings

On Information and CompLerSty

from environments t o phenotypes. I t is worth noting specifically t h a t every dynamical law o r equation of motion is of this form, as is shown by

Here, in traditional language, z is a vector of s t a t e s , a is a vector of e x t e r n a l con- trols (which together with s t a t e s constitutes e n v i r o n m e n t ) , and t h e phenotype is t h e tangent vector dz / d t a t t a c h e d t o t h e state z

.El

In this case, t h e n , t h e tangent vector o r phenotype constitutes e m c t ; t h e genome g is identified with formal cause, s t a t e z with material cause, and t h e o p e r a t o r f (..., a ) with efficient cause.

By analogy with t h e activation-inhibition networks and t h e i r associated informational s t r u c t u r e s , described above, we can consider formal quantities of t h e form

A s always, such a formal quantity r e p r e s e n t s an answer to a question: Lf (cause Is varied), t h e n (what happens t o effect)? This is t h e same question as w e asked in connection with the definition of activation-inhibition networks and their corre- lates, but now set in t h e wider c o n t e x t t o which our analysis of t h e Newtonian paradigm has led us. That is, we may now virtuaLly dlsplace a n y magnitude which affects t h e relation (7.6), whether it b e a genomic magnitude, an environmental magnitude, o r a state variable. In a precise sense, t h e effect of such a virtual dis- placement is measured by t h e quantity (7.7).

I t follows t h a t t h e r e a r e indeed different k i n d s of information. What kind of information we are dealing with depends on whether we apply t h e virtual displace- ment t o a genomic magnitude (associated with formal cause), an environmental mag- nitude (efficient cause), o r a s t a t e variable (material cause). Formally, we can now distinguish a t least t h e following three cases:

(1) Genomic information,

a la

^(effect)

a(genome)

,

dt

(2) Phenotypic information,

I \

(3) Environmental information,

t \

We confine ourselves herein t o t h e s e t h r e e , which generalize only t h e activation-inhibition p a t t e r n s described above.

W e now examine an important idea; namely, t h e t h r e e c a t e g o r i e s defined above a r e n o t e q u i v a l e n t . Before justifying this assertion, w e must briefly dis- cuss what is meant by equivalent. In generai, t h e mathematical assessment of t h e effects of perturbations (i.e. of real o r virtual displacements) is t h e province of

s t a b i l i t y . For example, t h e effect on subsequent dynamical behavior of modifying o r perturbing a system s t a t e is t h e province of Lyapunov stability of dynamical systems; t h a t of perturbing a control is p a r t of control theory: and t h a t of per- turbing a genome r e l a t e s t o s t r u c t u r a l stability. To e s t a b h h this firmly, l e t us consider genomic perturbations, o r m u t a t i o n s . A virtual displacement applied t o a genome g replaces t h e initial mapping f g dete.-mined b y g with a new mapping fg*. Mathematically, we say t h a t t h e two mappings, f and f ,, are equivalent, o r similar, o r conjugate, if t h e r e exist appropriate transformations

such that t h e diagram

commutes; t h a t is, if

f o r every e in E. Intuitively, this means t h a t a mutation ^g H g ' can b e counter- balanced, o r nullified, b y imposing suitable coordincrte t m n s f o m a t i o n s on t h e environments and phenotypes. S t a t e d y e t another way, a virtual displacement of genome can always b e counteracted b y corresponding displacements of environ- ment and phenotype so t h a t t h e resultant variation on effect vanishes.

We have elsewhere (Rosen. 1978) shown at great length t h a t this commuta- tivity may not always obtain; t h a t is, t h a t t h e r e may exist genomes which are bifurcation points. In any neighborhood of a bifurcating genome g , t h e r e exist genomes g ' f o r which f, and f,, fail t o b e conjugate.

With this background, wereturn to t h e question of whether t h e t h r e e kinds of information (genomic, phenotypic, and environmental) defined above are equivalent. Intuitively, equivalence would mean t h a t t h e effect of a virtual dis- placement 6g of genome, supposing all else is fixed, could equally well b e produced by a virtual displacement of environment, 6 a , o r of phenotype, S p . O r s t a t e d a n o t h e r way, t h e e f f e c t of a virtual displacement bg of genome can b e nullified by virtual displacements -6a and -bp of environment and phenotype, respectively.

This is simply a restatement of t h e definition of conjugacy o r similarity of map- pings.

If all f o m s of infomation are equivalent, i t follows t h a t t h e r e could b e no bifurcating genomes. We note in passing t h a t t h e assumption of equivalence of t h e t h r e e kinds of information defined above thus c r e a t e s t e m b l e ambiguities when i t comes t o e x p l a n a t i o n of particular effects. W e do not cocsider t h a t aspect h e r e , e x c e p t to say t h a t i t is perhaps very fortunate t h a t , as w e have seen, t h e y - a r e not equivalent.

On Infinnat%on a n d CompLertty 13

Let us exarcine one immediate consequence of t h e nonequivalence of genomic, environmental, and phenotypic information, and of t h e considerations which cul- minate in that conclusion. Long ago (cf. von Neumann, 1951; Burks, 1966) von ru'eu- mznn proposed an influential model for a self-reproducing automaton, and subse- quently, for automata which grow and develop. This model w a s based on t h e famous theorem of Turing (1936), which estabkshed t h e existence of a universal computer (universal Turing machine). From t h e existence of such a universal computer, von Neuma,.nn zsserted t h a t t h e r e must also exist a universzl constructor. Basically, he argued t h a t computation (i.e. following a program) and construction (following a blueprint) a r e both algorithmic processes, and t h a t anything holding for one class of algorithmic processes necessarily holds for any o t h e r class. Tkis universal con- structor formed t h e central ingredient of t h e self-reproducing automaton.

Now, a computer acts, in t h e language we have developed above, through t h e manipulation of efficient cause. A constructor, if t h e term is t o bear any resem- blance to its intuitive meaning, must essentially manipulate material cause. The inequivalence of t h e two categories of causality, in particular manifested by t h e nonequivalence of environmental and phenotypic information, means that we can- not blithely extrapolate f r o m results pertaining to efficient causation into t h e realm of material causation. Indeed, in addition to invalidating von Neumann's specific argument, we learn t h a t great care must be exercised in general when arguing from purely logical models (i.e. from models pertaining to efficient cause) to any kind of physical 'realization. such as developmental o r evolutionary biology (which pertain to material cause).

Thus, we realize how significant a r e t h e impacts of informational ideas, even within the confines of t h e Newtonian para-, in which t h e categories of causa- tion a r e essentially segregated into separate packages. We now consider what hap- pens when we vacate t h e comforting confines of t h e Newtonian paradigm.

Introduction to Complex Systems

Herein, I call any natural system for which t h e Newtonian para- is com- pletely valid z simple s y s t e m , or mechunism. Accordingly, a complex s y s t e m is one which, for one reason or another, resides outside this para-. We have already seen a hint of such systems in t h e preceding section; for example, sys-

tems whose activation-inhibition patterns u t j do not give rise t o exact differen- tials u,, dz,

.

However, some f u r t h e r words of motivation must precede a conclu- sion that-such systems are truly complex (i.e. reside fundamentally outside t h e Newtonian paradigm). W e must also justify our very usage of t h e term complex in this context.

What I have been calling t h e Newtonian paradigm ultimately devolves upon the c l a s s of d i s t i n c t mathematical d e s c r i p t i o n s which a system can have, and t h e relations which exist between these descriptions. As noted earlier, t h e basis of system description arising in this para&gm is t h e fundamental dualism between states and dynamical laws. Thus, t h e mathematical objects which can describe natural systems comprise a category which may be called general dynamical sys- tems. In a formal sense, i t appears t h a t any mathematical object resides in this category, because t h e Newtonian partition between states and dynamical laws exactly parallels t h e partition between propositions and production rules (rules of

14 R. Rosen

inference) which presently characterize all logical systems and logical theories.

Ifowever, we argue t h a t , although this category of general dynamical systems is large, it is not everything, and, indeed, it is far from large enough.

The Newtonian p v a d i g m asserts much more than simply t h a t every image of a natural system must belong t o a given category. I t a s s e r t s c e r t a i n relationships between such images. In particular (and this is t h e reductionistic content of t h e parad.lgxn), i t asserts t h a t among these images t h e r e is t h e universal one, which effectively maps on all t h e o t h e r s . Intuitively, this is t h e master description o r ultimate description, in which every s h r e d of physical reality has an e x a c t mathematical c o u n t e r p a r t ; in category-theoretic terms, i t is much like a f r e e object (a generalization of t h e concept of f r e e semigroup, f r e e group, etc.).[3]

There is still more. The ingredients of this ultimate description, b y t h e i r very nature, a r e themselves devoid of internal s t r u c t u r e ; t h e i r only changeable aspects a r e t h e i r relative positions and velocities. Given t h e forces acting between them, as Laplace noted long ago, everything t h a t happens in t h e e x t e r n a l world is in principle predictable and understandable. From this perspective, everything is determined; t h e r e a r e no mysteries, no surprises, no e r r o r s , no questions, and no information. This is as much true f o r quantum t h e o r y as f o r clns- sical; only t h e nature of s t a t e description has changed. And i t applies to every- thing, from atoms t o organisms t o galnues.

How does this universal picture manifest itself in biology? First, from t h e standpoint of t h e physicist, biology is concerned with a r a t h e r small class of extremely special (indeed, inordinately special) systems. In t h e theoretical physicist's quest f o r general and universal laws, t h e r e is thus not much contact with organisms. As f a r as h e o r s h e is concerned, what makes organisms speclal is not t h a t they transcend t h e physicist's para-, b u t r a t h e r t h a t t h e i r specifi- cation within t h e paradigm requires a plethora of special constraints and condi- tions, which must be superimposed on t h e universal cenons of system description and reduction. The determination of t h e s e special conditions is an empirical task;

essentially someone else's business. But i t is not doubted t h a t t h e relationship between physics and biology is t h e relationship between t h e general and t h e par- ticular.

The modern biologist, in general, avidly embraces this perspective.[4] Histor- ically, biology has only recently caught up with t h e Newtonian revolution which swept t h e rest of natural philosophy in t h e seventeenth century. The three- c e n t u r y lag arose because biology has no analog of t h e solar system; no way to make immediate and meaningful contact with t h e Newtonian paradigm. Not until physics and chemistry had elaborated t h e technical means t o probe microscopic properties of m a t t e r (including organic matter) was t h e idea of molecular biology even thinkable. And this did not happen until t h e 1930s.

A t p r e s e n t , t h e r e is still no single M e r e n t i a l chain which Links any impor- t a n t effect in physics t o any i a p o r t a n t e f f e c t in biology. This is a f a c t ; a datum; a piece of information. How a r e w e t o understand i t ? There a r e v v i o u s possibilities.

Kant, long ago, argued t h a t organisms could only be properly understood in terms of final czuses o r intentionality; hence, from t h e outset h e suggested t h a t organ- isms fall completely outside t h e canons of Newtonizn science, which a r e applicable to everything else. Indeed, t h e essential telic nature of organisms preciuded even t h e possibility t h a t z "Newton of t h e grassblzde" would come along, and do for biology what Newton did f o r physics. Another possibility is t h e one we have

On Infirmatton and CampLezity 15

already mentioned; we have simply not y e t characterized ail those special condi- tions which a r e necessary t o bring biology fully within t h e scope of universal phy- sical principles. Yet a t h i r d possibility has developed within biology itself, as a consequence of theories of evolution; i t is t h a t .much of biology is t h e result of a c c i d e n t s which are in p r i n c i p l e unpredictable a i d hence governed by no laws a t ail.151 In this view biology is as much a bran+ of history as of science. A t p r e s e n t , this last hypothesis lies in a s o r t of doublethink relation with reduction- ism; t h e two are quite inconsistent, but do allow modern biologists to enjoy t h e benefits of vitalism and mechanism together.

Yet a fourth view was e x p r e s s e d by Albert Einstein, who wrote in a l e t t e r t o Leo Szilard: "One can b e s t appreciate, from a study of Living things, how primitive physics still is

".

So, t h e p r e s e n t prevailing view in biology is t h a t t h e Newtonian canons are indeed universal, and we a r e lacking only knowledge of t h e special conditions and constraints which distinguish organisms from o t h e r natural systems within those canons. One way of describing this with a single word is t o assert t h a t organisms are c o m p k z . This word is not well defined, b u t it does connote several things. One of t h e s e is t h a t complexity is a system p r o p e r t y , no different from any o t h e r pro- p e r t y . Another is that t h e degree to which a system is complex can b e specified b y a number, o r set of numbers. These numbers may be i n t e r p r e t e d variously as t h e ciimensionality of a state space, o r t h e length of an algorithm, o r as.a cost in time o r energy incurred in solving system equations.

On a more empirical level, however, complexity is recognized differently, and characterized differently. If a system surprises us, o r does something we have not predicted, o r responds in a way w e have not anticipated; if i t makes e r r o r s ; if i t exhibits emergence of unexpected novelties of behavior, we also say t h a t t h e sys- t e m is complex. In s h o r t , complex systems are those which behave counter- intuitively.

Sometimes, of course, surprising behavior is simply t h e result of incomplete characterization; we can t h e n hunt f o r what is missing, and incorporate it into our system description. In this way, t h e planet Neptune was located from unexplained deviations of Uranus from i t s e x p e c t e d trajectory. But sometimes this is not t h e case; in t h e apparently analogous case of t h e anomalies of t h e trajectory of t h e planet Mercury, for instance, no amount of fiddling within t h e classical scenario succeeded and only a massive readjustment of t h e parachgm itself (via general relativity) availed.

From these f e w words of introduction, we can conclude t h a t t h e identification of complexity with situations where t h e Newtonian paradigm fails is in accord with t h e intuitive connotation of t h e term, and is a n alternative t o regarding as com- plex any situation which merely is technically difficult within t h e paradigm.

Now l e t us see where information fits into t h e s e considerations. We recall t h a t information is t h e actual o r potential response t o an interrogative, and t h a t every interrogative can b e put into t h e variational form: If b A , t h e n

6B?

The Newtonian paradigm asserts, among o t h e r things, t h a t t h e answers t o such interro- gatives follow from dynamical laws superimposed on manifolds of states. In t h e i r t u r n , these dynamical laws a r e special cases of e q u a t i o n s of s t a t e , which h k o r r e l a t e t h e values of system observables. Indeed, t h e concept of an observable w a s

16 R. Rosen

t h e point of d e p a r t u r e for our e n t i r e treatment of system description and representation (cf. Rosen, 1978); it w a s t h e connecting Link between t h e world of natural phenomena and t h e entirely different world of formal systems which we use t o describe and explain.

However, t h e considerations we have developed above suggest t h a t this world is not enough. We require also a world of variations, increments, and differentials of observables. I t is true t h a t every Linkage between observables implies a corresponding linkage between differentials, but as we have seen, t h e converse is not true. W e are t h u s drawn t o t h e notions t h a t a differential relation is a general- ized linkage and t h a t a differential form is a type of generalized observable. A dif- ferential form which is not t h e differential of an observable is thus an e n t i t y which assumes no definite numerical value (as an observable does), but which can b e incremented.

If we do think of differential forms as generalized observables, t h e n we must correspondingly generalize t h e notion of equation of s t a t e . A generalized equation of state thus becomes a linkage o r relation between ordinary observables and dif- ferentials o r generalized observables. Such generalized equations of s t a t e are t h e vehicles which answer questions of our variational fox-m: If b A , t h e n 6B?

But as we have repeatedly noted, such generalized equations of s t a t e do not usually follow from systems of dpnamical equations, as t h e y do in t h e Newtonian para-. Thus, we must find some alternative way of characterizing a system of this kind. Here is where t h e informational language introduced above comes t o t h e fore. Let us recall, f o r instance, how we defined t h e activation-inhibition net- work. W e found a family of functions u t , (i.e. of observables) which could b e thought of in t h e dynamical context as m&ulating t h e effect of an increment d r j on t h a t of another increment d f t . That is, t h e values of each observable, u t j , measure precisely t h e e x t e n t of activation o r inhibition which z j exerts on t h e rate a t which zt is changing.

.

In this language, a system falling outside the Newtonian p a r a w (i.e. a com- plex system) can have an activation-inhibition p a t t e r n , just as a dynamical (i.e.

simple) system does. Such p a t t e r n s are still families of functions (observables), u t j , and t h e p a t t e r n itself is manifested by t h e differential forms

But in this case, t h e r e is no global velocity observable, f i , t h a t can b e inter- p r e t e d as t h e rate of change of z t ; th e r e is only a velocity increment. I t should b e noted explicitly t h a t u t j , which define t h e activation-inhibition p a t t e r n , need not b e functions of 'zc alone, o r even functions of them a t all. Thus, t h e differen- tial forms which arise in this context a r e different from those with which mathematicians generally deal, a n d which can always b e regarded as cross sections of t h e cotangent bundle of a definite manifold of s t a t e s .

The n e x t level of information is t h e agonist-antagonist p a t t e r n , u t f k . In t h e category of dynamical systems, this is completely determined by t h e activation-inhibition p a t t e r n , and can b e obtained from t h e l a t t e r by differentia- tion:

On Injbrmcrtton and CompLezity 17

In our world of generalized observables and linkages, u i f k are independent of u t j , and must be posited separately: in other words, complex (non-Newtonian) sys- tems can have identical activation-inhibition patterns, but quite different agonis t-antagonis t patterns.

Exactly the same considerations can also be applied to every subsequent layer of the informational hierarchy; each is now independent of the others, and so must be posited separately. Hence a complex system requires an i n f i n i t e mathematical object for its description.

We cannot examine herein t h e mathematical details of t h e considerations sketched so briefly above. Suffice it to say that a complex system, defined by a hierarchy of informational levels of the type described, is quite a different object to a dynamical system. For one, i t is quite clear that there is no such thing as a s e t of s t a t e s , assignable to such a system once and for all. From this alone, we might expect that the nature of causality in such systems is vastly different to what i t is in the Newtonian paradigm; we come to this in a moment.

The totality of mathematical structures of the type we have defined above forms a category. In tkis category the class of general dynamical systems consti- tutes a very small subcategory. We a r e suggesting that the former provides a suit- able framework for the mathematical imaging of complex systems, while the latter, by definition, can only image simple systems or mechanisms. If these considera- tions a r e valid (and I believe they are), then the entire epistemology of our approach to natural systems is radically altered, and it is the basic notions of information which provide t h e natural ingredients.

There is, however, a profound relationship between the category of general dynamical (i.e. Newtonian) systems, and the Larger category in which it is embed- ded. This can only be indicated here, but it is important indeed. Namely, there is a precise sense in which an informational hierarchy can be a p p r o x i m a t e d , locally and temponrily, by a general dynamical system. With this notion of approximation there is an associated notion of l i m i t , and hence of topology. Using these ideas, it can be shown that what we call the category of complex systems is the completion, or limiting s e t , of the category of simple (i.e. d y n d c d ) systems.

The fact that complex systems can be approximated (albeit locally and tem- porarily) by simple ones is crucial. It explains precisely why the Newtonian para- digm has been so successful, and why, to this day, i t represents the only effective procedure for deaLing with system behavior. But in general, it is apparent that it can usually supply o n l y approximations, and in the universe of complex systems this amounts to replacing a complez system with a simp& s u b s y s t e m . Some of the profound consequences a r e considered in detail in Rosen (1978).

This relationship between complex systems and simple ones is, by its very nature, without a reduc tionistic counterpart. Indeed, what we presently under- stand as physics is seen in this Light as the science of simp& s y s t e m s . The relation between physics and biology is thus not a t all the relation of general t o particular: in fact, quite the contrary. It is not biology, but physics, which is too special. We can see from this perspective that biology and physics (i.e. contem- porary physics) develop as two divergent branches from a t h e o r y of complez sys- tems, which as yet can be glimpsed only very imperfectly.

The category of simple systems is, however, still the only one tnat we know how to use. But to study complex systems by means of approximating simple sys- tems resembles the position of early c a r t o h p h e r s , who were attempting to map a