Jerzy Paczynski, Tomasz Kregle wski
I n s t i t u t e of Automatic Control, Warsaw University of Technology.
Abetract
This paper presents remarks on the methodology of generation of computer models for nonlinear problems. The automatic differentiation of the model is discussed and implementation hints are given. The presentation is based on the experience gained during implementations of the nonlinear model generator, the IAC-DIDAS-N system and observations of other software produced under the contracted study agreement with IIASA.
1 From abstract model formulation to the generation of a computerized model
A fairly abstract formulation of the equation of a model (or process bounds) has the form
where z E B, denotes internal variables, u E B, denotes control variables, r E B, denotes disturbances and a E B, denotes parameters. P : B, x B, x B, x B, +
B,
is a nonlinear mapping of bounds.B,
is the space of bounds, usually isomorphic t o B,. The above equation is accompanied by the output equation in an explicit formwhere V : B, x B, x B, x B, -+ By is the output mapping. A wide spectrum of models -
including dynamic, distributed, etc. formulations - can be described in this general form by suitable choice of spaces and operators.
It is difficult t o draw more detailed conclusions from the general form of the equation of bounds. Therefore the concept of a resolving mapping R is introduced; it is an operator which determines z while given u, z, a. In most cases the resolving operator is realized in a feedback system, so the notation
z = R ( P ( z , u, z, a ) )
will be used. The existence and properties of this operator usually are investigated by using the contraction mapping theorem. Details depend very strongly on the spaces assumed, often i t can be a numerical iterative process. In special but very important for applications cases the bounds equation has an explicit form
z = Q(u, z, a ) .
Structural properties of the model can be visualized in the form of a generalized directed graph (Wierzbicki, 1984). The nodes of this graph can be of three sorts:
i) a node denoted by a circle corresponds to a variable,
ii) a node denoted by a double circle corresponds to the ordered n-tuple of variables and is labelled with it,
iii) a node denoted by a square corresponds to a value which is stabilized.
The arcs of the graph are labelled with operators; to improve readability the identity operators I can be omitted.
Typical examples are presented in Fig. 1 for the equations: a) P ( z , u) = 0 b) z = R(z, u).
Figure 1.
The arc labelled with a non-identity operator must start from a node of the "double circle" sort. Thus in case of a composite operator an additional node must be inserted. E.g.
the equation
= Q(S(u)) corresponds to the graph presented in Fig. lc.
From the implementation point of view a computer model can have one of the following forms:
-
a program in a simulation language,- a procedure (or a collection of them) in an universal programming language,
-
a program in a declarative language, - a set of entries in a spreadsheet.This list is by no means exhaustive, it is rather the indication of the wide spectrum of pos- sible solutions. However, t o build a computer model on the base of a "bare" language or
programming system requires specialized knowledge of the relevant aspects of the computer science and techniques, in addition to the specific science determined by the modelled prob- lem. Proficiency in this specific science is not necessarily accompanied by the proficiency in using software tools. Therefore the whole process can be time-consuming and frustrating the user.
Therefore, a modern software must be equipped with a user-friendly interface, which will hide most technical details from the user. In the described context it should posses three main properties:
i) Edition. It should be done in terms of the corresponding problem science only,
ii) Verification. A safe environment for model verification (performing partial or complete calculations, simulations etc.) should be provided,
iii) Modification. Easy transfer between two previous phases is necessary for efficient con- struction of a model.
Many examples of the need and development of such interfaces can be found in this book (see e.g. DIDAS, DINAS, HYBRID, DISCRET). More detailed references to papers on interfaces for nonlinear models are given in (Paczynski and Kreglewski, 1987).
2 Automatic differentiation of the model
-general concepts and implement at ional hints
Differentiation of a model arises in several contexts, e.g.
- series expansion,
- solution of implicit equations based on the implicit function theorem, - sensitivity analysis,
- optimization.
In any specific formulation of the abstract model class described above, it is necessary to check the existence and properties of the derivatives (Gateaux or Frechet) of the model. In the implementation practice the classical differential calculus is often used, but nevertheless a manual differentiation presents considerable problems in the case of large and complicated models. This process is very prone t o analytical errors which can make the model incon- sistent. The existence of such errors is, as a rule, very hard to detect. Awareness of this situation leads often t o the practical abandonment of theoretically promising methods, e.g.
using sensitivity models in simulation languages or calculating Hessians matrices in optimiza- tion. Implementation of the automatic differentiation plays therefore an important role in various problems of system analysis.
It will be assumed in the following that all necessary conditions of differentiability and of the applicability of the implicit function theorem etc. are fulfilled. The rules of differentiation of a model can be easily formulated in terms of its graph representation (Wierzbicki, 1984):
i) the graph of the total derivative retains the structure of the model,
ii) all non-identity arc labels are advanced one arc backwards and are replaced by the operators of appropriate derivatives,
iii) vacant labels are filled with the identity operators.
Total derivatives of the graphs presented in Fig. 1 are presented in Fig. 2.
Figure 2.
In every finite graph, it is possible to distinguish a finite number of closed feedback loops corresponding to implicit operations and a finite number of feedforward connections corresponding to composite operations. Thus the subsequent applications of the above rules gives the desired result.
It should be stressed that in some cases the graph may be rather a conceptual device than an element of implementation. In the creation of a sensitivity model it may be used explicitly, while in other cases it may be used implicitly by the application of recursive procedures.
Many problems of automatic differentiation (in more narrow context) have been discussed by Rall (1981.) but computer implementations presented there are rather obsolete. From the computational point of view the differentiation of the model resembles much the compilation process. General information about compilers can be found e.g. in (Aho and Ullman, 1977).
The model equations must be described in a form suitable for computer analysis. Thus the first step is the creation of the input language (the problem language). This language must assure an easy interface with the user and an easy interface with the differentiating program.
In practice some iterations are often needed until a mature form of the language is obtained.
The process of the automatic differentiation usually can be divided into some phases.
The first one is the lexical analysis, i.e. the division of characters of the input language into groups that logically belong together-into symbols as e.g. identifiers of variables, symbols of operators. The output of the lexical analyser is a stream of symbols, which is passed t o the next phase, t o the syntax analyser (parser). The parser checks whether the symbols appearing a t its input form a legal sequence of the input language (defined by its syntax rules) and produces an intermediate code. This code can be of very diverse nature, according both t o the details of the problems and t o the details of the particular implementation. A graph, a tree or a stack structure are typical examples. This intermediate form of the model
is actually differentiated and thus another structure in the intermediate form, representing the derivative, is produced.
Usually the derivative must be simplified to an acceptable form. This phase is one of the hardest t o implement due t o its internal complication and nonexistence of the "simplestn canonical forms. The goal of simplification depends heavily on details of the particular problem. In the case of a simulation program it can be the number and the interconnection pattern of the modelling blocks, in the case of the evaluation of values-the computation efficiency measured as time, when the output has the form of symbolic formulae-similiarity to the analogous result obtained by a mathematician.
In the final step this code is transferred into the form of the output language. Occasionally it can coincide with the input language, however many different forms are possible e.g. files with a simulation program or with a numerical procedure, a code for the calculations on a virtual computer or just formulae.
Under the contract with IIASA two differentiation packages were implemented for appli- cations in differentiable optimization packages and decision analysis and support system. The first package, used in the IAC-DIDAS-N system, is oriented towards efficient calculation of values of derivatives; the second, used in a nonlinear rnodel generator, is oriented on symbolic presentation of formulae of derivatives.
References
Aho A.V. and D. Ullman (1977). Principles of Compiler Design. Addison Wesley.
Paczynski J. and T . Kreglewski (1987). Nonlinear Model Generator. In: Theory, Software and Testing Examples for Decision Support Systems. WP-87-26, IIASA, Laxenburg, Austria.
Rall L.B. (1981). Automatic Differentiation: Techniques and Applications. Springer.
Wierzbicki A. (1984). Models and Sensitivity of Control Systems. Elsevier.