Interactive Information System for Technology Assessments

(1)

W O R K I N G P A P E R

I N T E R A C T I V E INFOXIilATION S Y S T E M F O R TECHNOLOGY A S S E S S M E N T S

A . K . A l a b y a n A . P . G o l o v i n e V . R . O k o r o k o v V . P o n o m a r e v

N o v e m b e r 1 9 8 7 W P - 8 7 - 1 2 1

I n t e r n a t i o n a l I n s t i t u t e for Applied Systems Analysis

(2)

I - N T l m A m INmRMATION

ETrsrm FOR

TM=HNOIXH;Y ASSESSMENTS

A.K. ALabyan A.F. Golovine

VR.

Okorokou 1.I Ponomareu

November

1 9 8 7

WP-87-121

Working P a p e r s are interim r e p o r t s on work of the International Institute f o r Applied Systems Analysis and have r e c e i v e d only limited review. Views or opinions e x p r e s s e d h e r e i n d o not necessarily r e p r e s e n t those of the Institute or of i t s National Member Organizations.

INTERNATIONAL INSTITUTE

FOR

APPLIED SYSTEMS ANALYSIS A-2361 Laxenburg, Austria

(3)

Evaluation and assessment of new technologies i s a r a t h e r complicated t a s k due to t h e involvement of g r o u p s of e x p e r t s , multiple c r i t e r i a c h a r a c t e r i z i n g s e v e r a l al- t e r n a t i v e s as well as incomplete information a b o u t t h e s e a l t e r n a t i v e s . E x p e r t analysis of new technologies by d i f f e r e n t a s p e c t s c a n b e one of t h e wayx of estimating t h e advantages a n d shortcomings of e a c h of them a n d of forecasting t h e i r development and usage.

Due to t h e c h a r a c t e r of t h e assessment p r o c e d u r e , especially in t h e g r o u p ex- p e r t situation, l a r g e amounts of information must b e processed and analyzed in o r d e r to find t h e final conclusion. Additionally, s e v e r a l f a c t o r s reflecting t h e quality of t h e r e s u l t s , quality of e x p e r t s opinions, etc. must b e calculated during t h e assessment p r o c e s s . T h e r e f o r e , this task should b e s u p p o r t e d by some comput- er based tools. The p a p e r p r e s e n t s s u c h a n information management system sup- porting t h e p r o c e s s of technology assessment. The system perfarms such functions like information collection a n d s t o r a g e , i n t e r a c t i o n with e x p e r t s a n d analysts, aggregation of information, g r a p h i c presentation of d a t a and r e s u l t s as well as computes s e v e r a l s t a t i s t i c a l f a c t o r s n e c e s s a r y to analyze t h e d a t a submitted by e x p e r t s . The system, being t h e f i r s t s t e p towards development of more advanced decision s u p p o r t systems h a s been applied at IIASA f o r analysis of s e v e r a l technologies f o r e n e r g y production.

Alexander B. Kurzhanski Chairman

System and Decision Sciences Frograrn

(4)

Content

INTRODUCTION

MAIN P R I N C I P L E S OF TAS ASSESSMENT PROCEDURE GENERAL STRUCTURE OF TAS SOME PRELIMINARY ASPECTS REFERENCES

(5)

I N T E ' R A m INRIRMATION

!3WlXM

FOR TECHNOLOGY ASSESS-

A.K. ALabyan, A.P. Golovine, J!R. Okmrokou and I.! P o n o m a r e u

1.

INTRODUCTION

T h e r e are s e v e r a l ways t o d e a l with t h e problems of technology assessments. The t r a d i t i o n a l o n e s use d i f f e r e n t s o r t s of economic analysis a n d are well known. The t r a d i t i o n a l a p p r o a c h i s useful f o r well-defined technologies, f o r which t h e r e l i a b i l i t y of t h e input d a t a i s highly verified. When we d e a l with some new technologies i t i s no l o n g e r t h e c a s e . T h e r e are many i s s u e s beyond t h e economic f a c t o r s s u c h as s a f e t y ,

R&D

problems, environmental a n d s o c i a l impacts and o t h e r s t h a t are of g r e a t i m p o r t a n c e b u t sometimes c a n n o t b e e v a l u a t e d nu- merically with economic calculations. While assessing t h e f u t u r e development of new technologies, i t i s possible to r e d u c e u n c e r t a i n t i e s relying on t h e opinions of e x p e r t s . E x p e r t analysis of new technologies by d i f f e r e n t a s p e c t s c a n b e o n e of t h e ways of estimating t h e a d v a n t a g e s a n d shortcomings of e a c h of them a n d of f o r e c a s t i n g t h e i r f u t u r e development a n d usage.

The method of a n e x p e r t computerized a n a l y s i s of technologies developed at IIASA a n d d e s c r i b e d h e r e i s based on a q u e s t i o n n a i r e (an example of t h e q u e s t i o n n a i r e t o b e filled o u t by o n e of t h e e x p e r t s i s p r e s e n t e d in Appendix I ) , surnmerizing a l l n e c e s s a r y information t a k e n f r o m t h e e x p e r t s a n d a n i n t e r a c - t i v e c o m p u t e r system t h a t makes a l l calculations, d a t a p r o c e s s i n g , and r e p r e s e n t a t i o n s .

This f i r s t v e r s i o n of technology assessments (TAS) d e s c r i b e s an information system f o r t h e policy m a k e r s as a tool f o r t h e full-scale analysis of d i f f e r e n t t e c h - nologies.

Before d e s c r i b i n g t h e p r o p o s e d p r o c e d u r e of technology a s s e s s m e n t s some preliminary c o n s i d e r a t i o n s should b e made c o n c e r n i n g t h e problem of t h e human possibiUties to make estimates. Some assumptions c o n c e r n i n g t h e models of human information processing a n d decision making could b e found in t h e works devoted to sociological r e s e a r c h a n d in modern t r e n d s in e x p e r t systems design (see, f o r example 11, 2, 31).

(6)

R e s e a r c h e r s engaged in measurement a n d mathematical modelling of human phenomena meet t h e problem to s u b j e c t human b e h a v i o u r t o numerical analysis. T h e r e i s s t r o n g c r i t i c i s m now t h a t e s s e n t i a l individual c h a r a c t e r i s t i c s are swallowed u p in t h e sameness of quantity. Indeed to a c e r t a i n e x t e n t a same- n e s s i s a s s e r t e d when applying measurment to human phenomena. However, t w o points should b e r e c a l l e d . F i r s t , measuring c e r t a i n p r o p e r t i e s c o n n e c t e d with human a c t i v i t i e s n e e d n o t imply t h a t t w o c a s e s , when t h e s e p r o p e r t i e s are identical, c a n n o t d i f f e r in many o t h e r r e s p e c t s . Indeed o n c e t h e s e similarities are known i t may b e e a s i e r to c o n c e n t r a t e on t h e d i f f e r e n c e s between individuals.

Second, some s c a l e s of measurement are more r e s t r i c t i v e t h a n o t h e r s . The identification of o b j e c t s by c a t e g o r i e s i n t o which t h e y f i t , or by r a n k s , c a p t u r e s some q u a l i t a t i v e similarities. A t t h e same time f e w e r presumptions are made a b o u t t h e i r s a m e n e s s , as i s t h e case when t h e y p o s s e s s i d e n t i c a l values on a m e t r i c s c a l e . Notwithstanding t h i s c r i t i c i s m pointing to the limitations of measurment, however, t h e r e i s a n i n c r e a s i n g recognition t h a t a qualitative a p p r o a c h need n o t eschew measurment.

In r e c e n t y e a r s s o c i a l s c i e n t i s t s h a v e b e e n m o r e a n d more c o n c e r n e d with measuring q u a l i t i e s in o r d e r to g r a p p l e with complex c o n f i g u r a t i o n s a n d un- c e r t a i n t i e s i n h e r e n t in human p e r c e p t i o n a n d b e h a v i o u r . The difficulties assosi- a t e d with measuring a n d numerical a n a l y s i s of human a c t i v i t i e s remain im- mense. Techniques of q u a l i t a t i v e d a t a analysis are e s s e n t i a l in a n y e f f o r t to i n c o r p o r a t e non-numerical information e x t r a c t e d from humans. But i t i s n e c e s s a r y sometimes to a c h i e v e e v e n more: to g e t some numerical c h a r a c t e r i s t i c s as t o human estimations of some p r o c e s s e s or systems p e r f o r m a n c e .

A major f a c t o r which c a n a f f e c t t h e ease with which p e o p l e u s e a n e x p e r t system i s t h e a b i l i t y of t h e system to t a i l o r i t s b e h a v i o u r to t h e s p e c i f i c f e a t u r e s a n d needs of a n individual u s e r . This i s most d e s i r a b l e where o n e p a r t i c u l a r system i s to b e used by p e o p l e with s u b s t a n t i a l l y differing backgrounds. To b e effec- t i v e , s y s t e m s should know who t h e i r u s e r s are a n d t h e c o n t e x t in which t h e y are t r y i n g to work. T h e r e are s e v e r a l ways in which a system c a n t a i l o r i t s b e h a v i o u r towards d i f f e r e n t u s e r s . The most simple i s w h e r e t h e u s e r i s a s k e d t o classify himself at t h e beginning of i n t e r a c t i o n as belonging t o a c e r t a i n c a t e g o r y . In more s o p h i s t i c a t e d a p p r o a c h s a c l u s t e r a n a l y s i s i s used. T h e r e are v a r i o u s t y p e s of u s e r information which should b e included i n t o t h e system.

These g e n e r a l l y include knowledge a b o u t a u s e r ' s l e v e l of competence, h i s in- t e r e s t s , values, a p t i t u d e s , goals, e x p e c t a t i o n s a n d assumptions a n d e v e n knowledge a b o u t t h e u s e r ' s model of how t h e system works. In t h e r e a l decision p r o c e s s e a c h e x p e r t c e r t a i n l y h a s h i s own understanding of t h e s t r a t e g y

(7)

t h a t influences h i s assignments. Besides individual c a p a b i l i t i e s , e v e n h i s p r e s e n t motivations are of i m p o r t a n c e . To a c h i e v e positive r e s u l t s i t i s n e c e s s a r y n o t only to v e r i f y t h e initial assignments b u t a l s o to divide t h e e x p e r t s i n t o national, p r o f e s s i o n a l a n d o t h e r g r o u p s b e c a u s e t h e i r opinions could d i f f e r .

While a n a l p i n g s u c h a problem as technology assessements, i t becomes clear t h a t t h e main problem i s n o t only to c h o o s e c o r r e c t l y t h e set of a l t e r n a - t i v e s , c r i t e r i a a n d t h e measurement scale b u t also to a r r a n g e t h e p r o c e d u r e f o r a c c u r a t e v e r i f i c a t i o n of t h e o u t p u t s of e x p e r t s a c t i v i t i e s t h a t could b e provided with mistakes. Moreover a n e x p e r t c a n c h a n g e h i s mind while analyzing t h e a n s w e r s of o t h e r e x p e r t s . The v e r i f i c a t i o n p r o c e d u r e should include possibilities to r e c o n s i d e r t h e initial assumptions c o n c e r n i n g a l t e r n a t i v e s , c r i t e r i a a n d cer- tainly numerical a n d q u a l i t a t i v e assignments t a k e n f r o m e x p e r t s .

The f i r s t problem t h a t a r i s e s i s how to c h o o s e t h e b e s t scale to g e t e x p e r t information. I t i s well known

[I, 21

t h a t to r e c i e v e r e l i a b l e estimations, i t i s n e c e s s a r y to p r e s e n t t h e scale t h a t i s formulated i n a h a b i t u a l f o r e x p e r t s manner. Usually a n e x p e r t i s a s k e d to d e t e r m i n e quantitatively t h e l e v e l of quality of a l t e r n a t i v e s . And t h e e x p e r t should assign t h e a c c o r d a n c e between t h e q u a n t a t i v e estimation and t h i s level. I t i s c l e a r t h a t t h i s a c c o r d a n c e i s d e t e r - mined d i f f e r e n t l y by d i f f e r e n t e x p e r t s . Such r e s u l t s o b t a i n e d in t h i s manner c a n h a v e valuable mistakes. I t i s b e t t e r if t h e scale i s v e r b a l ( f o r example "good",

"fair", "bad") b u t a g a i n t h i s estimation c a n b e d i f f e r e n t l y c o n n e c t e d with t h e nu- m e r i c a l merits. The s c a l e should b e flexible enough to t r y t h e d i f f e r e n t accor- d a n c e between v e r b a l conclusions a n d t h e s e merits.

Another problem i s providing n o n c o n t r a d i c t o r y a n d t r a n s i t i v e assignments.

(Noncontradictory assignments give t h e same estimations in t h e same conditions. Transitive assignments are s u b j e c t e d to t h e condition: if a

>

b a n d b

>

c t h a n a

>

c ) . B e f o r e formulating t h e decision r u l e one must b e s u r e t h a t at least 80

-

^90%of t h e assignments fulfill t h i s r e q u i r e m e n t s .

These preliminary c o n s i d e r a t i o n s r e l a t e s t r o n g l y to t h e problems of decision making in t h e framework of m u l t i c r i t e r i a i l l - s t r u c t u r e d problems. Human f a c t o r s influence s t r o n g l y t h e s u c c e s s of t h e assessment p r o c e d u r e of p r o b - lems, systems a n d s i t u a t i o n s . F o r t h e s e problems in which q u a l i t a t i v e , ill-defined f a c t o r s are dominant, t h e c h o s e n set of evaluation criteria i s o f t e n s u b j e c t i v e a n d r a t i n g s assigned by e x p e r t s to t h e given a l t e r n a t i v e s b y e a c h c r i t e r i o n c a n b e q u i t e d i f f e r e n t .

I t i s a l s o well known t h a t e x p e r t s c a n d e a l with n o more t h a n f i v e to s e v e n c r i t e r i a if we would like to h a v e r e l i a b l e r e s u l t s of t h e a s s e s s m e n t p r o c e d u r e . A t t h e same time t h e initial number of c r i t e r i a i s o f t e n much g r e a t e r . The possible

(8)

solution c a n b e to r e d u c e t h e i r number o n t h e b a s i s of t h e preliminary analysis o f t h e i r sameness a n d to g r o u p them.

Some human f a c t o r s r e l a t e d to t h e decision p r o c e s s are summerized in Table 1.

Methods of m u l t i c r i t e r i a decision making d i f f e r by t h e modes of forming t h e g e n e r a l i z e d e s t i m a t e s f o r e a c h a l t e r n a t i v e o n t h e b a s i s of d a t a e x t r a c t e d from e x - p e r t s . Let's d e s c r i b e some of them keeping in mind t h e i r potential usefulness f o r t h e problem of technology assessment.

Direct Methods

In t h e s e methods t h e r e l a t i o n between g e n e r a l i z e d e s t i m a t e s (utility functions) a n d e s t i m a t e s b y s e p a r a t e c r i t e r i a i s p r e d e f i n e d . In most cases gen- e r a l i z e d c r i t e r i o n p r e s e n t s a l i n e a r weighed combination of s e p a r a t e c r i t e r i a . These methods are d e s c r i b e d e l s e w h e r e (see, f o r example [4]. More so- phisticated methods u s e a s p i r a t i o n levels and t a k e i n t o a c c o u n t d i s a g r e e m e n t f a c t o r s [S].

Pairwise Comparison Methods

In t h e s e methods DM c h o o s e s between s e l e c t e d p a i r s of a l t e r n a t i v e s [6].

These methuds give as a r u l e r a t h e r r e l i a b l e solutions b u t are time consuming.

They are mostly used f o r t h e small-scale problems with few a l t e r n a t i v e s and c r i - t e r i a .

Compensation Methods

In t h e s e methods [7] e s t i m a t e s f o r o n e a l t e r n a t i v e are t r i e d to b e compen- s a t e d b y e s t i m a t e s f o r a n o t h e r o n e in o r d e r t o c h o o s e t h e b e t t e r one. These methods are c o n s i d e r e d to b e t h e m o s t user-friendly as at o n e time a n e x p e r t d e a l s only with p a i r s of a l t e r n a t i v e s . All shortcomings a n d a d v a n t a g e s f o r both a l t e r n a - tives are analysed and c r o s s e d o u t b y p a i r s to see what is l e f t at t h e e n d of t h i s p r o c e d u r e .

(9)

Table 1. Human factors r e l a t e d to t h e decision p r o c e s s . 1. Human c a p a c i t i e s in information processing are

r a t h e r limited b u t flexibility of humans, t h e i r adaptability and e x p e r i e n c e make i t possible t o r e l y on t h e i r e x p e r t estimations.

2. Human capabilities depend on t h e type of t h e

problem and on t h e way of obtaining t h e r e l e v a n t information from people.

3. Short-term memory c a p a c i t y is Limited. I t c a n p r o c e s s only s e v e r a l s t r u c t u r a l d a t a units.

4 . Man e i t h e r a d a p t s to a complex problem or tries to a d a p t i t t o h i s own capabilities.

Humans are usually a b l e to l e a r n f r o m previous a c t i o n s (mostly by try-and-see technique).

6. Solving unique problems often leads to conflicting

and differing a n s w e r s during t h e decision process.

i

^?- Human c a p a c i t i e s during t h e decision p r o c e s s

depend strongly o n t h e way t h e problem is formulated.

8. More adequate are methods of eliciting information f r o m humans t h a t use habitual qualitative s c a l e s but not numerical ones.

9. The complexity of t h e decision problem i n c r e a s e s with t h e number of c r i t e r i a , quantity of estimates on t h e c r i t e r i o n s c a l e a n d with t h e number of t h e resulting quality c l a s s e s .

P e r s o n a l , professional, national a n d o t h e r individual motivations influence strongly t h e assignments of e x p e r t s . 11. Interinfluence of opinions of e x p e r t s engaged in one

problem c a n lead t o changes in t h e i r initial assignments.

12. Humans make errors during t h e decision p r o c e s s due to inadequate understanding of t h e p a r t i c u l a r problems.

c a r e l e s s n e s s or o t h e r f a c t o r s .

1

^13. Human estimates c a n b e c o n t r a d i c t o r y and non-transitive.

1

^14. Humans p r e f e r t h e information to b e r e p r e s e n t e d more in images, g r a p h s than by t a b l e s with numbers.

Axiomatic Yethoda

In t h e s e methods [4] some c h a r a c t e r i s t i c f e a t u r e s of t h e utility function are postulated reflecting t h e p r e f e r e n c e s of DM. During t h e assessment p r o c e d u r e t h e s e p r e f e r e n c e s a r e verified and adjusted.

(10)

Interactive Methods

They are used e f f e c t i v e l y if t h e p a r t i a l model of t h e system i s known a n d p r e f e r e n c e s a n d r e l a t i o n s between d i f f e r e n t c r i t e r i a are a n a l p e d a n d i n t e r a c t i v e - ly modified [B, 9, 10 ].

I t should be noted t h a t p r a c t i c a l application of most of t h e a b o v e d e s c r i b e d methods f o r i l l - s t r u c t u r e d problems h a s r a t h e r n o t b e e n hopeful. One of t h e r e a s o n i s t h a t e x p e r t s c a n n o t a s s i g n r e l i a b l e numerical e s t i m a t e s ( r a t i n g s ) f o r a l t e r n a - t i v e s b y a lot of c r i t e r i a at o n c e without analysing t h e opinions of o t h e r ex- p e r t s a n d without some discussions.

Summerizing t h e b r i e f overview of t h e existing methods, having i n mind t o c h o o s e t h e b e s t f o r t h e problem of technology assessments, i t i s c l e a r t h a t to ob- t a i n r e l i a b l e r e s u l t s f o r a r e a s o n a b l e p e r i o d of time i t i s n e c e s s a r y to combine a d v a n t a g e s of d i f f e r e n t methods i n t o one p r o c e d u r e .

In o u r a p p r o a c h w e combined some elements of t h e d i r e c t method of con- s t r u c t i n g t h e g e n e r a l i z e d utility function as a combination of weighed r a t i n g s b y e a c h c r i t e r i o n f o r a l l a l t e r n a t i v e s with i n t e r a c t i v e computerized v e r i f i c a t i o n p r o c e d u r e . During t h i s p r o c e d u r e , initial assignments of e x p e r t s are a v e r a g e d . A s p e c i a l m e a s u r e

-

Mean S q u a r e Deviation

-

i s i n s e r t e d t o c l a r i f y t h e d i s a g r e e - ments between e x p e r t s . Pairwise comparison i s used f o r t h e v e r i f i c a t i o n of t h e initial e x p e r t assignments.

In t h i s p a p e r w e p r e s e n t t h e initial p r i n c i p l e s (Section 2), a s s e s s m e n t pro- c e d u r e (Section 3) and s t r u c t u r e of t h e system (Section 4 ) . Some programming as- p e c t s are d e s c r i b e d in S e c t i o n 5. TAS now i s being implemented f o r t h e assessment of e n e r g y technologies. H e r e we p r e s e n t I n t e r a c t i v e Information System f o r Tech- nology Assessments as a t o o l f o r providing full-scale information to t h e policy mak- e r to analyze t h e i n t e r n a t i o n a l e x p e r i e n c e in e n e r g y systems. I t should b e pointed o u t t h a t t h i s f i r s t v e r s i o n of TAS d o e s n o t claim

to

p r o v i d e him with t h e decision r u l e to c h o o s e t h e p a r t i c u l a r technology f o r h i s p u r p o s e s b u t m o r e to stimulate h i s decision p r o c e s s o n t h e b a s i s of v a r y i n g opinions, including national a n d p e r s o n a l motivations, d i s a g r e e m e n t f e a t u r e s a n d some a v e r a g e s . I t i s u p to the policy mak- e r to make a decision a f t e r analyzing t h e full set of information s t o r e d i n TAS.

Based on t h e e x p e r i e n c e of t h e case s t u d y o n e n e r g y technology assessments, i t i s planned, as a s e c o n d s t e p , to t u r n to formulating decision algorithms.

R e f e r i n g

to

t h e a b o v e mentioned difficulties to formulate t h e decision r u l e b a s e d o n e x p e r t opinions c o n c e r n i n g t h e f i n a l c h o i c e of technologies f o r a p a r t i c u l a r u s e r , i t becomes clear t h a t t h e problem should b e divided at least i n t o two s t a g e s .

(11)

The f i r s t v e r s i o n of t h e technology assessments system c a n b e c o n s t r u c t e d taking i n t o a c c o u n t human f a c t o r s of decision making a n d some preliminary assumptions a b o u t t h e p r o c e s s of calculating t h e o u t p u t merits.

2. MAIN PRINClPLES

OF TAS

The main p r i n c i p l e s of TAS are p r e s e n t e d in Table 2.

Table 2. Main p r i n c i p l e s of TAS.

1. Openess

TAS i s c o n s t r u c t e d of s e v e r a l u n i v e r s a l modules with a s t a n d a r d i n t e r f a c e . I t allows to a d d a n d modify t h e system f o r o t h e r a p p l i c a t i o n s of t h e s a m e kind.

2. Flexibility

I t i s r a t h e r simple to r e f o r m u l a t e t h e main problem, l i s t of technologies a n d cri- t e r i a , to c h a n g e weights of c r i t e r i a a n d t h e i r s c a l e , to r e f o r m t h e o u t p u t calculations, forms of information r e p r e s e n t a t i o n , etc.

3. User-friendliness

After t h e preliminary professional adjustments t h e system c a n b e used by a non-professional programming u s e r . I t includes a n h i e r a r c h i c a l HELP-system.

4. D a t a p r o c e s s i n g

The d a t a p r o c e s s o r c o n s i s t s of a number of small BASIC p r o g r a m s t h a t c a n b e e a s i l y e d i t e d f o r t h e p a r t i c u l a r u s e r .

5. G r a p h i c s

A s p e c i a l g r a p h i c s subsystem i s provided to show a n y kind of d a t a s t o r e d in In- t r o d u c t o r y , R e s u l t a n t a n d Verification Data Bases.

6. Modes of i n t e r p r e t a t i o n of t h e o u t p u t f i g u r e s

The c r i t e r i a s c a l e c o n s i s t s of s e v e r a l a n s w e r s l e v e l s (L): 0 - none, 1

-

^{b a d ,}2

-

p o o r , 3

-

f a i r , 4

-

^good,5

-

e x c e l l e n t . They c a n b e i n t e r p r e t e d in t w o modes.

A) Numerical in which e a c h l e v e l i s assigned a r a t i n g (R): R

=

N

*

^{L, where}

^N

^{i s a}

s c a l e c o e f f i c i e n t which c a n b e v a r i e d during t h e a n a l y s i s of t h e r e s u l t s .

B) Non-numerical in which p e r c e n t a g e of a l l answer levels in t h e o u t p u t d a t a i s c a l c u l a t e d .

7. Verification of assignments

A s p e c i a l susbsystem i s developed to v e r i f y t h e assignments of e x p e r t s by t h e pairwise comparison of t h o s e of a given e x p e r t , o t h e r e x p e r t s a n d a v e r a g e s . 8. Disagreements analysis

A d i s a g r e e m e n t f a c t o r i s i n t r o d u c e d as a mean s q u a r e deviation of assignments from a v e r a g e s to c h a r a c t e r i z e t h e d i f f e r e n c e i n opinions.

9. Modification of c r i t e r i a

A s p e c i a l p r o c e d u r e i s s u g g e s t e d to r e c o n s i d e r t h e l i s t of c r i t e r i a a n d to r e d u c e t h e i r number o n t h e b a s i s of t h e a n a l y s i s of t h e initial assignments a n d r e s u l t a n t

(12)

d a t a analysis.

10. E x p e r t s g r o u p analysis

In o r d e r to t a k e i n t o account t h e d i f f e r e n c e s between motivations of various ex- p e r t g r o u p s a s p e c i a l filtering subsystem c a n s e l e c t and show t h e asignments of d i f f e r e n t e x p e r t groups (country, specialization, etc .).

11. C r i t e r i a g r o u p analysis

A l l c r i t e r i a are grouped and a v e r a g e d output p a r a m e t e r s are calculated f o r e a c h g r o u p relying more on t h e enlarged estimates t h a n on detailed analysis of a l a r g e number of c r i t e r i a .

3. ASSESSMENT

_PROCEDURE

Taking into account t h e above said, t h e assessment p r o c e d u r e c a n b e divided i n t o d i f f e r e n t stages.

Choice ^ofAlternutivea

In m o s t cases altenatives t o b e assessed are specified by t h e g r o u p of cus- t o m e r s and DM who initiate t h e assessment p r o c e d u r e . Alternatives

-

e n e r g y technologies to b e assessed are listed in Table 3. Their illustrative definition i s p r e s e n t e d in Appendix 2.

Table 3. List of technologies

1. Lurgi P r e s s u r e Coal Gasification 2. Hydropymlysis f o r coal conversion

3. Coal conversion by s u p e r c r i t i c a l e x t r a c t i o n

4 . Combined c y c l e power s t a t i o n with integrated coal gasification 5. High t e m p e r a t u r e g a s cooled reactors

6. Gas t u r b i n e s

7 . 'SASOL1'-type coal lfquifaction plant

8. Low p r e s s u r e n a t u r a l g a s to methanol conversion 9. Geothermal e n e r g y

10. E l e c t r o t h e r m a l hydrogen

11. High efficiency membrane complex methane production 12. S u p e r h e a t pump e n e r g y accumulation

13. Fuel cell power plant

1 4 . Gasification in molten i r o n bath

(13)

I t should b e pointed o u t t h a t t h e a b o v e mentioned technologies a n d e n e r g y systems based o n t h e i r use h a v e a l r e a d y showed good o p e r a t i n g q u a l i t i e s (like t h e SASOL p l a n t , Lurgi P r e s s u r e Coal Gasification). So t h e i r main c a p a b i l i t i e s a n d f e a t u r e s are r a t h e r to b e a s s e s s e d by e x p e r t s . A t t h e same time some of t h e i r p e r f o r m a n c e i m p a c t s c a n n o t b e e a s i l y estimated by q u a n t a t i v e methods. Another f a c t o r t h a t influences t h e i r c h o i c e f o r t h e assessment i s t h e i r i n c r e a s i n g usage in d i f f e r e n t c o u n t r i e s , e a c h having i t s own e x p e r i e n c e , t r a d i t i o n s a n d conditions.

And t h e main aim of t h e assessment p r o c e d u r e i s to c l a r i f y t h e potential p r e f e r e n c e s f o r e a c h of t h e teachnologies in d i f f e r e n t c o u n t r i e s . IIASA seems t h e p r o p e r p l a c e f o r s u c h an i n t e r n a t i o n a l analysis of e n e r g y technologies.

C h o i c e o f the Set of C r i t e r i a

A s a r u l e t h e set of c r i t e r i a f o r t h e e x p e r t analysis i s c h o o s e n o n t h e b a s i s of t h e e x p e r i a n c e of DM engaged in t h i s a c t i v i t i e s a n d works of s c i e n t i s t s in t h e field. I t is n a t u r a l t h a t DM who Launched t h e assessment p r o c e d u r e f i r s t would l i k e to h a v e much more c r i t e r i a t h a n n e c e s s a r y

-

not to f o r g e t a n y of t h e a s p e c t s of t h e problem. I t o f t e n l e a d s to a r a t h e r big set of c r i t e r i a which c a n n o t b e handled p r o p e r l y by human e x p e r t s . S p e c i a l p r o c e d u r e s are a r r a n g e d to de- crease t h e initial number of c r i t e r i a to make t h e r e s u l t s m o r e r e l i a b l e a n d useful.

In o u r case 23 c r i t e r i a w e r e initially c h o s e n ( s e e Table 4).

Table 4. List of c r i t e r i a .

1. R&D Time Requirement 2. Costs of R&D

3. P r o b a b i l i t y of R&D S u c c e s s

4. Capability of I n d u s t r i a l M a n u f a c t u r e r 5. Availability of Material a n d R e s o u r c e s 6. Institutional B a r r i e r s

7. S o c i a l Acceptability 8. Level of Pollution 9. Flexibility of Siting

10. Waste Handling a n d Disposal 11. Availability of Cleaning 12. Consequences of Accident 13. S a f e t y C h a r a c t e r i s t i c s

14. Adaptability t o Types a n d S o u r c e s of Fuel 15. Outage R a t e

16. Risk of High Damage 17. Capital Cost

18. Construction P e r i o d

19. Efficiency of E n e r g y a n d R e s o u r c e s Utilization 20. Multiproducts Availability

(14)

21. 0 &

M

Requirements

22. Commercial Acceptability of P r o d u c t

23. Availability a n d P r i c e s of N a t u r a l R e s o u r c e s

These c r i t e r i a c h a r a c t e r i z e t h e possibilities f o r t h e s u c c e s s f u l development of e n e r g y technologies, i t s economic p r o p e r t i e s a n d f a c t o r s r e l a t e d to s a f e t y , flexibility a n d environmental consequences. T h e i r l i s t claims on comprising a l l sorts of p a r a m e t e r s n e c e s s a r y

to

e s t i m a t e t h e i r development.

The q u e s t i o n n a i r e c o n s i s t s of a number of t a b l e s , e a c h of them having dif- f e r e n t questions c o n c e r n i n g v a r i o u s a s p e c t s of a l l technologies u n d e r c o n s i d e r a - tion, weights of t h e e v a l u a t i o n c r i t e r i a , a n d r a t i n g s f o r e a c h technology a n d c r i t e r i o n .

As mentioned a b o v e , t h e t r a d i t i o n a l economic a p p r o a c h i s useful f o r well- developed technologies, where t h e quality of input d a t a i s good. F o r new technolo- g i e s , especially at t h e s t a g e of r e s e a r c h development, t h e r e are many i s s u e s beyond t h e question of c a p i t a l a n d o p e r a t i n g c o s t s . Among them are s o c i a l a c c e p - tability, level of pollution, availability of n e c e s s a r y m a t e r i a l s a n d r e s o u r c e s , cost a n d time r e q u i r e d f o r R&D, a n d many o t h e r s .

In o r d e r to h a v e b e t t e r measurements f o r t h e a s s e s s m e n t of new e n e r g y technologies a t IIASA, a method, b a s e d o n t h e e x p e r t ' s a n a l y s i s of t h e many c r i t e r i a , w a s developed by many a u t h o r s of v a r i o u s c o u n t r i e s . Two v e r y similar a p p r o a c h e s w e r e developed by t h e Western IES Consortium [ll] a n d by Russian s c i e n t i s t s [lZ, 131. A set of evaluation c r i t e r i a w a s p r o p o s e d , including 2 3 v a r i a b l e s c o n c e r n e d with major f a c t o r s of technology development a n d implementation. All c r i t e r i a are divided i n t o f o u r g r o u p s (Figure 1 ) :

G r o u p I c o n t a i n s c r i t e r i a to assess possibilities of t h e s u c c e s s f u l development of a technology up to t h e s t a g e of a pilot i n d u s t r i a l plant.

Among t h e c r i t e r i a are two more g e n e r a l t h a n t h e o t h e r s in t h e g r o u p . They a p p l y to a p p l i c a t i o n s of t h e technology of s c a l e . G r o u p I1 includes environmental consequences of t h e technology a s s e s s e d

and possibilities of management of t h e environmental e f f e c t .

G r o u p I11 is dealing with m a t t e r s of s a f e t y , r e l i a b i l i t y a n d technological flexibility of a technology.

Group IV includes c h a r a c t e r i s t i c s which are needed to assess economic p r o - p e r t i e s of a technology a n d t h e e x p e c t e d economical s i t u a t i o n when t h e technology i s implemented.

(15)

Assignment of C r i t e r i a Weighb

The weights f o r t h e given c r i t e r i a c a n b e o b t a i n e d f r o m t h e e x p e r t s or b y means of s p e c i a l mathematical p r o c e d u r e s [1,2]. W e c o n s i d e r t h e weights f o r t h e c r i t e r i a e q u a l f o r a l l technologies b e c a u s e t h e y are d e p e n d e n t mainly o n political, s o c i a l , economic, a n d o t h e r conditions a n d not on t h e p a r t i c u l a r t y p e of technolo- BY-

M g n m e n t of R a t i n g s

S e v e r a l technologies are usually s e l e c t e d a n d b r i e f e d i n t h e q u e s t i o n n a i r e to- g e t h e r with t h e c r i t e r i a c h o s e n . F i r s t , e a c h e x p e r t must e s t i m a t e t h e weights of t h e c r i t e r i a a n d t h e n p u t down t h e r a t i n g s f o r e a c h technology b y e a c h c r i t e r i o n . These r a t i n g s are divided i n t o five levels. E a c h l e v e l c a n b e r e p r e s e n t e d in t w o d i f f e r e n t forms: v e r b a l conclusions ( f o r example, e x c e l l e n t , good, f a i r , p o o r , b a d ) a n d numerical v a l u e s ( f o r example: 5, 4 , 3, 2, 1

-

f i v e being t h e h i g h e s t r a n k i n g ) . All t h e r a t i n g s are multiplied by c o r r e s p o n d i n g v a l u e s of weighing c o e f f i c i e n t s to form a score f o r e a c h technology b y e a c h c r i t e r i o n a n d t h e total (sum) score of e a c h technology. F o r d i f f e r e n t p u r p o s e s t h e decision m a k e r c a n h a v e e i . t h e r t h e r e s u l t a n t p e r c e n t a g e of v a r i o u s l e v e l s of v e r b a l conclusions or numerical estimations of means a n d o t h e r s t a t i s t i c a l v a l u e s of r e s u l t a n t p a r a m e t e r s f o r e a c h tech- nolog y

.

O u t p u t Figures

E a c h technology j is e v a l u a t e d by e x p e r t k with c r i t e r i o n

C,,

w h e r e j

=

1 ,... m, k = 1,

...

L , i = ¹

,...

n. E a c h c r i t e r i o n h a s i t s own weight c o e f f i c i e n t W1 assigned by e a c h e x p e r t . On t h e b a s i s of t h e s e estimations some c h a r a c t e r i s t i c o u t p u t v a l u e s are c a l c u l a t e d .

Score SIJk i s c a l c u l a t e d as

w h e r e Rllk i s t h e r a t i n g f o r jth technology a n d ith c r i t e r i o n a s s i g n e d by kth e x p e r t

Average weight c o e f f i c i e n t s f o r c r i t e r i o n : C w , k k

AW,

= -

L

a n d mean s q u a r e d e v i a t i o n s of weight coefficients:

(16)

MSDA, =

L

The a v e r a g e score a n d mean s q u a r e deviation are c a l c u l a t e d f o r e a c h t e c h - nology a n d e a c h c r i t e r i o n :

A s e p a r a t e t a b l e p r e s e n t s t h e t o t a l scores f o r e a c h technology e v a l u a t e d by e a c h e x p e r t :

A final t a b l e c o n t a i n s i n t e g r a t e d e s t i m a t e s of a l l t h e technologies 1, by a v e r a g i n g t h e t o t a l s c o r e s f o r e a c h technology assigned by e a c h e x p e r t :

Deviation of e x p e r t s opinions are estimated by :

Denote r

-

t h e index of a c r i t e r i a g r o u p : r

=

l,..s, where s

-

number of g r o u p s (in o u r case s

=

4).

The a v e r a g e s c o r e a n d MSD are c a l c u l a t e d f o r e a c h e x p e r t a n d e a c h technolo- g y f o r e a c h c r i t e r i a g r o u p

w h e r e i,is a number of c r i t e r i a i n e a c h g r o u p (xi,=i).

(17)

All t h e o u t p u t p a r a m e t e r s d e s c r i b e d a b o v e form t h e Numerical Data Base as a number of t a b l e s .

V e r i f i c a t i o n P r o c e d u r e

Based on t h e r e s u l t s of initial e x p e r t assignments t h i s p r o c e d u r e includes t h e d e t a i l e d analy-sis of a l l c r i t e r i a divided i n t o d i f f e r e n t g r o u p s - to c l e r i f y t h e i r c o n t r a d i c t i v e n e s s a n d sameness. This will make i t possible to d e c r e a s e t h e i r number a n d to l e a v e t h o s e t h a t are most i m p o r t a n t f o r W e c o n c r e t e assessment p r o c e d u r e . Afterwards a l l e x p e r t s c a n o b s e r v e t h e obtained r e s u l t s a n d c o m p a r e t h e i r e s t i m a t e s with a v e r a g e values taking i n t o a c c o u n t t h e d i s a g r e e m e n t f a c t o r s (MSD). I t will a l l o w to modify t h e i r initial assignments or

-

if t h e y d o not a g r e e with o t h e r opinions

-

to comment t h e i r decisions.

A t t h e final s t a g e a l l information beginning with t h e initial o u t p u t d a t a to t h e v e r i f i e d o n e i s p r e s e n t e d f o r a l l p a r t i c i p a n t s with a l l comments a n d g r a p h i c a l im- a g e s . I t will allow n o t only to h a v e a v a r a g e d a b s t r a c t i v e r e s u l t s b u t to d e s c r i b e d i f f e r e n c e s in opinions b a s e d on national, p r o f e s s i o n a l and o t h e r f a c t o r s . In t h i s case t h e r e s u l t s of t h e a s s e s s m e n t p r o c e d u r e c a n b e used by d i f f e r e n t national and s o c i a l g r o u p s a n d a l l f o r e c a s t s will b e more r e l i a b l e .

C r i t e r i a Modification

I t i s well known t h a t if a n e x p e r t i s t o d e a l with a lot of c r i t e r i a h i s estimations are n o t r e l i a b l e ( s e e Table 1 ) . That's why a s p e c i a l p r o c e d u r e i s implemented to r e d u c e t h e i r initial number by grouping them.

F o r t h i s p u r p o s e a l l c a l c u l a t e d scores f o r c r i t e r i a g r o u p s (based o n assignments f o r t h e full set of c r i t e r i a ) are compared with t h e assignments made f o r t h e c r i t e r i a g r o u p s ( s e e Appendix 1 ) . Verifying t h e s e t w o r e s u l t s will make i t possible to u s e only g r o u p c r i t e r i a assignments in t h e f u t u r e .

4.

GENERAL

STRUCTURE OF

TAS

The g e n e r a l s t r u c t u r e of TAS is shown in F i g u r e 2. I t c o n s i s t s of t h e In- t r o d u c t o r y Data B a s e , which stores a l l t h e information t a k e n by t h e question- n a i r e f r o m t h e e x p e r t s , t h e I m p o r t p r o g r a m , which b r i n g s t h i s information to t h e Data P r o c e s s o r , The Data P r o c e s s o r , including d i f f e r e n t f i l t e r s a n d a n a l y z e r s to make a l l d a t a t r a n s f o r m a t i o n s r e p r e s e n t i n g i t in t h e most convenient way, a n d t h e E z p o r t p r o g r a m , which p u t s t h e p r o c e s s e d d a t a i n t o t h e R e s u L t a n t D a t a Base.

(18)

TAS h a s a h i e r a r c h i c a l menu system. When t h e u s e r e n t e r s TAS h e watches t h e Main Menu o n the s c r e e n (Figure 3) with a l l n e c e s s a r y positions beginning with t h e i n t r o d u c t i o n of e x p e r t s a n d t h e c h o i c e of t h e c r i t e r i a u p to some editing positions

-

to a d j u s t TAS p a c k a g e s f o r t h e n e e d s of t h e p a r t i c u l a r u s e r .

F i r s t position of t h e Main Menu is to e n t e r E x p e r t s D a t a Base (Figure 4 )

-

to i n t r o d u c e or c h a n g e a l l t h e information a b o u t t h e e x p e r t s of t h e a s s e s s m e n t p r o c e d u r e . The n e x t s t e p i s to assign weights f o r t h e given c r i t e r i a . When o n e e n t e r s t h e a p p r o p r i a t e position of t h e Main Menu h e finds t h e Submenu t h a t allows to formulate t h e l i s t of c r i t e r i a . Afterwards a s p e c i a l window a p p e a r s i n which e a c h e x p e r t c a n manipulate t h e values of t h e c r i t e r i a weights while t h e i r normalization (by t h e r u l e t h a t t h e i r sum i s equal to 1 0 0 ) i s being done au- tomatically.

S p e c i a l e x p o r t p r o c e d u r e i n t r o d u c e s information a b o u t e x p e r t s a n d c h o s e n values of t h e c r i t e r i a weightv to t h e I n t r o d u c t o r y Data Base (IDB) (Figure 5). A f t e r t h e a n a l y s i s of t h e q u e s t i o n n a i r e r a t i n g s f o r e a c h e x p e r t , technology a n d c r i t e r i o n are i n t r o d u c e d to IDB to s e r v e as a b a s i s f o r f u r t h e r calculations. Or i t c a n b e done d i r e c t l y i n IDB.

The n e x t t w o positions of t h e Main Menu allow to p r o c e s s a l l i n t r o d u c t o r y d a t a a n d to e x p o r t t h e o u t p u t d a t a to t h e Resultant Data Bases (Numerical and Non- numerical).

When t h e u s e r e n t e r s t h e Numerical R e s u l t a n t Data Base h e c a n see t h e NRDB Submenu a n d c a n o b s e r v e a l l t h e o u t p u t f i g u r e s o n t h e s c r e e n as t a b l e s or g r a p h s of d i f f e r e n t kinds a n d c a n h a v e them p r i n t e d o u t (Figures 6-13). This b a s e includes a s e p a r a t e f r a m e f o r Averages a n d MSD of weights a n d scores f o r e a c h technology, a f r a m e with c a l c u l a t e d scores f o r g r o u p s of c r i t e r i a a n d a f r a m e with i n t e g r a t e d r e s u l t s f o r e a c h of t h e e x p e r t s a n d technologies i n t e g r a t e d euti- mates a v e r a g e d by a l l e x p e r t s .

The Non-numerical Resultant Data Base c o n s i s t s of a number of f r a m e s . E a c h f r a m e r e p r e s e n t s t h e p e r c e n t a g e of d i f f e r e n t a n s w e r l e v e l s f o r e a c h technology (Figure 14).

E n t e r i n g t h e 'Verification' position of t h e Main Menu a f t e r pointing t h e technology-number a n d e x p e r t - n u m b e r f o r t h e comparison of a v e r a g e d o u t p u t m e r i t s , a l l t h e a p p r o p r i a t e information i s t a k e n from IDB a n d NRDB a n d i n t r o d u c e d to t h e Verification D a t a Base 1 (VDB1) (Figure 1 5 ) .

(19)

Taking i n t o a c c o u n t t h e problems of t h e use of a big ammount c r i t e r i a t h a t c o n t r a d i c t s sometimes with human e x p e r t f a c t o r s a n o t h e r v e r i f i c a t i o n p r o c e d u r e w a s s u g g e s t e d to d e a l with t h e o u t p u t s connected with t h e c r i t e r i a g r o u p s . To c h e c k t h e r e l i a b i l i t y of t h e initial. assignments with e a c h of 23 c r i t e r i a , a s e p a r a t e position i s provided in t h e q u e s t i o n n a i r e in which t h e e x p e r t s are to assign weights a n d r a t i n g s f o r e a c h technology by t h e f o u r mentioned c r i t e r i a g r o u p s . This information i s c o m p a r e d with o u t p u t g r o u p scores c a l c u l a t e d o n t h e b a s i s of initial scores f o r t h e 23 c r i t e r i a (in VDB2).

5.

SOME

PROG-G ASPECTS

TAS i s based o n d i f f e r e n t main modules t h a t were i n t e g r a t e d to s o l v e technology a s s e s s m e n t problems. Some of t h e modules w e r e worked o u t i n t h e Com- puting C e n t r e of t h e USSR Academy of Sciences. They include SPECTR

-

a d a t a o r i e n t e d b a s e system by which a l l mentioned d a t a b a s e s w e r e built; SPOUT a n d SPIN

-

p r o g r a m s f o r importing a n d e x p o r t i n g f i l e s f r o m / t o d a t a b a s e s to/from calculation a n d analysing p r o g r a m s ; p r o g r a m s t h a t are i n t e g r a t e d in Data Processor and s e r v e as means f o r t h e a p p r o p r i a t e d a t a transforma- tion; FILTERS

-

^tomake national a n d p r o f e s s i o n a l samples from a l l f r a m e s of t h e Resultant Data Base; a n d LEXICON

-

f i l e s e d i t o r . Some additional p a c k a g e s (DG a n d CHART) are used to r e p r e s e n t information o n t h e s c r e e n (as g r a p h i c s and plots) a n d to h a v e i t p r i n t e d . All menus of TAS w e r e built using module DLG t h a t p r o v i d e s e a s y modification of menu positions a n d i s b a s e d o n t h e c a l l of DOS e x e c u t a b l e p a c k a g e s (position Dialogue S c e n a r i o in t h e Main Menu c a l l s e d i t o r in which a l l menus c a n b e changed if n e c e s s a r y ) . File TAS.DOC c o n t a i n s t h e full d e s c r i p t i o n of TAS. One c a n a l s o g e t some i n s t r u c t i o n s f o r using TAS by means of HELP f a c i l i t i e s a n d file TAS.CTL.

P r o g r a m s f o r Data Processor are w r i t t e n o n BASIC. T h e r e f o r e , t h e y c a n b e e a s i l y modified by t h e u s e r f o r h i s own p u r p o s e s a n d f o r t h e given s t r u c t u r e s of t h e R e s u l t a n t Data Bases. S e p a r a t e position of t h e Main Menu allows to e n t e r GWBASIC e d i t o r .

TAS i s b a s e d on t h e use of IBM-type p e r s o n a l c o m p u t e r s with t h e h a r d disk c o l o u r display a n d RAM n o l e s s t h a n 512k. I t i s provided with developed HELP fa- c i l i t i e s a n d c a n b e used e v e n by non-specialists in c o m p u t e r programming.

In case of n e c e s s a r y modifications of t h e forms of d a t a b a s e s , f i l t e r s , o r some o t h e r s u p p l e m e n t a r y p r o g r a m s i n TAS, consultations of professional p r o - g r a m m e r s will p r o b a b l y be neadad to h e l p t h e u s e r while t h e normal o p e r a t i o n of TAS i s a r a t h e r simple p r o c e d u r e .

(20)

Supplied with the installation procedure TAS requires no l e s s than 3Mbt of the hard disk space.

(21)

REFERENCES

1. L a r i c h e v O.I., Zuev Y .A., Gnedenko L.S. Method Z a p r o s to s o l v e i l l - s t r u c t u r e d c h o i c e problems upon many c r i t e r i a . M o s c o w , VNIISI, 1979.

2. V.Yadov. Sociological r e s e a r c h . Moscow, Nauka, 1972.

3. B e r r y D.C, B r o a d b e n t D.E. E x p e r t systems a n d t h e man-machine i n t e r f a c e . Part 2: The u s e r i n t e r f a c e . E x p e r t systems 1987, v.4, No.1.

4. Keeney R., Raiffa H. Decision with multiple objectives: p r e f e r e n c e s a n d t r a d e - offs. N.Y., Wiley, 1976.

5. Lewandowski A., Johnson S., Wierzbicki A. A p r o t o t y p e s e l e c t i o n committee decision a n a l y s i s a n d s u p p o r t system, SCDAS: T h e o r e t i c a l b a c k g r o u n d a n d com- p u t e r implementation. P r e p r i n t s of 7-th I n t e r n a t i o n a l C o n f e r e n c e o n Multicri- t e r i a Decision Making. Kyoto, J a p a n , August 18-22, 1986.

6 . Zionts S., Wallenius J . R e c e n t developments i n our a p p r o a c h to multicri- t e r i a decision making. In; I n t e r a c t i v e Decision A n a l p i s . Springer-Verlag , 1984.

7. Mac Crimon

K.R.,

Sin I.K. Making trade-off. Decision S c i e n c e s , 1974, n.5.

8. A.K.Alabyan, V.M. Afanasiev. Feasible Domain method a n d i n t e r a c t i v e system f o r t h e design a n d c o n t r o l of dynamic m u l t i c r i t e r i a systems. P r e p r i n t s of 7-th I n t e r n a t i o n a l C o n f e r e n c e o n Multicriteria Decision Making. Kyoto, J a p a n , August 18

-

22, 1986.

9 . H.Nakayama, Y .Sawaragi

.

Satisficing trade-off method f o r multiob jective programming. Jn : " I n t e r a c t i v e Decision Analysis.

"

Springer-Verlag, Berlin, 1984.

1 0 . M.Grauer. A.Lewandowski, A. Wierzbicki. DIDASS

-

t h e o r y , implementation a n d e x p e r i e n c e . In : "Interactive Decision Analysis". S p r i n g e r - V e r l a g , Berlin, 1984.

1.1. k r s e s s m e n t s a n d measurements f o r e n e r g y c o n v e r s i o n systemr. IES Consor- tium P a p e r , Sweden, S e p t e m b e r 1986.

1 2 . Okorokov V.R. Management of t h e e l e c t r i c a l power Systems, Leningrad State Univ., 1976.

1 3 . Okorokov V.R., Shavelev, e d s . Complex a n a l y s i s of t h e e f f e c t i v e n e s s of techn- i c a l decisions i n e n e r g e t i c s . Leningrad, E n e r g y , 1985.

(22)

EVALUATION CRITERIA

F i g u r e 1. Groups o f C r i t e r i a .

I M E R A C T M SYSTEM FOR TECHNOLOGY ASSESSMENTS GROUP 1

Succe8dul development

1

TECHNOLOGIES SPECIFICATION GROUP 2

Environmental contequances

F i g u r e

2.

S t r u c t u r e o f TAS.

-

E X E M S CRITERIA FORMULATION GROUP 3

Wdy, rellablllty and flexlblllty

,

GROUP 4 Economlc

propertlet

I 1

RATINGS ASSIGNMENT WEIGHTS ASSIGNMENT

4 -

INTRODUCTORY DATA BASE

A

I

DATA PROCESSING

CRITERIA MODIFICATION

1,

-

VERIFICATION

b

DATA BASE

I

^RESULTANTDATA BASES

1 1'

(23)

I I

I E X P E R T S I N T R O D U C T I O N I

I W E I G H T S ASSIGNMENT I

I

INTRODUCTORY D A T A BASE ( I D B ) 1 I N U M E R I C A L D A T A PROCESSING

I

INON-NUMEHICRL DATA PROCESSING I

l R E S U L T A N T DATA B A S E ( N u m e r i c a l ) I l R E S U L T A N T D A T A B A S E ( N o n - n u m e r i c a l ) I

I V E R I F I C A T I O N I

I D I A L O G U E SCENAR I

O

I

I D A T A PROCESSOR E D I T I N G I

I I

F i g u r e 3. Main Menu.

(24)

(25)

. - - -

t..j r.4 1.3

* *

^I..:#^e^{M t.:!}

*

^I..:^P"j^!..:I ^M^piP<I t..:~ I..:!

. - - -

b-l e ti3 ti3 e

*

^e^P.7 ^{iJ3 b3} ^e^{ti3 ti3}^{e e}^L3^{ti? ti3}

. - - -

e e e Pi I.? I..> Pi ct ct P:I

*

^Pi

*

^{P I}^ctI.-:! I..? e

. - - -

e

ti3

*

^ti3^e

*

b3 ti3 b3 e ti3 e e e I.? ti3 ti3

*

. - - -

p:1 e ?..I

*

^e^-^I

* *

^e^p:¹

*

¹^p^: ^{I.? Xl}^r.i

* *

^I..:!

. - - -

Kl r.1 t..:i I.:l

*

^pi

* * * * *

^:1^t. ^t..:1

*

^!..:I I..:! I..? ¹^:^:^r

(26)

- - -

b I::! 4 b [E [E Q . - I O

. . .

.-I 8 1.7

- - -

[E +

*

^b^b

[ E b - +

. . .

.-I 4 Fl

- - -

ti3 li3 Li3

. .

r.4

.

Q r4 '3

.-I .-I

- - -

L3 M

b b

00 b3 M

;c-';

.-I

- - -

in L 8 Z

. 4 4 L .4 f L - C rn ^{. 4 .4}

m n c l

It 3

4J

-

. ~t a-

c a r 0 0 U i L

.4 U

< ^c

^Y

J 0

4J A .4 6 4

. 4 (U

c ^U>

C 0 (U

u m - l

- - -

4 b 00

- - -

. 4 i- G

L E

a i o ^It

$ 4

a

L!

. Z f

L 4 J

.

^.4⁴ill ^.4Oi f

-

^{6 1}

U ic:

n Y

Z . ill 0 in

ill

9 /

U L

i

^i~

H I

(27)

(28)

(29)

t..:1 P4

-

iD il3 t:i

-

!?

I' I

t..:1

- g

'J I.':!

-

b3 I3 N

-

b3

r.4

I..?

-

t..:1 t-.:1 t..:1

-

'-' a

9 *:-I

8 .

-

&

:

; J

-

0' d- 1:)

-

i..; d- 1.q

-

d- l..?

M I

# ..

-

a 'L'

t::i

!i?

-

z

W

- a

LC LC 3

-

>

0 4'

0 L C

4'

0

-

4

-

o L >

L O O

o n 4

- G O 0

+

⁰

(30)

(31)

(32)

a=, 4 r.4 I.?!

*

+ ~ 4 t - ? e 1 1 ? ~ b m @ + + 4 + +

I l l l l l l i l l t i l l

I-!-!-!-!-!-!-+!-!-!-!-!-!-

(33)

. . .

-

8-8 ,=, '2 ,2; *=a :2 '='

,>

,=, 2, :=, 2, sz. ,=, '3

:>

c.4 ^.A^;! :'.i r.4 T.4 7.; r.4 ^i'.;^.Fi7.: t:;1

r.4

^r.71r., t:;1 r.4 ⁴

+ *3 a c r a

. m M .,+ &j U)

5 C L L Us.d C 0 U in

+'

m

n

E 2

> c a

I t . d . 6 a!

(34)

- - -

n ^{I J ~}⁹³r-4 e ^~3 ^KIe ^A ^~3r.4 m ⁱ¹³^pi^ti;

b b 4 i e b b e ^"3b r.1

* ^q

^C.4 ^b3^Nr.4

ICI t..:~

. . .

-Ir.4

*

^m^KI^M

*

^b^{I J ~} ⁹^H. ^~4 e ^{IJT N}⁴

@ N e @ 6 0 P f f i b 7 4 P 0 m b a b + @ 0

A 4 A pi 4 -I r.4 -I

.-I

1.3 ;.'>

- - -

1.7 I;,I::# r.4 i.3 p:{

*

^b⁹^a^N ^{,=, q}r.: '.! 4 : f: 4

A A

m D

O O f 4 c a c c ~ i '