W O R K I N G P A P E R
I N T E R A C T I V E INFOXIilATION S Y S T E M F O R TECHNOLOGY A S S E S S M E N T S
A . K . A l a b y a n A . P . G o l o v i n e V . R . O k o r o k o v V . P o n o m a r e v
N o v e m b e r 1 9 8 7 W P - 8 7 - 1 2 1
I n t e r n a t i o n a l I n s t i t u t e for Applied Systems Analysis
I - N T l m A m INmRMATION
ETrsrm FOR
TM=HNOIXH;Y ASSESSMENTSA.K. ALabyan A.F. Golovine
VR.
Okorokou 1.I PonomareuNovember
1 9 8 7WP-87-121
Working P a p e r s are interim r e p o r t s on work of the International Institute f o r Applied Systems Analysis and have r e c e i v e d only limited review. Views or opinions e x p r e s s e d h e r e i n d o not necessarily r e p r e s e n t those of the Institute or of i t s National Member Organizations.
INTERNATIONAL INSTITUTE
FOR
APPLIED SYSTEMS ANALYSIS A-2361 Laxenburg, AustriaEvaluation and assessment of new technologies i s a r a t h e r complicated t a s k due to t h e involvement of g r o u p s of e x p e r t s , multiple c r i t e r i a c h a r a c t e r i z i n g s e v e r a l al- t e r n a t i v e s as well as incomplete information a b o u t t h e s e a l t e r n a t i v e s . E x p e r t analysis of new technologies by d i f f e r e n t a s p e c t s c a n b e one of t h e wayx of estimat- ing t h e advantages a n d shortcomings of e a c h of them a n d of forecasting t h e i r development and usage.
Due to t h e c h a r a c t e r of t h e assessment p r o c e d u r e , especially in t h e g r o u p ex- p e r t situation, l a r g e amounts of information must b e processed and analyzed in o r d e r to find t h e final conclusion. Additionally, s e v e r a l f a c t o r s reflecting t h e quality of t h e r e s u l t s , quality of e x p e r t s opinions, etc. must b e calculated during t h e assessment p r o c e s s . T h e r e f o r e , this task should b e s u p p o r t e d by some comput- er based tools. The p a p e r p r e s e n t s s u c h a n information management system sup- porting t h e p r o c e s s of technology assessment. The system perfarms such functions like information collection a n d s t o r a g e , i n t e r a c t i o n with e x p e r t s a n d analysts, aggregation of information, g r a p h i c presentation of d a t a and r e s u l t s as well as computes s e v e r a l s t a t i s t i c a l f a c t o r s n e c e s s a r y to analyze t h e d a t a submitted by e x p e r t s . The system, being t h e f i r s t s t e p towards development of more advanced decision s u p p o r t systems h a s been applied at IIASA f o r analysis of s e v e r a l techno- logies f o r e n e r g y production.
Alexander B. Kurzhanski Chairman
System and Decision Sciences Frograrn
Content
INTRODUCTION
MAIN P R I N C I P L E S OF TAS ASSESSMENT PROCEDURE GENERAL STRUCTURE OF TAS SOME PRELIMINARY ASPECTS REFERENCES
I N T E ' R A m INRIRMATION
!3WlXMFOR TECHNOLOGY ASSESS-
A.K. ALabyan, A.P. Golovine, J!R. Okmrokou and I.! P o n o m a r e u
1.
INTRODUCTIONT h e r e are s e v e r a l ways t o d e a l with t h e problems of technology assess- ments. The t r a d i t i o n a l o n e s use d i f f e r e n t s o r t s of economic analysis a n d are well known. The t r a d i t i o n a l a p p r o a c h i s useful f o r well-defined technologies, f o r which t h e r e l i a b i l i t y of t h e input d a t a i s highly verified. When we d e a l with some new technologies i t i s no l o n g e r t h e c a s e . T h e r e are many i s s u e s beyond t h e economic f a c t o r s s u c h as s a f e t y ,
R&D
problems, environmental a n d s o c i a l impacts and o t h e r s t h a t are of g r e a t i m p o r t a n c e b u t sometimes c a n n o t b e e v a l u a t e d nu- merically with economic calculations. While assessing t h e f u t u r e develop- ment of new technologies, i t i s possible to r e d u c e u n c e r t a i n t i e s relying on t h e opinions of e x p e r t s . E x p e r t analysis of new technologies by d i f f e r e n t a s p e c t s c a n b e o n e of t h e ways of estimating t h e a d v a n t a g e s a n d shortcomings of e a c h of them a n d of f o r e c a s t i n g t h e i r f u t u r e development a n d usage.The method of a n e x p e r t computerized a n a l y s i s of technologies developed at IIASA a n d d e s c r i b e d h e r e i s based on a q u e s t i o n n a i r e (an example of t h e q u e s t i o n n a i r e t o b e filled o u t by o n e of t h e e x p e r t s i s p r e s e n t e d in Appendix I ) , surnmerizing a l l n e c e s s a r y information t a k e n f r o m t h e e x p e r t s a n d a n i n t e r a c - t i v e c o m p u t e r system t h a t makes a l l calculations, d a t a p r o c e s s i n g , and r e p r e s e n t a t i o n s .
This f i r s t v e r s i o n of technology assessments (TAS) d e s c r i b e s an information system f o r t h e policy m a k e r s as a tool f o r t h e full-scale analysis of d i f f e r e n t t e c h - nologies.
Before d e s c r i b i n g t h e p r o p o s e d p r o c e d u r e of technology a s s e s s m e n t s some preliminary c o n s i d e r a t i o n s should b e made c o n c e r n i n g t h e problem of t h e human possibiUties to make estimates. Some assumptions c o n c e r n i n g t h e models of human information processing a n d decision making could b e found in t h e works devoted to sociological r e s e a r c h a n d in modern t r e n d s in e x p e r t systems design (see, f o r example 11, 2, 31).
R e s e a r c h e r s engaged in measurement a n d mathematical modelling of hu- man phenomena meet t h e problem to s u b j e c t human b e h a v i o u r t o numerical analysis. T h e r e i s s t r o n g c r i t i c i s m now t h a t e s s e n t i a l individual c h a r a c t e r i s t i c s are swallowed u p in t h e sameness of quantity. Indeed to a c e r t a i n e x t e n t a same- n e s s i s a s s e r t e d when applying measurment to human phenomena. However, t w o points should b e r e c a l l e d . F i r s t , measuring c e r t a i n p r o p e r t i e s c o n n e c t e d with human a c t i v i t i e s n e e d n o t imply t h a t t w o c a s e s , when t h e s e p r o p e r t i e s are identical, c a n n o t d i f f e r in many o t h e r r e s p e c t s . Indeed o n c e t h e s e similarities are known i t may b e e a s i e r to c o n c e n t r a t e on t h e d i f f e r e n c e s between individuals.
Second, some s c a l e s of measurement are more r e s t r i c t i v e t h a n o t h e r s . The identification of o b j e c t s by c a t e g o r i e s i n t o which t h e y f i t , or by r a n k s , c a p t u r e s some q u a l i t a t i v e similarities. A t t h e same time f e w e r presumptions are made a b o u t t h e i r s a m e n e s s , as i s t h e case when t h e y p o s s e s s i d e n t i c a l values on a m e t r i c s c a l e . Notwithstanding t h i s c r i t i c i s m pointing to the limitations of measurment, however, t h e r e i s a n i n c r e a s i n g recognition t h a t a qualitative a p p r o a c h need n o t eschew measurment.
In r e c e n t y e a r s s o c i a l s c i e n t i s t s h a v e b e e n m o r e a n d more c o n c e r n e d with measuring q u a l i t i e s in o r d e r to g r a p p l e with complex c o n f i g u r a t i o n s a n d un- c e r t a i n t i e s i n h e r e n t in human p e r c e p t i o n a n d b e h a v i o u r . The difficulties assosi- a t e d with measuring a n d numerical a n a l y s i s of human a c t i v i t i e s remain im- mense. Techniques of q u a l i t a t i v e d a t a analysis are e s s e n t i a l in a n y e f f o r t to i n c o r p o r a t e non-numerical information e x t r a c t e d from humans. But i t i s n e c e s s a r y sometimes to a c h i e v e e v e n more: to g e t some numerical c h a r a c t e r i s t i c s as t o hu- man estimations of some p r o c e s s e s or systems p e r f o r m a n c e .
A major f a c t o r which c a n a f f e c t t h e ease with which p e o p l e u s e a n e x p e r t system i s t h e a b i l i t y of t h e system to t a i l o r i t s b e h a v i o u r to t h e s p e c i f i c f e a t u r e s a n d needs of a n individual u s e r . This i s most d e s i r a b l e where o n e p a r t i c u l a r sys- tem i s to b e used by p e o p l e with s u b s t a n t i a l l y differing backgrounds. To b e effec- t i v e , s y s t e m s should know who t h e i r u s e r s are a n d t h e c o n t e x t in which t h e y are t r y i n g to work. T h e r e are s e v e r a l ways in which a system c a n t a i l o r i t s b e h a v i o u r towards d i f f e r e n t u s e r s . The most simple i s w h e r e t h e u s e r i s a s k e d t o classify himself at t h e beginning of i n t e r a c t i o n as belonging t o a c e r t a i n c a t e g o r y . In more s o p h i s t i c a t e d a p p r o a c h s a c l u s t e r a n a l y s i s i s used. T h e r e are v a r i o u s t y p e s of u s e r information which should b e included i n t o t h e system.
These g e n e r a l l y include knowledge a b o u t a u s e r ' s l e v e l of competence, h i s in- t e r e s t s , values, a p t i t u d e s , goals, e x p e c t a t i o n s a n d assumptions a n d e v e n knowledge a b o u t t h e u s e r ' s model of how t h e system works. In t h e r e a l decision p r o c e s s e a c h e x p e r t c e r t a i n l y h a s h i s own understanding of t h e s t r a t e g y
t h a t influences h i s assignments. Besides individual c a p a b i l i t i e s , e v e n h i s p r e s e n t motivations are of i m p o r t a n c e . To a c h i e v e positive r e s u l t s i t i s n e c e s s a r y n o t only to v e r i f y t h e initial assignments b u t a l s o to divide t h e e x p e r t s i n t o national, p r o f e s s i o n a l a n d o t h e r g r o u p s b e c a u s e t h e i r opinions could d i f f e r .
While a n a l p i n g s u c h a problem as technology assessements, i t becomes clear t h a t t h e main problem i s n o t only to c h o o s e c o r r e c t l y t h e set of a l t e r n a - t i v e s , c r i t e r i a a n d t h e measurement scale b u t also to a r r a n g e t h e p r o c e d u r e f o r a c c u r a t e v e r i f i c a t i o n of t h e o u t p u t s of e x p e r t s a c t i v i t i e s t h a t could b e provided with mistakes. Moreover a n e x p e r t c a n c h a n g e h i s mind while analyzing t h e a n s w e r s of o t h e r e x p e r t s . The v e r i f i c a t i o n p r o c e d u r e should include possibilities to r e c o n s i d e r t h e initial assumptions c o n c e r n i n g a l t e r n a t i v e s , c r i t e r i a a n d cer- tainly numerical a n d q u a l i t a t i v e assignments t a k e n f r o m e x p e r t s .
The f i r s t problem t h a t a r i s e s i s how to c h o o s e t h e b e s t scale to g e t e x p e r t in- formation. I t i s well known
[I, 21
t h a t to r e c i e v e r e l i a b l e estimations, i t i s n e c e s s a r y to p r e s e n t t h e scale t h a t i s formulated i n a h a b i t u a l f o r e x p e r t s manner. Usually a n e x p e r t i s a s k e d to d e t e r m i n e quantitatively t h e l e v e l of quality of a l t e r n a t i v e s . And t h e e x p e r t should assign t h e a c c o r d a n c e between t h e q u a n t a t i v e estimation and t h i s level. I t i s c l e a r t h a t t h i s a c c o r d a n c e i s d e t e r - mined d i f f e r e n t l y by d i f f e r e n t e x p e r t s . Such r e s u l t s o b t a i n e d in t h i s manner c a n h a v e valuable mistakes. I t i s b e t t e r if t h e scale i s v e r b a l ( f o r example "good","fair", "bad") b u t a g a i n t h i s estimation c a n b e d i f f e r e n t l y c o n n e c t e d with t h e nu- m e r i c a l merits. The s c a l e should b e flexible enough to t r y t h e d i f f e r e n t accor- d a n c e between v e r b a l conclusions a n d t h e s e merits.
Another problem i s providing n o n c o n t r a d i c t o r y a n d t r a n s i t i v e assignments.
(Noncontradictory assignments give t h e same estimations in t h e same condi- tions. Transitive assignments are s u b j e c t e d to t h e condition: if a
>
b a n d b>
c t h a n a>
c ) . B e f o r e formulating t h e decision r u l e one must b e s u r e t h a t at least 80-
90% of t h e assignments fulfill t h i s r e q u i r e m e n t s .These preliminary c o n s i d e r a t i o n s r e l a t e s t r o n g l y to t h e problems of deci- sion making in t h e framework of m u l t i c r i t e r i a i l l - s t r u c t u r e d problems. Human f a c t o r s influence s t r o n g l y t h e s u c c e s s of t h e assessment p r o c e d u r e of p r o b - lems, systems a n d s i t u a t i o n s . F o r t h e s e problems in which q u a l i t a t i v e , ill-defined f a c t o r s are dominant, t h e c h o s e n set of evaluation criteria i s o f t e n s u b j e c t i v e a n d r a t i n g s assigned by e x p e r t s to t h e given a l t e r n a t i v e s b y e a c h c r i t e r i o n c a n b e q u i t e d i f f e r e n t .
I t i s a l s o well known t h a t e x p e r t s c a n d e a l with n o more t h a n f i v e to s e v e n c r i t e r i a if we would like to h a v e r e l i a b l e r e s u l t s of t h e a s s e s s m e n t p r o c e d u r e . A t t h e same time t h e initial number of c r i t e r i a i s o f t e n much g r e a t e r . The possible
solution c a n b e to r e d u c e t h e i r number o n t h e b a s i s of t h e preliminary analysis o f t h e i r sameness a n d to g r o u p them.
Some human f a c t o r s r e l a t e d to t h e decision p r o c e s s are summerized in Table 1.
Methods of m u l t i c r i t e r i a decision making d i f f e r by t h e modes of forming t h e g e n e r a l i z e d e s t i m a t e s f o r e a c h a l t e r n a t i v e o n t h e b a s i s of d a t a e x t r a c t e d from e x - p e r t s . Let's d e s c r i b e some of them keeping in mind t h e i r potential usefulness f o r t h e problem of technology assessment.
Direct Methods
In t h e s e methods t h e r e l a t i o n between g e n e r a l i z e d e s t i m a t e s (utility func- tions) a n d e s t i m a t e s b y s e p a r a t e c r i t e r i a i s p r e d e f i n e d . In most cases gen- e r a l i z e d c r i t e r i o n p r e s e n t s a l i n e a r weighed combination of s e p a r a t e c r i t e r i a . These methods are d e s c r i b e d e l s e w h e r e (see, f o r example [4]. More so- phisticated methods u s e a s p i r a t i o n levels and t a k e i n t o a c c o u n t d i s a g r e e m e n t f a c t o r s [S].
Pairwise Comparison Methods
In t h e s e methods DM c h o o s e s between s e l e c t e d p a i r s of a l t e r n a t i v e s [6].
These methuds give as a r u l e r a t h e r r e l i a b l e solutions b u t are time consuming.
They are mostly used f o r t h e small-scale problems with few a l t e r n a t i v e s and c r i - t e r i a .
Compensation Methods
In t h e s e methods [7] e s t i m a t e s f o r o n e a l t e r n a t i v e are t r i e d to b e compen- s a t e d b y e s t i m a t e s f o r a n o t h e r o n e in o r d e r t o c h o o s e t h e b e t t e r one. These methods are c o n s i d e r e d to b e t h e m o s t user-friendly as at o n e time a n e x p e r t d e a l s only with p a i r s of a l t e r n a t i v e s . All shortcomings a n d a d v a n t a g e s f o r both a l t e r n a - tives are analysed and c r o s s e d o u t b y p a i r s to see what is l e f t at t h e e n d of t h i s p r o c e d u r e .
Table 1. Human factors r e l a t e d to t h e decision p r o c e s s . 1. Human c a p a c i t i e s in information processing are
r a t h e r limited b u t flexibility of humans, t h e i r adaptability and e x p e r i e n c e make i t possible t o r e l y on t h e i r e x p e r t estimations.
2. Human capabilities depend on t h e type of t h e
problem and on t h e way of obtaining t h e r e l e v a n t information from people.
3. Short-term memory c a p a c i t y is Limited. I t c a n p r o c e s s only s e v e r a l s t r u c t u r a l d a t a units.
4 . Man e i t h e r a d a p t s to a complex problem or tries to a d a p t i t t o h i s own capabilities.
Humans are usually a b l e to l e a r n f r o m previous a c t i o n s (mostly by try-and-see technique).
6. Solving unique problems often leads to conflicting
and differing a n s w e r s during t h e decision process.
i
?- Human c a p a c i t i e s during t h e decision p r o c e s sdepend strongly o n t h e way t h e problem is formulated.
8. More adequate are methods of eliciting information f r o m humans t h a t use habitual qualitative s c a l e s but not numerical ones.
9. The complexity of t h e decision problem i n c r e a s e s with t h e number of c r i t e r i a , quantity of estimates on t h e c r i t e r i o n s c a l e a n d with t h e number of t h e resulting quality c l a s s e s .
P e r s o n a l , professional, national a n d o t h e r individual motivations influence strongly t h e assignments of e x p e r t s . 11. Interinfluence of opinions of e x p e r t s engaged in one
problem c a n lead t o changes in t h e i r initial assignments.
12. Humans make errors during t h e decision p r o c e s s due to inadequate understanding of t h e p a r t i c u l a r problems.
c a r e l e s s n e s s or o t h e r f a c t o r s .
1
13. Human estimates c a n b e c o n t r a d i c t o r y and non-transitive.1
14. Humans p r e f e r t h e information to b e r e p r e s e n t e d more in images, g r a p h s than by t a b l e s with numbers.Axiomatic Yethoda
In t h e s e methods [4] some c h a r a c t e r i s t i c f e a t u r e s of t h e utility function are postulated reflecting t h e p r e f e r e n c e s of DM. During t h e assessment p r o c e d u r e t h e s e p r e f e r e n c e s a r e verified and adjusted.
Interactive Methods
They are used e f f e c t i v e l y if t h e p a r t i a l model of t h e system i s known a n d p r e f e r e n c e s a n d r e l a t i o n s between d i f f e r e n t c r i t e r i a are a n a l p e d a n d i n t e r a c t i v e - ly modified [B, 9, 10 ].
I t should be noted t h a t p r a c t i c a l application of most of t h e a b o v e d e s c r i b e d methods f o r i l l - s t r u c t u r e d problems h a s r a t h e r n o t b e e n hopeful. One of t h e r e a s o n i s t h a t e x p e r t s c a n n o t a s s i g n r e l i a b l e numerical e s t i m a t e s ( r a t i n g s ) f o r a l t e r n a - t i v e s b y a lot of c r i t e r i a at o n c e without analysing t h e opinions of o t h e r ex- p e r t s a n d without some discussions.
Summerizing t h e b r i e f overview of t h e existing methods, having i n mind t o c h o o s e t h e b e s t f o r t h e problem of technology assessments, i t i s c l e a r t h a t to ob- t a i n r e l i a b l e r e s u l t s f o r a r e a s o n a b l e p e r i o d of time i t i s n e c e s s a r y to combine a d v a n t a g e s of d i f f e r e n t methods i n t o one p r o c e d u r e .
In o u r a p p r o a c h w e combined some elements of t h e d i r e c t method of con- s t r u c t i n g t h e g e n e r a l i z e d utility function as a combination of weighed r a t i n g s b y e a c h c r i t e r i o n f o r a l l a l t e r n a t i v e s with i n t e r a c t i v e computerized v e r i f i c a t i o n p r o c e d u r e . During t h i s p r o c e d u r e , initial assignments of e x p e r t s are a v e r a g e d . A s p e c i a l m e a s u r e
-
Mean S q u a r e Deviation-
i s i n s e r t e d t o c l a r i f y t h e d i s a g r e e - ments between e x p e r t s . Pairwise comparison i s used f o r t h e v e r i f i c a t i o n of t h e initial e x p e r t assignments.In t h i s p a p e r w e p r e s e n t t h e initial p r i n c i p l e s (Section 2), a s s e s s m e n t pro- c e d u r e (Section 3) and s t r u c t u r e of t h e system (Section 4 ) . Some programming as- p e c t s are d e s c r i b e d in S e c t i o n 5. TAS now i s being implemented f o r t h e assessment of e n e r g y technologies. H e r e we p r e s e n t I n t e r a c t i v e Information System f o r Tech- nology Assessments as a t o o l f o r providing full-scale information to t h e policy mak- e r to analyze t h e i n t e r n a t i o n a l e x p e r i e n c e in e n e r g y systems. I t should b e pointed o u t t h a t t h i s f i r s t v e r s i o n of TAS d o e s n o t claim
to
p r o v i d e him with t h e decision r u l e to c h o o s e t h e p a r t i c u l a r technology f o r h i s p u r p o s e s b u t m o r e to stimulate h i s decision p r o c e s s o n t h e b a s i s of v a r y i n g opinions, including national a n d p e r s o n a l motivations, d i s a g r e e m e n t f e a t u r e s a n d some a v e r a g e s . I t i s u p to the policy mak- e r to make a decision a f t e r analyzing t h e full set of information s t o r e d i n TAS.Based on t h e e x p e r i e n c e of t h e case s t u d y o n e n e r g y technology assessments, i t i s planned, as a s e c o n d s t e p , to t u r n to formulating decision algorithms.
R e f e r i n g
to
t h e a b o v e mentioned difficulties to formulate t h e decision r u l e b a s e d o n e x p e r t opinions c o n c e r n i n g t h e f i n a l c h o i c e of technologies f o r a p a r t i c u l a r u s e r , i t becomes clear t h a t t h e problem should b e divided at least i n t o two s t a g e s .The f i r s t v e r s i o n of t h e technology assessments system c a n b e c o n s t r u c t e d taking i n t o a c c o u n t human f a c t o r s of decision making a n d some preliminary as- sumptions a b o u t t h e p r o c e s s of calculating t h e o u t p u t merits.
2. MAIN PRINClPLES
OF TAS
The main p r i n c i p l e s of TAS are p r e s e n t e d in Table 2.
Table 2. Main p r i n c i p l e s of TAS.
1. Openess
TAS i s c o n s t r u c t e d of s e v e r a l u n i v e r s a l modules with a s t a n d a r d i n t e r f a c e . I t allows to a d d a n d modify t h e system f o r o t h e r a p p l i c a t i o n s of t h e s a m e kind.
2. Flexibility
I t i s r a t h e r simple to r e f o r m u l a t e t h e main problem, l i s t of technologies a n d cri- t e r i a , to c h a n g e weights of c r i t e r i a a n d t h e i r s c a l e , to r e f o r m t h e o u t p u t cal- culations, forms of information r e p r e s e n t a t i o n , etc.
3. User-friendliness
After t h e preliminary professional adjustments t h e system c a n b e used by a non-professional programming u s e r . I t includes a n h i e r a r c h i c a l HELP-system.
4. D a t a p r o c e s s i n g
The d a t a p r o c e s s o r c o n s i s t s of a number of small BASIC p r o g r a m s t h a t c a n b e e a s i l y e d i t e d f o r t h e p a r t i c u l a r u s e r .
5. G r a p h i c s
A s p e c i a l g r a p h i c s subsystem i s provided to show a n y kind of d a t a s t o r e d in In- t r o d u c t o r y , R e s u l t a n t a n d Verification Data Bases.
6. Modes of i n t e r p r e t a t i o n of t h e o u t p u t f i g u r e s
The c r i t e r i a s c a l e c o n s i s t s of s e v e r a l a n s w e r s l e v e l s (L): 0 - none, 1
-
b a d , 2-
p o o r , 3
-
f a i r , 4-
good, 5-
e x c e l l e n t . They c a n b e i n t e r p r e t e d in t w o modes.A) Numerical in which e a c h l e v e l i s assigned a r a t i n g (R): R
=
N*
L, whereN
i s as c a l e c o e f f i c i e n t which c a n b e v a r i e d during t h e a n a l y s i s of t h e r e s u l t s .
B) Non-numerical in which p e r c e n t a g e of a l l answer levels in t h e o u t p u t d a t a i s c a l c u l a t e d .
7. Verification of assignments
A s p e c i a l susbsystem i s developed to v e r i f y t h e assignments of e x p e r t s by t h e pairwise comparison of t h o s e of a given e x p e r t , o t h e r e x p e r t s a n d a v e r a g e s . 8. Disagreements analysis
A d i s a g r e e m e n t f a c t o r i s i n t r o d u c e d as a mean s q u a r e deviation of assign- ments from a v e r a g e s to c h a r a c t e r i z e t h e d i f f e r e n c e i n opinions.
9. Modification of c r i t e r i a
A s p e c i a l p r o c e d u r e i s s u g g e s t e d to r e c o n s i d e r t h e l i s t of c r i t e r i a a n d to r e d u c e t h e i r number o n t h e b a s i s of t h e a n a l y s i s of t h e initial assignments a n d r e s u l t a n t
d a t a analysis.
10. E x p e r t s g r o u p analysis
In o r d e r to t a k e i n t o account t h e d i f f e r e n c e s between motivations of various ex- p e r t g r o u p s a s p e c i a l filtering subsystem c a n s e l e c t and show t h e asignments of d i f f e r e n t e x p e r t groups (country, specialization, etc .).
11. C r i t e r i a g r o u p analysis
A l l c r i t e r i a are grouped and a v e r a g e d output p a r a m e t e r s are calculated f o r e a c h g r o u p relying more on t h e enlarged estimates t h a n on detailed analysis of a l a r g e number of c r i t e r i a .
3. ASSESSMENT
PROCEDURETaking into account t h e above said, t h e assessment p r o c e d u r e c a n b e divided i n t o d i f f e r e n t stages.
Choice of Alternutivea
In m o s t cases altenatives t o b e assessed are specified by t h e g r o u p of cus- t o m e r s and DM who initiate t h e assessment p r o c e d u r e . Alternatives
-
e n e r g y technologies to b e assessed are listed in Table 3. Their illustrative definition i s p r e s e n t e d in Appendix 2.Table 3. List of technologies
1. Lurgi P r e s s u r e Coal Gasification 2. Hydropymlysis f o r coal conversion
3. Coal conversion by s u p e r c r i t i c a l e x t r a c t i o n
4 . Combined c y c l e power s t a t i o n with integrated coal gasification 5. High t e m p e r a t u r e g a s cooled reactors
6. Gas t u r b i n e s
7 . 'SASOL1'-type coal lfquifaction plant
8. Low p r e s s u r e n a t u r a l g a s to methanol conversion 9. Geothermal e n e r g y
10. E l e c t r o t h e r m a l hydrogen
11. High efficiency membrane complex methane production 12. S u p e r h e a t pump e n e r g y accumulation
13. Fuel cell power plant
1 4 . Gasification in molten i r o n bath
I t should b e pointed o u t t h a t t h e a b o v e mentioned technologies a n d e n e r g y systems based o n t h e i r use h a v e a l r e a d y showed good o p e r a t i n g q u a l i t i e s (like t h e SASOL p l a n t , Lurgi P r e s s u r e Coal Gasification). So t h e i r main c a p a b i l i t i e s a n d f e a t u r e s are r a t h e r to b e a s s e s s e d by e x p e r t s . A t t h e same time some of t h e i r p e r f o r m a n c e i m p a c t s c a n n o t b e e a s i l y estimated by q u a n t a t i v e methods. Another f a c t o r t h a t influences t h e i r c h o i c e f o r t h e assessment i s t h e i r i n c r e a s i n g usage in d i f f e r e n t c o u n t r i e s , e a c h having i t s own e x p e r i e n c e , t r a d i t i o n s a n d conditions.
And t h e main aim of t h e assessment p r o c e d u r e i s to c l a r i f y t h e potential p r e f e r e n c e s f o r e a c h of t h e teachnologies in d i f f e r e n t c o u n t r i e s . IIASA seems t h e p r o p e r p l a c e f o r s u c h an i n t e r n a t i o n a l analysis of e n e r g y technologies.
C h o i c e o f the Set of C r i t e r i a
A s a r u l e t h e set of c r i t e r i a f o r t h e e x p e r t analysis i s c h o o s e n o n t h e b a s i s of t h e e x p e r i a n c e of DM engaged in t h i s a c t i v i t i e s a n d works of s c i e n t i s t s in t h e field. I t is n a t u r a l t h a t DM who Launched t h e assessment p r o c e d u r e f i r s t would l i k e to h a v e much more c r i t e r i a t h a n n e c e s s a r y
-
not to f o r g e t a n y of t h e a s p e c t s of t h e problem. I t o f t e n l e a d s to a r a t h e r big set of c r i t e r i a which c a n n o t b e handled p r o p e r l y by human e x p e r t s . S p e c i a l p r o c e d u r e s are a r r a n g e d to de- crease t h e initial number of c r i t e r i a to make t h e r e s u l t s m o r e r e l i a b l e a n d useful.In o u r case 23 c r i t e r i a w e r e initially c h o s e n ( s e e Table 4).
Table 4. List of c r i t e r i a .
1. R&D Time Requirement 2. Costs of R&D
3. P r o b a b i l i t y of R&D S u c c e s s
4. Capability of I n d u s t r i a l M a n u f a c t u r e r 5. Availability of Material a n d R e s o u r c e s 6. Institutional B a r r i e r s
7. S o c i a l Acceptability 8. Level of Pollution 9. Flexibility of Siting
10. Waste Handling a n d Disposal 11. Availability of Cleaning 12. Consequences of Accident 13. S a f e t y C h a r a c t e r i s t i c s
14. Adaptability t o Types a n d S o u r c e s of Fuel 15. Outage R a t e
16. Risk of High Damage 17. Capital Cost
18. Construction P e r i o d
19. Efficiency of E n e r g y a n d R e s o u r c e s Utilization 20. Multiproducts Availability
21. 0 &
M
Requirements22. Commercial Acceptability of P r o d u c t
23. Availability a n d P r i c e s of N a t u r a l R e s o u r c e s
These c r i t e r i a c h a r a c t e r i z e t h e possibilities f o r t h e s u c c e s s f u l develop- ment of e n e r g y technologies, i t s economic p r o p e r t i e s a n d f a c t o r s r e l a t e d to s a f e t y , flexibility a n d environmental consequences. T h e i r l i s t claims on comprising a l l sorts of p a r a m e t e r s n e c e s s a r y
to
e s t i m a t e t h e i r development.The q u e s t i o n n a i r e c o n s i s t s of a number of t a b l e s , e a c h of them having dif- f e r e n t questions c o n c e r n i n g v a r i o u s a s p e c t s of a l l technologies u n d e r c o n s i d e r a - tion, weights of t h e e v a l u a t i o n c r i t e r i a , a n d r a t i n g s f o r e a c h technology a n d c r i t e r i o n .
As mentioned a b o v e , t h e t r a d i t i o n a l economic a p p r o a c h i s useful f o r well- developed technologies, where t h e quality of input d a t a i s good. F o r new technolo- g i e s , especially at t h e s t a g e of r e s e a r c h development, t h e r e are many i s s u e s beyond t h e question of c a p i t a l a n d o p e r a t i n g c o s t s . Among them are s o c i a l a c c e p - tability, level of pollution, availability of n e c e s s a r y m a t e r i a l s a n d r e s o u r c e s , cost a n d time r e q u i r e d f o r R&D, a n d many o t h e r s .
In o r d e r to h a v e b e t t e r measurements f o r t h e a s s e s s m e n t of new e n e r g y tech- nologies a t IIASA, a method, b a s e d o n t h e e x p e r t ' s a n a l y s i s of t h e many c r i t e r i a , w a s developed by many a u t h o r s of v a r i o u s c o u n t r i e s . Two v e r y similar a p p r o a c h e s w e r e developed by t h e Western IES Consortium [ll] a n d by Russian s c i e n t i s t s [lZ, 131. A set of evaluation c r i t e r i a w a s p r o p o s e d , including 2 3 v a r i a b l e s c o n c e r n e d with major f a c t o r s of technology development a n d implementation. All c r i t e r i a are divided i n t o f o u r g r o u p s (Figure 1 ) :
G r o u p I c o n t a i n s c r i t e r i a to assess possibilities of t h e s u c c e s s f u l develop- ment of a technology up to t h e s t a g e of a pilot i n d u s t r i a l plant.
Among t h e c r i t e r i a are two more g e n e r a l t h a n t h e o t h e r s in t h e g r o u p . They a p p l y to a p p l i c a t i o n s of t h e technology of s c a l e . G r o u p I1 includes environmental consequences of t h e technology a s s e s s e d
and possibilities of management of t h e environmental e f f e c t .
G r o u p I11 is dealing with m a t t e r s of s a f e t y , r e l i a b i l i t y a n d technological flexibility of a technology.
Group IV includes c h a r a c t e r i s t i c s which are needed to assess economic p r o - p e r t i e s of a technology a n d t h e e x p e c t e d economical s i t u a t i o n when t h e technology i s implemented.
Assignment of C r i t e r i a Weighb
The weights f o r t h e given c r i t e r i a c a n b e o b t a i n e d f r o m t h e e x p e r t s or b y means of s p e c i a l mathematical p r o c e d u r e s [1,2]. W e c o n s i d e r t h e weights f o r t h e c r i t e r i a e q u a l f o r a l l technologies b e c a u s e t h e y are d e p e n d e n t mainly o n political, s o c i a l , economic, a n d o t h e r conditions a n d not on t h e p a r t i c u l a r t y p e of technolo- BY-
M g n m e n t of R a t i n g s
S e v e r a l technologies are usually s e l e c t e d a n d b r i e f e d i n t h e q u e s t i o n n a i r e to- g e t h e r with t h e c r i t e r i a c h o s e n . F i r s t , e a c h e x p e r t must e s t i m a t e t h e weights of t h e c r i t e r i a a n d t h e n p u t down t h e r a t i n g s f o r e a c h technology b y e a c h c r i t e r i o n . These r a t i n g s are divided i n t o five levels. E a c h l e v e l c a n b e r e p r e s e n t e d in t w o d i f f e r e n t forms: v e r b a l conclusions ( f o r example, e x c e l l e n t , good, f a i r , p o o r , b a d ) a n d numerical v a l u e s ( f o r example: 5, 4 , 3, 2, 1
-
f i v e being t h e h i g h e s t r a n k i n g ) . All t h e r a t i n g s are multiplied by c o r r e s p o n d i n g v a l u e s of weighing c o e f f i c i e n t s to form a score f o r e a c h technology b y e a c h c r i t e r i o n a n d t h e total (sum) score of e a c h technology. F o r d i f f e r e n t p u r p o s e s t h e decision m a k e r c a n h a v e e i . t h e r t h e r e s u l t a n t p e r c e n t a g e of v a r i o u s l e v e l s of v e r b a l conclusions or numerical estima- tions of means a n d o t h e r s t a t i s t i c a l v a l u e s of r e s u l t a n t p a r a m e t e r s f o r e a c h tech- nolog y.
O u t p u t Figures
E a c h technology j is e v a l u a t e d by e x p e r t k with c r i t e r i o n
C,,
w h e r e j=
1 ,... m, k = 1,
...
L , i = 1,...
n. E a c h c r i t e r i o n h a s i t s own weight c o e f f i c i e n t W1 as- signed by e a c h e x p e r t . On t h e b a s i s of t h e s e estimations some c h a r a c t e r i s t i c o u t p u t v a l u e s are c a l c u l a t e d .Score SIJk i s c a l c u l a t e d as
w h e r e Rllk i s t h e r a t i n g f o r jth technology a n d ith c r i t e r i o n a s s i g n e d by kth e x p e r t
Average weight c o e f f i c i e n t s f o r c r i t e r i o n : C w , k k
AW,
= -
L
a n d mean s q u a r e d e v i a t i o n s of weight coefficients:
MSDA, =
L
The a v e r a g e score a n d mean s q u a r e deviation are c a l c u l a t e d f o r e a c h t e c h - nology a n d e a c h c r i t e r i o n :
A s e p a r a t e t a b l e p r e s e n t s t h e t o t a l scores f o r e a c h technology e v a l u a t e d by e a c h e x p e r t :
A final t a b l e c o n t a i n s i n t e g r a t e d e s t i m a t e s of a l l t h e technologies 1, by a v e r a g i n g t h e t o t a l s c o r e s f o r e a c h technology assigned by e a c h e x p e r t :
Deviation of e x p e r t s opinions are estimated by :
Denote r
-
t h e index of a c r i t e r i a g r o u p : r=
l,..s, where s-
number of g r o u p s (in o u r case s=
4).The a v e r a g e s c o r e a n d MSD are c a l c u l a t e d f o r e a c h e x p e r t a n d e a c h technolo- g y f o r e a c h c r i t e r i a g r o u p
w h e r e i,is a number of c r i t e r i a i n e a c h g r o u p (xi,=i).
All t h e o u t p u t p a r a m e t e r s d e s c r i b e d a b o v e form t h e Numerical Data Base as a number of t a b l e s .
V e r i f i c a t i o n P r o c e d u r e
Based on t h e r e s u l t s of initial e x p e r t assignments t h i s p r o c e d u r e includes t h e d e t a i l e d analy-sis of a l l c r i t e r i a divided i n t o d i f f e r e n t g r o u p s - to c l e r i f y t h e i r c o n t r a d i c t i v e n e s s a n d sameness. This will make i t possible to d e c r e a s e t h e i r number a n d to l e a v e t h o s e t h a t are most i m p o r t a n t f o r W e c o n c r e t e assess- ment p r o c e d u r e . Afterwards a l l e x p e r t s c a n o b s e r v e t h e obtained r e s u l t s a n d c o m p a r e t h e i r e s t i m a t e s with a v e r a g e values taking i n t o a c c o u n t t h e d i s a g r e e m e n t f a c t o r s (MSD). I t will a l l o w to modify t h e i r initial assignments or
-
if t h e y d o not a g r e e with o t h e r opinions-
to comment t h e i r decisions.A t t h e final s t a g e a l l information beginning with t h e initial o u t p u t d a t a to t h e v e r i f i e d o n e i s p r e s e n t e d f o r a l l p a r t i c i p a n t s with a l l comments a n d g r a p h i c a l im- a g e s . I t will allow n o t only to h a v e a v a r a g e d a b s t r a c t i v e r e s u l t s b u t to d e s c r i b e d i f f e r e n c e s in opinions b a s e d on national, p r o f e s s i o n a l and o t h e r f a c t o r s . In t h i s case t h e r e s u l t s of t h e a s s e s s m e n t p r o c e d u r e c a n b e used by d i f f e r e n t national and s o c i a l g r o u p s a n d a l l f o r e c a s t s will b e more r e l i a b l e .
C r i t e r i a Modification
I t i s well known t h a t if a n e x p e r t i s t o d e a l with a lot of c r i t e r i a h i s estima- tions are n o t r e l i a b l e ( s e e Table 1 ) . That's why a s p e c i a l p r o c e d u r e i s implemented to r e d u c e t h e i r initial number by grouping them.
F o r t h i s p u r p o s e a l l c a l c u l a t e d scores f o r c r i t e r i a g r o u p s (based o n assign- ments f o r t h e full set of c r i t e r i a ) are compared with t h e assignments made f o r t h e c r i t e r i a g r o u p s ( s e e Appendix 1 ) . Verifying t h e s e t w o r e s u l t s will make i t possible to u s e only g r o u p c r i t e r i a assignments in t h e f u t u r e .
4.
GENERAL
STRUCTURE OFTAS
The g e n e r a l s t r u c t u r e of TAS is shown in F i g u r e 2. I t c o n s i s t s of t h e In- t r o d u c t o r y Data B a s e , which stores a l l t h e information t a k e n by t h e question- n a i r e f r o m t h e e x p e r t s , t h e I m p o r t p r o g r a m , which b r i n g s t h i s information to t h e Data P r o c e s s o r , The Data P r o c e s s o r , including d i f f e r e n t f i l t e r s a n d a n a l y z e r s to make a l l d a t a t r a n s f o r m a t i o n s r e p r e s e n t i n g i t in t h e most convenient way, a n d t h e E z p o r t p r o g r a m , which p u t s t h e p r o c e s s e d d a t a i n t o t h e R e s u L t a n t D a t a Base.
TAS h a s a h i e r a r c h i c a l menu system. When t h e u s e r e n t e r s TAS h e watches t h e Main Menu o n the s c r e e n (Figure 3) with a l l n e c e s s a r y positions beginning with t h e i n t r o d u c t i o n of e x p e r t s a n d t h e c h o i c e of t h e c r i t e r i a u p to some editing positions
-
to a d j u s t TAS p a c k a g e s f o r t h e n e e d s of t h e p a r t i c u l a r u s e r .F i r s t position of t h e Main Menu is to e n t e r E x p e r t s D a t a Base (Figure 4 )
-
to i n t r o d u c e or c h a n g e a l l t h e information a b o u t t h e e x p e r t s of t h e a s s e s s m e n t p r o c e d u r e . The n e x t s t e p i s to assign weights f o r t h e given c r i t e r i a . When o n e e n t e r s t h e a p p r o p r i a t e position of t h e Main Menu h e finds t h e Submenu t h a t al- lows to formulate t h e l i s t of c r i t e r i a . Afterwards a s p e c i a l window a p p e a r s i n which e a c h e x p e r t c a n manipulate t h e values of t h e c r i t e r i a weights while t h e i r normalization (by t h e r u l e t h a t t h e i r sum i s equal to 1 0 0 ) i s being done au- tomatically.S p e c i a l e x p o r t p r o c e d u r e i n t r o d u c e s information a b o u t e x p e r t s a n d c h o s e n values of t h e c r i t e r i a weightv to t h e I n t r o d u c t o r y Data Base (IDB) (Figure 5). A f t e r t h e a n a l y s i s of t h e q u e s t i o n n a i r e r a t i n g s f o r e a c h e x p e r t , technol- ogy a n d c r i t e r i o n are i n t r o d u c e d to IDB to s e r v e as a b a s i s f o r f u r t h e r calcula- tions. Or i t c a n b e done d i r e c t l y i n IDB.
The n e x t t w o positions of t h e Main Menu allow to p r o c e s s a l l i n t r o d u c t o r y d a t a a n d to e x p o r t t h e o u t p u t d a t a to t h e Resultant Data Bases (Numerical and Non- numerical).
When t h e u s e r e n t e r s t h e Numerical R e s u l t a n t Data Base h e c a n see t h e NRDB Submenu a n d c a n o b s e r v e a l l t h e o u t p u t f i g u r e s o n t h e s c r e e n as t a b l e s or g r a p h s of d i f f e r e n t kinds a n d c a n h a v e them p r i n t e d o u t (Figures 6-13). This b a s e includes a s e p a r a t e f r a m e f o r Averages a n d MSD of weights a n d scores f o r e a c h technology, a f r a m e with c a l c u l a t e d scores f o r g r o u p s of c r i t e r i a a n d a f r a m e with i n t e g r a t e d r e s u l t s f o r e a c h of t h e e x p e r t s a n d technologies i n t e g r a t e d euti- mates a v e r a g e d by a l l e x p e r t s .
The Non-numerical Resultant Data Base c o n s i s t s of a number of f r a m e s . E a c h f r a m e r e p r e s e n t s t h e p e r c e n t a g e of d i f f e r e n t a n s w e r l e v e l s f o r e a c h technology (Figure 14).
E n t e r i n g t h e 'Verification' position of t h e Main Menu a f t e r pointing t h e technology-number a n d e x p e r t - n u m b e r f o r t h e comparison of a v e r a g e d o u t p u t m e r i t s , a l l t h e a p p r o p r i a t e information i s t a k e n from IDB a n d NRDB a n d i n t r o d u c e d to t h e Verification D a t a Base 1 (VDB1) (Figure 1 5 ) .
Taking i n t o a c c o u n t t h e problems of t h e use of a big ammount c r i t e r i a t h a t c o n t r a d i c t s sometimes with human e x p e r t f a c t o r s a n o t h e r v e r i f i c a t i o n p r o c e d u r e w a s s u g g e s t e d to d e a l with t h e o u t p u t s connected with t h e c r i t e r i a g r o u p s . To c h e c k t h e r e l i a b i l i t y of t h e initial. assignments with e a c h of 23 c r i t e r i a , a s e p a r a t e position i s provided in t h e q u e s t i o n n a i r e in which t h e e x p e r t s are to as- sign weights a n d r a t i n g s f o r e a c h technology by t h e f o u r mentioned c r i t e r i a g r o u p s . This information i s c o m p a r e d with o u t p u t g r o u p scores c a l c u l a t e d o n t h e b a s i s of initial scores f o r t h e 23 c r i t e r i a (in VDB2).
5.
SOME
PROG-G ASPECTSTAS i s based o n d i f f e r e n t main modules t h a t were i n t e g r a t e d to s o l v e tech- nology a s s e s s m e n t problems. Some of t h e modules w e r e worked o u t i n t h e Com- puting C e n t r e of t h e USSR Academy of Sciences. They include SPECTR
-
a d a t a o r i e n t e d b a s e system by which a l l mentioned d a t a b a s e s w e r e built; SPOUT a n d SPIN-
p r o g r a m s f o r importing a n d e x p o r t i n g f i l e s f r o m / t o d a t a b a s e s to/from calculation a n d analysing p r o g r a m s ; p r o g r a m s t h a t are i n t e g r a t e d in Data Processor and s e r v e as means f o r t h e a p p r o p r i a t e d a t a transforma- tion; FILTERS-
to make national a n d p r o f e s s i o n a l samples from a l l f r a m e s of t h e Resultant Data Base; a n d LEXICON-
f i l e s e d i t o r . Some additional p a c k a g e s (DG a n d CHART) are used to r e p r e s e n t information o n t h e s c r e e n (as g r a p h i c s and plots) a n d to h a v e i t p r i n t e d . All menus of TAS w e r e built using module DLG t h a t p r o v i d e s e a s y modification of menu positions a n d i s b a s e d o n t h e c a l l of DOS e x e c u t a b l e p a c k a g e s (position Dialogue S c e n a r i o in t h e Main Menu c a l l s e d i t o r in which a l l menus c a n b e changed if n e c e s s a r y ) . File TAS.DOC c o n t a i n s t h e full d e s c r i p t i o n of TAS. One c a n a l s o g e t some i n s t r u c t i o n s f o r using TAS by means of HELP f a c i l i t i e s a n d file TAS.CTL.P r o g r a m s f o r Data Processor are w r i t t e n o n BASIC. T h e r e f o r e , t h e y c a n b e e a s i l y modified by t h e u s e r f o r h i s own p u r p o s e s a n d f o r t h e given s t r u c t u r e s of t h e R e s u l t a n t Data Bases. S e p a r a t e position of t h e Main Menu allows to e n t e r GWBASIC e d i t o r .
TAS i s b a s e d on t h e use of IBM-type p e r s o n a l c o m p u t e r s with t h e h a r d disk c o l o u r display a n d RAM n o l e s s t h a n 512k. I t i s provided with developed HELP fa- c i l i t i e s a n d c a n b e used e v e n by non-specialists in c o m p u t e r programming.
In case of n e c e s s a r y modifications of t h e forms of d a t a b a s e s , f i l t e r s , o r some o t h e r s u p p l e m e n t a r y p r o g r a m s i n TAS, consultations of professional p r o - g r a m m e r s will p r o b a b l y be neadad to h e l p t h e u s e r while t h e normal o p e r a t i o n of TAS i s a r a t h e r simple p r o c e d u r e .
Supplied with the installation procedure TAS requires no l e s s than 3Mbt of the hard disk space.
REFERENCES
1. L a r i c h e v O.I., Zuev Y .A., Gnedenko L.S. Method Z a p r o s to s o l v e i l l - s t r u c t u r e d c h o i c e problems upon many c r i t e r i a . M o s c o w , VNIISI, 1979.
2. V.Yadov. Sociological r e s e a r c h . Moscow, Nauka, 1972.
3. B e r r y D.C, B r o a d b e n t D.E. E x p e r t systems a n d t h e man-machine i n t e r f a c e . Part 2: The u s e r i n t e r f a c e . E x p e r t systems 1987, v.4, No.1.
4. Keeney R., Raiffa H. Decision with multiple objectives: p r e f e r e n c e s a n d t r a d e - offs. N.Y., Wiley, 1976.
5. Lewandowski A., Johnson S., Wierzbicki A. A p r o t o t y p e s e l e c t i o n committee de- cision a n a l y s i s a n d s u p p o r t system, SCDAS: T h e o r e t i c a l b a c k g r o u n d a n d com- p u t e r implementation. P r e p r i n t s of 7-th I n t e r n a t i o n a l C o n f e r e n c e o n Multicri- t e r i a Decision Making. Kyoto, J a p a n , August 18-22, 1986.
6 . Zionts S., Wallenius J . R e c e n t developments i n our a p p r o a c h to multicri- t e r i a decision making. In; I n t e r a c t i v e Decision A n a l p i s . Springer-Verlag , 1984.
7. Mac Crimon
K.R.,
Sin I.K. Making trade-off. Decision S c i e n c e s , 1974, n.5.8. A.K.Alabyan, V.M. Afanasiev. Feasible Domain method a n d i n t e r a c t i v e system f o r t h e design a n d c o n t r o l of dynamic m u l t i c r i t e r i a systems. P r e p r i n t s of 7-th I n t e r n a t i o n a l C o n f e r e n c e o n Multicriteria Decision Making. Kyoto, J a p a n , August 18
-
22, 1986.9 . H.Nakayama, Y .Sawaragi
.
Satisficing trade-off method f o r multiob jective pro- gramming. Jn : " I n t e r a c t i v e Decision Analysis."
Springer-Verlag, Berlin, 1984.1 0 . M.Grauer. A.Lewandowski, A. Wierzbicki. DIDASS
-
t h e o r y , implementation a n d e x p e r i e n c e . In : "Interactive Decision Analysis". S p r i n g e r - V e r l a g , Berlin, 1984.1.1. k r s e s s m e n t s a n d measurements f o r e n e r g y c o n v e r s i o n systemr. IES Consor- tium P a p e r , Sweden, S e p t e m b e r 1986.
1 2 . Okorokov V.R. Management of t h e e l e c t r i c a l power Systems, Leningrad State Univ., 1976.
1 3 . Okorokov V.R., Shavelev, e d s . Complex a n a l y s i s of t h e e f f e c t i v e n e s s of techn- i c a l decisions i n e n e r g e t i c s . Leningrad, E n e r g y , 1985.
EVALUATION CRITERIA
F i g u r e 1. Groups o f C r i t e r i a .
I M E R A C T M SYSTEM FOR TECHNOLOGY ASSESSMENTS GROUP 1
Succe8dul development
1
TECHNOLOGIES SPECIFICATION GROUP 2Environmental contequances
F i g u r e
2.
S t r u c t u r e o f TAS.-
E X E M S CRITERIA FORMULATION GROUP 3Wdy, rellablllty and flexlblllty
,
GROUP 4 Economlc
propertlet
I 1
RATINGS ASSIGNMENT WEIGHTS ASSIGNMENT
4
-
INTRODUCTORY DATA BASE
A
I
DATA PROCESSING
CRITERIA MODIFICATION
1,
-
VERIFICATIONb
DATA BASE
I
RESULTANT DATA BASES1 1'
I I
I E X P E R T S I N T R O D U C T I O N I
I W E I G H T S ASSIGNMENT I
I
INTRODUCTORY D A T A BASE ( I D B ) 1 I N U M E R I C A L D A T A PROCESSINGI
INON-NUMEHICRL DATA PROCESSING I
l R E S U L T A N T DATA B A S E ( N u m e r i c a l ) I l R E S U L T A N T D A T A B A S E ( N o n - n u m e r i c a l ) I
I V E R I F I C A T I O N I
I D I A L O G U E SCENAR I
O
II D A T A PROCESSOR E D I T I N G I
I I
F i g u r e 3. Main Menu.
. - - -
t..j r.4 1.3
* *
I..:# e M t.:!*
I..: P"j !..:I M pi P<I t..:~ I..:!. - - -
b-l e ti3 ti3 e
*
e P.7 iJ3 b3 e ti3 ti3 e e L3 ti? ti3. - - -
e e e Pi I.? I..> Pi ct ct P:I
*
Pi*
P I ct I.-:! I..? e. - - -
e
ti3*
ti3 e*
b3 ti3 b3 e ti3 e e e I.? ti3 ti3*
. - - -
p:1 e ?..I
*
e -I* *
e p:1*
1p: I.? Xl r.i* *
I..:!. - - -
Kl r.1 t..:i I.:l
*
pi* * * * *
:1 t. t..:1*
!..:I I..:! I..? 1::r- - -
b I::! 4 b [E [E Q . - I O
. . .
.-I 8 1.7
- - -
[E +
*
b b[ E b - +
. . .
.-I 4 Fl
- - -
ti3 li3 Li3
. .
r.4.
Q r4 '3
.-I .-I
- - -
L3 M
b b
00 b3 M
;c-';
.-I
- - -
in L 8 Z
. 4 4 L .4 f L - C rn . 4 .4
m n c l
It 3
4J
-
. ~t a-
c a r 0 0 U i L
.4 U
< c
YJ 0
4J A .4 6 4
. 4 (U
c U >
C 0 (U
u m - l
- - -
4 b 00
- - -
. 4 i- G
L E
a i o It
$ 4
a
L!
. Z f
L 4 J
.
.4 4 ill .4 Oi f-
6 1U ic:
n Y
Z . ill 0 in
ill
9 /
U L
i
i~H I
t..:1 P4
-
iD il3 t:i
-
!?
I' It..:1
- g
'J I.':!
-
b3 I3 N
-
b3
r.4
I..?
-
t..:1 t-.:1 t..:1
-
'-' a
9 *:-I
8 .
-
&
:
; J
-
0' d- 1:)
-
i..; d- 1.q
-
d- l..?
M I
# ..
-
a 'L'
t::i
!i?
-
z
W- a
LC LC 3
-
>
>
0 4'
0 L C
4'
0
-
4
-
o L >
L O O
o n 4
- G O 0
+
0a=, 4 r.4 I.?!
*
+ ~ 4 t - ? e 1 1 ? ~ b m @ + + 4 + +
I l l l l l l i l l t i l l
I-!-!-!-!-!-!-+!-!-!-!-!-!-
. . .
-
8-8 ,=, '2 ,2; *=a :2 '='
,>
,=, 2, :=, 2, sz. ,=, '3:>
c.4 .A;! :'.i r.4 T.4 7.; r.4 i'.; .Fi 7.: t:;1
r.4
r.71 r., t:;1 r.4 4+ *3 a c r a
. m M .,+ &j U)
5 C L L Us.d C 0 U in
+'
mn
E 2> c a
I t . d . 6 a!- - -
n I J ~ 93 r-4 e ~3 KI e A ~3 r.4 m i13 pi ti;
b b 4 i e b b e "3 b r.1
* q
C.4 b3 N r.4ICI t..:~
. . .
-Ir.4*
m KI M*
b I J ~ 9 H. ~4 e IJT N 4@ N e @ 6 0 P f f i b 7 4 P 0 m b a b + @ 0
A 4 A pi 4 -I r.4 -I
.-I
1.3 ;.'>- - -
1.7 I;,I::# r.4 i.3 p:{
*
b 9 a N ,=, q r.: '.! 4 : f: 4A A
m D
O O f 4 c a c c ~ i '