NOT FOR QUOTATION WITHOUT THE PERMISSION OF THE AUTHORS
A
CONCEPTUAL
FXAldE FOREVALUATION OF HIGH
LEVELDECISION SUPPORT
P a t r i c k C. H u m p h r e y s J a n o s Vecsenyi
August 1985 CP-85-38
CoLLaborative P a p e r s r e p o r t work which h a s not been performed solely at t h e International Institute f o r Applied Systems Analysis and which h a s r e c e i v e d only limited review. Views o r opinions e x p r e s s e d h e r e i n do not necessarily r e p r e s e n t those of t h e Institute, i t s National Member Organizations, o r o t h e r organizations supporting t h e work.
INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS 2361 Laxenburg, Austria
PREFACE
This Collaborative P a p e r p r e s e n t s r e s e a r c h p e r f o r m e d u n d e r t h e a u s p i c e s of t h e I n t e r a c t i v e Decision Analysis (IDA) P r o j e c t in t h e Systems a n d Decision Sci- e n c e s P r o g r a m . The work r e p o r t e d h e r e e x t e n d s p r e v i o u s methodological s t u d i e s of t h e IDA P r o j e c t on decision s u p p o r t systems t o encompass c o n s i d e r a t i o n s of problem s t r u c t u r i n g a n d knowledge r e p r e s e n t a t i o n , which are typically i m p o r t a n t f o r s t r a t e g i c planning a n d s u p p o r t of high level decision makers.
A l e x a n d e r K u r z h a n s k i S D P r o g r a m L e a d e r
CONTENTS
I INTRODUCTION
I1 DEVELOPMENT OF T H E CONCEPTUAL FRAMEThrORK
I11 REVIEW OF T H E C U R R E N T STATE OF AVAILASLE DECISIOK THEORY AND R E S E A R C H IN REGARD TO STXUCTERING DECISION PROBLEMS
I V LEVELS OF REPRESENTATION OF DECISION PROBLEMS
V ROLES AND MOTIVATIOKS I N T H E DECISION XAKING P R O C E S S VI CASE S T U D I E S ORGANIZED WITHIN T H E CONCEPTUAL FRAME VII A SUPPORTING CATALOGUE OF METHODS FOR DECISION PROBLEM
STRUCTURING R E F E R E N C E S
APPENDIX: IIASA COLLABORATIVE P R O J E C T ON EVALUATION OF DECISION S U P P O R T SYSTEMS
A ORGANISATION OF TI-IE PROJECT
B
OBJECTIVES AND S C O P E OF T H E P R O J E C T C EXPECTED FINAL R E S U L T SD P H A S E S OF THE P R O J E C T E TASKS OF T H E P R O J E C T
A CONCXPTUAL
FOX EVALUATIONOF HIGH
L%VELDECISION S W O X T
Patrick C. Humphreys and Janos Vecsenyi
London S c h o o l of Economics a n d P o l i t i c a l S c i e n c e
S u r e a u of Systems Analysis Xungarian State Office of Technical Development
I. INTRODUCTION
High l e v e l m a n a g e r s in s t a t e administrations a n d in t h e e x e c u t i v e b o a r d s of l a r g e companies are, a n d will continue t o b e , f o r c e d t o make decisions in newiy- o c c u r r i n g , n o n - r e p e a t e d s i t u a t i o n s w h e r e t h e s t r u c t u r e of t h e problem i s , of n e c e s s i t y , initially u n c l e a r (e.g. making p l a n s f o r developing new a r e a s of activi- t y , developing new p r o d u c t mixes, s e l e c t i n g between competing r e s e a r c h p r o j e c t s , e t c . ) .
These so-called i l l - s t r u c t u r e d complex decision p r o b l e m s c a n n o t b e solved in e v e r y c a s e by using t r a d i t i o n a l "in-house" decision making p r o c e s s e s a n d methods d u e to:
( a ) t h e complezity of t h e p r o b l e m s , i.e. t o o many conditions, c o n s t r a i n t s , a n d c o n s e q u e n c e s must b e simultaneousiy c o n s i d e r e d (e.g. l a c k of r e s o u r c e s , m a r ~ e t competition, competing i n t e r e s t g r o u p s , e t c . ) ;
(b) t h e u n c e r t a i n t y r e l a t e d t o t h e o b j e c t i v e s a n d p r e f e r e n c e s of t h o s e con- c e r n e d , a n d r e g a r d i n g e x t e r n a l conditions, e t c . ; a n d
( c ) t h e Lack of avaiLabLe i n f o r m a t i o n c o n n e c t e d with t h e complexity of t h e p r o b - lems, t h e u n c e r t a i n t y , a n d th,e problem-solving methods themselves.
The limitations of t r a d i t i o n a l methods in handling p r o b l e m s with t h e s e c h a r a c - t e r i s t i c s o f t e n not e v i d e n t t o t h e decision maiters, who are t h e "problem owners"
(Checkland, 1981), s o t r a d i t i o n a l p r o c e s s e s a n d methods a r e q u i t e o f t e n used in ill-defined problem s i t u a t i o n s . A s a c o n s e q u e n c e , t h e r e i s n o r e g u l a r demand f o r t h e implementation of decision s u p p o r t systems, i n t e r a c t i v e decision making o r methods in s t r a t e g i c decision making. This p a p e r p r o v i d e s t h e c o n c e p t u a l f r a m e
f o r a c o l l a b o r a t i v e p r o j e c t seeking some answers t o t h e question of why t h i s is s o . The p r o j e c t is based on a c o n t a c t between IIASA a n d tine Hungarian Xational Xember Organization ( t h e Bureau of Systems Analysis of t h e Hungarian S t a t e Office f o r Technical Development). Brief d e t a i l s of t h e organization of t.he p r o j e c t a r e given in t h e Appendix.
Quite a number of decision s u p p o r t models a n d s o f t w a r e pacitages a t t e m p t t o provide a s s i s t a n c e with t h e solution of s u c h decision problems, b u t many of them a r e not r e a l l y used by t h e a c t u a l decision m a k e r s , f o r a number of r e a s o n s : (i) some of them a r e too a r t i f i c i a l , using models and language t h a t a r e too a b s t r a c t , and a r e difficult f o r top-level decision m a k e r s t o u n d e r s t a n d ; (ii) some of t h e models a n d methods do n o t c o n s i d e r t h e decision makers' own p r e f e r e n c e s and judgements; and (iii) a number of them a r e not i n t e r a c t i v e o r c o o p e r a t i v e , so t h a t t h e decision m a k e r and t h o s e c o n c e r n e d d o n o t i n t e r a c t during t h e decision-making p r o c e s s , e i t h e r with e a c h o t h e r o r with t h e decision s u p p o r t system chosen, o r (iv) on t h e c o n t r a r y , some of t h e methods demand t h e a c t i v e p a r t i c i p a t i o n of t h e p a r - t i e s involved in decision making in ways t h e y find inconvenient, though, f o r exam- ple, revealing conveniently hidden assumptions, motivations, e t c .
II. DEXELOPMI3NT OF THE CONCEPTUAL
FRAMEWORKMost DSSs developed and used t o d a t e have been c o n s t r u c t e d f o r occasional clients. In t h e f u t u r e , a n i n c r e a s e d demand f o r methods of a p p r o a c h i n g ill- s t r u c t u r e d problems c a n b e e x p e c t e d . Two main a p p r o a c h e s may b e identified.
One a p p r o a c h involves developing methodologies a n d a conscious a w a r e n e s s of decision-making methods within organizations, s o t h a t t r a i n e d decision m a k e r s may e x p e r i m e n t with DSSs f o r s t r u c t u r i n g ill-defined problems independently of o t h e r individual o r g r o u p decision-making p r o c e d u r e s . The o t h e r a p p r o a c h invoives e m - ploying decision a n a l y s t s from outside t h e organization c o n c e r n e d t o help s t r u c - t u r e t h e decision problem and supply t h e n e c e s s a r y p r o c e d u r e s and methods.
F o r c r e a t i n g a c o n c e p t u a l framework t o inform r e s e a r c h within t h i s a p p r o a c h of t h i s p r o j e c t t h e following c o n c e p t s have been c o n s i d e r e d t o b e important.
1. R e q u i s i t e D e c i s i o n Modelling.
Phillips (1982) d e s c r i b e s t h e c r i t e r i a r e q u i r e d to develop a r e q u i s i t e decision model as follows: "It i s n e c e s s a r y to involve a l l t h o s e who a r e in some way r e s p o n s i b l e f o r a s p e c t s of t h e decision in t h e development of t h e r e q u i s i t e model. The p r o c e s s of building t h e model is i t e r a t i v e and consultative, and when no new intuitions e m e r g e a b o u t t h e problem, t h e model i s c o n s i d e r e d t o
b e requisite."
Ensuring r e q u i s i t e decision modelling r e q u i r e s p s y c n o l o g i c a l v a l i d a t i o n of t h e decision method, d e s c r i b e d by Larichev (1984). In p a r t i c u i a r , psycnoiogi- c a l validation r e q u i r e s t h a t t h e o p e r a t i o n s p r e s c r i b e d within a system aiding decision making at a n y level must match t h e information p r o c e s s i n g capabili- t i e s of t h e u s e r at t h a t level, and t h e language h e o r s h e empioys in e x e r c i s - ing t h e s e capabilities.
2. The cievelopment of l e v e l s of r e p r e s e n t a t i o n of d e c i s i o n p r o b l e m s , within which t h e t h e o r e t i c a l basis f o r t h e problem s t r u c t u r i n g language and i n t e r a c - tive modules a r e embedded and e x p l i c a t e d , t o g e t h e r with a portfolio of exam- p l e s illustrating i t s p r a c t i c a l applications (Humphreys, 1984, 1985).
3 . The r o l e s a n d m o t i v a t i o n s of t h e p a r t i c i p a n t s i n the d e c i s i o n - m a k i n g p r o - c e s s as a n a l t e r n a t i v e basis f o r t h e selection o r design of decision s u p p o r t methods (Vari and Vecsenyi, 1984).
m.
REXIEXOF
THE CURXENT STATEOF AVAILABLE
DECISION THEORY AND RESEARCH IN REGARD TO STRUCTUmG DECISION PROBLEXSDina Berkeley and P a t r i c k Humphreys p r e p a r e d t h i s review d u r i n g 1981-82 t o g e t h e r with a n assessment of how t h e body of published r e s e a r c h o n " h e u r i s t i c s and biases" u s e s o r misuses t h i s material. The review was published in Acta P s y c h o l o g i c a . u n d e r t h e t i t l e "Structuring decision problems and t h e 'bias h e u r i s - tic"' (Berkeley & Humphreys, 1982). In t h a t p a p e r , Berkeley & Humphreys identi- fied s e v e n d i f f e r e n t t y p e s of u n c e r t a i n t y which h a v e t o b e c o n s i d e r e d in g e n e r a t - ing a r e p r e s e n t a t i o n of a decision problem, showing how f o u r of t h e s e w e r e t a k e n explicitly into a c c o u n t in models within t h e p r o v i n c e of decision t h e o r y , viz.:
(i) Uncertainty a b o u t t h e p r o b a b i l i t i e s of outcomes of subsequent e v e n t s , condi- tional on what h a s p r e c e d e d them in t h e act-event sequence between immediate a c t s and consequences;
(ii) Uncertainty a b o u t t h e p r o b a b i l i t i e s of outcomes of subsequent e v e n t s condi- tional on t h e o c c u r r e n c e of o t h e r e v e n t s e x t r a n e o u s t o t h e s e q u e n c e in (i);
(iii) Uncertainty a b o u t how t o i n c o r p o r a t e p r i o r information (e.g. r e s u l t s of p r i o r sampling, b a s e r a t e in a r e f e r e n c e population) in determining t h e probability of a subsequent e v e n t ;
(iv) Uncertainty a b o u t how t o conceptualize t h e worth of consequences: assessing a consequence's utility r e q u i r e s t h e g e n e r a t i o n of a single number d e s c r i b i n g i t s (holistic) worth. When more t h a n one c r i t e r i o n of "worth" i s involved, un-
c e r t a i n t y c a n a r i s e a b o u t how to combine t h e s e c r i t e r i a .
B e r k e l e y & IIumpnreys identified v a r i o u s p r o b l e m s a s s o c i a t e d with t h i s ap- p r o a c h , showing how i t s adoption h a s led to d e f i c i e n c i e s in t h e v a s t m a j o r i t y of r e s e a r c h a s s o c i a t e d with it. E f f e c t s identified in t h a t r e p o r t were availability of t a s k s , s u b j e c t s a n d e x p l a n a t i o n s ; r e p r e s e n t a t i v e n e s s of findings; a n d a n c h o r i n g a n d adjustment of e x p l a n a t i o n s .
I t i s c l e a r t h a t f o u r lines of development in both t h e o r y a n d methodology i s u r g e n t l y needed if t h e s e implications a r e to b e followed up. These t h r e e l i n e s con- c e r n :
1. The development of p r o b l e m s t r u c t u r i n g methods;
2. S p e c i f i c a t i o n of knowledge r e p r e s e n t a t i o n in f o r m s a p p r o p r i a t e f o r s u p p o r t - ing t h e o p e r a t i o n of t h e s e methods;
3. S p e c i f i c a t i o n of l e v e l s of a b s t r a c t i o n in decision making t a s k s , witn e a c h r e - l a t e d t o (i) l e v e l of knowledge r e p r e s e n t a t i o n a p p r o p r i a t e f o r t h e t a s k , a n d (ii) a n a p p r o p r i a t e l e v e l of r e s p o n s i b i l i t y in a n o r g a n i z a t i o n a l c o n t e x t .
These t h r e e l i n e s of development a r e intimately i n t e r r e l a t e d , as d e s c r i b e d below.
EY.
LEVELS OF REPRESENTATION OF DECISION PROBLEXSDeveloping problem s t r u c t u r i n g c a l c u l i led t o a w a r e n e s s t h a t a l l decision mak- ing calculi need to b e embedded within a framework of qualitatively d i s t i n c t levels of knowledge r e p r e s e n t a t i o n (Humphreys dc B e r k e l e y , 1983). The p a p e r o n Han- dling u n c e r t a i n t y : Levels of r e p r e s e n t a t i o n of decision problems (Humphreys &
B e r k e l e y , 1 9 8 4 a ) g i v e s a d e t a i l e d a c c o u n t of t h e c h a r a c t e r i s t i c s of f i v e l e v e l s of r e p r e s e n t a t i o n within decision problems. I t d e s c r i b e s r e l a t i o n s between t h e l e v e l s , r e q u i r e m e n t s f o r decision s u p p o r t at e a c h l e v e l , a n d implications of t h e multi-level s c h e m e f o r s u p p o r t i n g o r g a n i z a t i o n a l decision making.
Achieving r e q u i s i t e decision modelling (Phillips, 1982; 1984) in a n y p a r t i c u l a r situation may r e q u i r e t h a t t h r e e remaining t y p e s of u n c e r t a i n t y a r e a l s o r e s o l v e d . Humphreys a n d B e r k e l e y (1982) identified t h e s e a s :
(v) P r o c e d u r a l u n c e r t a i n t y , i.e. " u n c e r t a i n t y c o n c e r n i n g means t o handle o r p r o - c e s s t h e d e c i s i o n , e . g . specifiying r e l e v a n t u n c e r t a i n t i e s , what information t o s e e k a n d w h e r e , how t o invent a l t e r n a t i v e s a n d assess c o n s e q u e n c e s , e t c . "
( H o g a r t h , Michaud & Mery, 1980);
(vi) Uncertainty a b o u t how t h e decision m a k e r will f e e l , a n d wish t o a c t , having a r - r i v e d at a s u b s e q u e n t a c t c h o i c e point a f t e r intervening e v e n t s h a v e unfolded in r e a l time;
(vii) U n c e r t a i n t y a b o u t t h e e x t e n t o n e p o s s e s s e s a g e n c y f o r inducing c h a n g e s in t h e p r o b a b i l i t i e s of s u b s e q u e n t e v e n t s (conditional on a c t s y e t t o b e t a k e n , as in (i) a b o v e ) t h r o u g h being a b l e t o a l t e r r e l a t i o n s between s t a g e s of t h e world (Savage, 1954).
All t h e s e t h r e e t y p e s of u n c e r t a i n t y a r e usually p r e s e n t in i l l - s t r u c t u r e d deci- sion making s i t u a t i o n s , t h e f o c u s of t h i s p r o j e c t . Resolving t h e s e u n c e r t a i n t i e s im- plies in p r a c t i c e t e m p o r a r i l y "fixing" t h e way in which a decision problem i s locat- e d . Difficulties in resolving u n c e r t a i n t i e s in doing t h i s were d e s c r i b e d a n d t h e need f o r a problem s t r u c t u r i n g calculus was identified t o a r t i c u l a t e t h i s p r o c e s s .
The a l t e r n a t i v e " h e u r i s t i c s and biases" a p p r o a c h t o t h e study of human deci- sion making was c o n t r a s t e d as imposing s t r u c t u r e , assuming common u n a e r s t a ~ d i n g . This l a t t e r a p p r o a c h was shown t o involve (i) t h e "naturalization" of t h e small world in which t h e decision problem i s l o c a t e d , a n d (ii) t h e utilization of normative models as "ideal types", leading t o t h e u s e of t h e "bias" argument in discussing sub- jects' p e r f o r m a n c e in decision t a s k s .
The k e y a d v a n t a g e of t h e "levels" framework i s t h a t i t p e r m i t s i n t e g r a t i o n a c r o s s levels: resultis of o p e r a t i o n s at a h i g h e r level define t h e c o n s t r a i n t s at lower levels. Moreover t h e s e levels of problem r e p r e s e n t a t i o n m i r r o r Jaques' (1976) a c c o u n t of l e v e l s of a b s t r a c t i o n of organizational r o l e s , from s h o p a n d of- f i c e f l o o r (level 1 ) t o managing d i r e c t o r (level 5). A comparison of t h e l e v e l s within Jaques' scheme a n d r e q u i r e m e n t f o r e f f e c t i v e decision s u p p o r t at e a c h l e v e l i s summarized in Table 1 ( r e p r i n t e d from Humphreys, 1984). We c o n s i d e r t h e findings from t h i s comparison t o b e of c r u c i a l importance f o r t h e design and implementation of multilevel decision s u p p o r t systems (DSSs).
Decision aiding s o f t w a r e t o o l s with c a p a b i l i t i e s at t h e f i r s t t h r e e levels are comparatively well developed. These comprise level 1 systems aiming at providing
"best assessments" (e.g. most management information systems; systems f o r elicit- ing a n d c a l i b r a t i n g p r o b a b i l i t y assessments), level 2 systems e x p l o r i n g h y p o t h e s e s r a t h e r t h a n r e p o r t i n g "facts" (e.g. most e x p e r t systems) a n d l e v e l 3 systems capa- ble of c a p t u r i n g a n d editing t h e s t r u c t u r e of a n a s p e c t of a problem. We identified a complete a b s e n c e in p r a c t i c a l a p p l i c a t i o n s of DSS possessing t h e ability of t o work with t h e decision m a k e r ' s own problem s t r u c t u r i n g language in linking t o a p - p r o p r i a t e l e v e l 3 systems; a b i l i t y t o d e t e r m i n e t h e bounds of a problem t h r o u g h s c e n a r i o g e n e r a t i o n ) . The problem h e r e i s n o t simply a f a i l u r e of automated DSS
TABLE 1: Comparison of Demand characteristia of tasks facing personnel having responsibilities a r a given organisational level with structuring capabilities required in representing decision problems at
that level
(characteristics of levels 8 to 10 can, in theory, be ascertained by extrapolation from levels 3 to 5 respectively)
T h e span .- c Organisational inherent in
5
.-
level in problem*
Level employment representation$
number herarchy. at given level...
q = s az Z $ 7 Chairman MID of 20-50 years
3 s - corporate group;
E m ? head of large e q P
0 5 s
.-
government4: &
department.- 2 8
=
554Demand characteristics of tasks facing personnel with responsibility at given level Anticipat~on of changes it!
sociological, technological, demographic and political developments; leading corporate strategic development to meet them.
Structuring capabilities required in representing
decision problems at given Number of existing level (decision suppon must DSS incorporating also include capabilities at support formalised at all lower levels). given level.
Isomorphic with level 2. None except can conduct
sensitivity analysis.
simulating changes in level 5 representations; assessing their impact within cultural StNCtUrC.
d ~e
-
=
a * u s 6 Corporate group/ 10-20 years Coordination of social and Isomorphic with level 1. None 2z
8 2 . 8
sector executive. theoretical systems; except each node is now a'g as
translation of corporate level 5 problemV) m u strategic development into representation within fixed
business direction. cultural structure.
5 Corporate 5-10 years Problem not dealt with in Articulation of principles for None subsidiary1 context set wholly from conditional (goal) closing of
enterprise above; can modify bound- an open system, andlor re- managing director aries of business within opening of a conditionally
policy i.e, define work closed system (e.3. through
..
c-
.s system. x e n a n o generation).- c
.=
.a r 4 General 2-5 years Detachment from specific Selecting/interfacing Very few
= w
U 0
u
-
management. (of cases, seeing them capabilitv between structural ( ~ r o t o t ~ p e s )5 ; e.g. development. representative examples of types (requires use of problem
-
a production or issues calling for develop- structuring language).4 2
*D sales, within work ment of a system.
5 "U system).
e 5 V ;
.-
4 ;=
3 Department 1-2 years Control of trend of tasks Re-structuring capability A fewE % E
ManageriaY , and problems arising. Extra- within single fixed structural-
principal special~st . polat~on from trend to ways type (e.3. attribute generation.Z
3=
.a 5 Z of formulating problems. ~n multi-attribute model).
4 : "
- .= - 2 Front-line 3 months Formal operational, can Man~pulation of data on one Many s . 5 2 manageriaY to 1 year anticipate changes in tasks variable at a time within fixed
3 2 9
> professional. due to any one of: demand. structure (c.g. sensitivity
s u m object, production resource, analys~s).
2 2 % pathway. or pathway
resource.
1 Shop and office less than Concrere operarional limtted Estimation of values at nodes ,Many floor. 3 months to tasks concretely and within fixed structure (e.g.
phyxially at hand. information retrieval system).
Abstracted from E. Jaques: Free Enterprise, Fair Employment (321
design a t h i g h e r levels. Humphreys & Berkeley (1984) conclude:
"It is not a c t u a l l y advisable t o a t t e m p t t o formalize level 5 s c e n a r i o gen- e r a t i o n t e c h n i q u e s a n d level 4 probiem s t r u c t u r i n g languages into au- tomated decision s u p p o r t systems. A t level 5, decision m a k e r s ' s c e n a r i o s need t o b e e x p l o r e d r a t h e r t h a n f i t t e d i n t o formal s t r u c t u r e s . A t level 4 i t i s b e t t e r t o develop t e c h n i q u e s f o r t h e psychological validation of t h e decision m a k e r ' s own problem s t r u c t u r i n g language t h a n t o t r y t o invent a universal problem s t r u c t u r i n g language t h a t will h a v e t o b e t a u g h t from s c r a t c h t o high level decision makers." (p. 3 0 )
These findings point towards t h e following two-fold r e s e a r c h s t r a t e g y :
1 . To assemble a n d s t a n d a r d i z e s p e c i f i c a t i o n s on t h e elements of a l i b r a r y of problem s t r u c t u r i n g methods, e a c h method being programmed as a complete s o f t w a r e module implementing a level 3 problem s t r u c t u r i n g calculus a n d sup- p o r t i n g l e v e l 2 a n d l e v e l 1 p r o c e d u r e s . The a p p r o p r i a t e specification of t h e c o n t e n t s of a n y l i b r a r y of t h i s t y p e held by a p a r t i c u l a r decision aiding/decision a n a l y s i s g r o u p o r institution will d e p e n d upon t h e t y p e s of de- cision problems t o b e handled with i t s s u p p o r t ) .
A major t e c h n i c a l o b j e c t i v e f o r t h e c o l l a b o r a t i v e work within t h i s p r o j e c t will t h u s b e b e s t t o r e s e a r c h a framework f o r a c a t a l o g u e of s p e c i f i c a t i o n s of de- cision problem s t r u c t u r i n g methods a n d s o f t w a r e , f o r publication by IIASA.
This c a t a l o g u e will b e invaluable f o r DSS r e s e a r c h e r s , d e v e l o p e r s a n d imple- m e n t e r s , as i t will p r o v i d e a unique a n d comprehensive information r e s o u r c e when specifying, a s s e s s i n g a n d i n t e r f a c i n g i n t e r a c t i v e decision aiding methods which h a v e t h e c a p a b i l i t y t o s e r v e as elements in decision problems a n d s t r u c - t u r i n g l i b r a r i e s of t h e t y p e outlined a b o v e .
R e s e a r c h is e s s e n t i a l which will p r o v i d e t h e b a s i s f o r t h e development, valida- tion a n d implementation in p r a c t i c e of DSS providing e f f e c t i v e s u p p o r t at h i g h e r l e v e l s (i.e. l e v e l 4 a n d a b o v e ) w h e r e i t is e s s e n t i a l t o s u p p o r t t h e na- t u r a l problem s t r u c t u r i n g languages used by decision m a k e r s a n d by skilled decision a n a l y s t s working i n t e r a c t i v e l y with them.
V. ROLES
AND
MOTIVATIONS XXTHE
EECISIOW MAKING P R O C E S SIn designing a n d using new decision a n a l y t i c methods, n o t only t h e c h a r a c - t e r i s t i c s of t h e decision problem, b u t a l s o t h e r o l e s a n d motivations of t h e p a r t i c i - p a n t s in t h e decision-making p r o c e s s need t o b e c o n s i d e r e d .
Decision making in a n o r g a n i s a t i o n a l c o n t e x t implies a p r o c e s s with s e v e r a l a c t o r s . A s a b a s i s f o r understanding t h e motivations a n d problems of individuals as members of o r g a n i s a t i o n s Vari a n d Vecsenyi (1984) h a v e found i t useful t o investi- g a t e t h e i r r o l e s in comparison with t h o s e of o t h e r p a r t i c i p a n t s in t h e decision making p r o c e s s . The p r i n c i p l e r o l e s t h a t t h e y identified in t h e i r p r e v i o u s s t u d i e s (Humphreys, Vari, Vecsenyi, 1982; Vari, Vecsenyi, 1983) a r e t h o s e of d e c i s i o n m a k e r s , p r o p o s e r s , e x p e r t s , a n d p a r t i c i p a n t s p r i m a r i l y c o n c e r n e d w i t h i m p l e - m e n t a t i o n .
1. The d e c i s i o n m a k e r s h a v e e x e c u t i v e power t o d e f i n e t h e u s e of o u t p u t s of t h e decision making.
2. The p r o p o s e r s h a v e power t o make recommendations t o t h e decision makers.
3. The e z p e r t s ' function i s t o supply inputs t o t h e c u r r e n t l y modelled problem s t r u c t u r e .
4. Those c o n c e r n e d w i t h i m p l e m e n t a t i o n play a n a c t i v e r o l e in t h e r e a l i s a t i o n of t h e a c c e p t e d solution. In s i t u a t i o n s w h e r e a decision s u p p o r t system o r de- cision aiding t e c h n i q u e s are applied two f u r t h e r r o l e s c a n b e defined. These are:
5. The c l i e n t who i n i t i a t e s t h e decision s u p p o r t .
6 . The c o n s u l t a n t s o r d e c i s i o n a n a l y s t s who a d v i s e on methods of problem r e p r e s e n t a t i o n a n d decision making p r o c e d u r e s . They are in a position t o f a - c i l i t a t e t h e c o l l a b o r a t i o n of t h e p a r t i e s involved, t h e communication of t h e r e s u l t s , e t c .
Another r e l a t e d a p p r o a c h t o t h e identification of r o l e s of p a r t i c i p a n t s w a s d e s c r i b e d by Checkland (1981) from a systems analysis p e r s p e c t i v e . Checkland identified t h e following r o l e s :
1. C l i e n t : He who wants t o know o r d o something a n d commissions t h e s t u d y . The implication i s t h a t h e c a n c a u s e something .to h a p p e n as a r e s u l t of t h e s t u d y . 2. Decision t a k e r : The r o l e p l a y e r in a human a c t i v i t y system who c a n a l t e r i t s
c o n t e n t (its a c t i v i t i e s ) a n d t h e a r r a n g e m e n t s within t h e systems (subsystems) a n d who c a n d e c i d e r e s o u r c e allocation within t h e system.
3 . Problem owner: He who h a s a feeling of u n e a s e a b o u t a s i t u a t i o n , e i t h e r a s e n s e of mismatch between 'what is' a n d 'what might b e ' or a vague feeling t h a t things could b e b e t t e r a n d who wishes something were d o n e a b o u t it. The problem owner may n o t b e a b l e t o d e f i n e what h e would r e g a r d as a 'solution!, a n d may not b e a b l e to a r t i c u l a t e t h e feeling of u n e a s e in a n y p r e c i s e way.
4 . P r o b l e m s o l v e r : A p e r s o n o r p e r s o n s anxious to b r i n g a b o u t improvement in a problem s i t u a t i o n .
Despite t h e a p p a r e n t similarities, some d i f f e r e n c e s are a l s o obvious. The most important p o i n t i s t h a t in C h e c k l a n d ' s system t h e d i f f e r e n t i a t i n g f a c t o r i s t h e a c - tors' role as r e l a t e d t o t h e p r o b l e m c o n t e n t . In o u r c a s e , h o w e v e r , i t i s t h e r o l e played in t h e d e c i s i o n m a k i n g / p r o b l e m s o l v i n g p r o c e s s . From t h e point of view of o u r assumptions a b o u t s u p p o r t i n g d e c i s i o n m a k i n g a c t i v i t i e s t h e l a t t e r a p p r o a c h seems t o b e more r e a s o n a b l e .
Vari a n d Vecsenyi came to t h e conclusion t h a t t h e decision aiding methods should s u p p o r t t h e whole d e c i s i o n making p r o c e s s r a t h e r t h a n s u p p o r t i n g only t h e d e v i c e itself in t h e way s u g g e s t e d by t r a d i t i o n a l decision t h e o r y . They a r g u e t h a t decision making in o r g a n i z a t i o n a l c o n t e x t s c o m p r i s e s n o t only a s e r i e s of a c t i v i t i e s a n d a t t i t u d e s to t h e p r o b l e m c o n t e n t a n d to t h e decision making p r o c e s s .
According t o t h e findings of Vari a n d Vecsenyi, most of t h e motivational f a c - t o r s c a n b e t r a c e d t o t h e d e s i r e f o r c o n t r o l l i n g ( t h r o u g h u n d e r s t a n d i n g a n d i n f l u e n c i n g ) t h e t h r e e p h a s e s of t h e d e c i s i o n m a k i n g p r o c e s s , i.e. gaining o r maintaining c o n t r o l o v e r (i) t h e p l a n n i n g of t h e d e c i s i o n (problem s t r u c t u r i n g , analysing a n d p r o p o s a l formulating), (ii) t h e c h o i c e p r o c e s s a n d (iii) t h e i m p l e m e n - t a t i o n of t h e decision.
One of t h e most f r e q u e n t motivating f a c t o r s f o r t h e decision m a k e r s in apply- ing decision s u p p o r t methods i s to i n c r e a s e t h e i r c o n t r o l of t h e d e c i s i o n m a k i n g . A s pointed o u t by P f e i f f e r (1981), t h e c o n t r o l of t h e decision making i s o n e of t h e main s o u r c e s of p o w e r in o r g a n i s a t i o n s . In s e l e c t i n g a method which c a n p r o v i d e a d e q u a t e decision s u p p o r t we c o n s i d e r h e r e t h e d e c i s i o n m a k e r as a c l i e n t . In t h i s c a s e t h e a n a l y s t ( c o n s u l t a n t ) should h e l p him to i n c r e a s e t h e c o n t r o l o v e r t h e de- cision making. A m o r e p r o f o u n d a n a l y s i s of t h i s f a c t o r will, h o w e v e r , c a l l a t t e n t i o n t o t h e n e e d f o r distinguishing between c o n t r o l o v e r t h e d i f f e r e n t p h a s e s of t h e de- cision making p r o c e s s .
The decision m a k e r ' s c o n t r o l o v e r t h e p l a n n i n g i s maximal when a l l t h e s t e p s are t a k e n by himself (e.g
.
definition of g o a l s , o p t i o n s , outcomes, c r i t e r i a , t r a d e - o f f s , e v a l u a t i o n s of o p t i o n s a n d t h e a g g r e g a t i o n of d a t a t h r o u g h decision r u l e s ,e t c . ) . This implies a situation c h a r c t e r i s e d as i n d i v i d u a l decision making within a n organisational c o n t e x t . Larichev (1984) calls t h i s "holistic c h o i c e " in c o n t r a s t t o c a s e s in which t h e decision maker h a s n o t enough e x p e r t i s e o r information f o r estimating t h e a l t e r n a t i v e solutions on his own.
This t y p e of individual decision making o c c u r s r e l a t i v e l y r a r e i y in organisa- tional c o n t e x t s , and g e n e r a l l y in small-scale problems. The complexity of o r g a n i s a - tional problems makes i t n e c e s s a r y f o r t h e decision m a k e r t o invite e x p e r t s t o sup- plement t h e information lacking f o r t h e decision. E x p e r t s a r e employed most fre- quently f o r giving assessments a b o u t e v e n t s and outcomes, while t h e definition of t h e p r e f e r e n c e s t r u c t u r e i s likely t o remain u n d e r t h e decision m a k e r ' s c o n t r o l . This i s what L a r i c h e v c a l l s " c r i t e r i a - e x p e r t s choice". This, however, implies a de- c r e a s e in t h e decision m a k e r ' s c o n t r o l o v e r t h e planning, and a wider r a n g e of possibilities f o r them t o s u p e r v i s e t h e p r o p o s a l s based o n questioning t h e e x p e r t s ' judgements.
The decision m a k e r ' s c o n t r o l o v e r t h e planning i s minimal when t h e decision m a k e r does not p a r t i c i p a t e in i t at all. In t h i s c a s e , only e x p e r t s a r e involved.
However, in using t h e r e s u l t s of t h i s kind of decision s u p p o r t t h e decision m a k e r h a s maximal f r e e dom s i n c e e i t h e r (i) no p r o p o s a l will h a v e b e e n p u t f o r w a r d by t h e e x p e r t s , d u e t o lack of information a b o u t t h e decision m a k e r ' s p r e f e r e n c e s o r (ii) t h e r e will h a v e b e e n a p r o p o s a l r e f l e c t i n g t h e e x p e r t ' s p r e f e r e n c e s . Such c a s e s o f t e n o c c u r i n p r a c t i c e .
A s f a r as t h e c o n t r o l o v e r t h e c h o i c e i s c o n c e r n e d , t h e situation i s quite dif- f e r e n t . Lock (1983) pointed o u t t h a t "it i s r e m a r k a b l y difficult f o r a manager in a supposedly ' r a t i o n a l ' r o l e t o a r g u e against a supposedly rationally d e r i v e d solu- tion t o a decision problem". Consequently, t h e more involved is t h e decision m a k e r in t h e planning p r o c e s s , t h e more uncomfortable h e will f e e l a b o u t revising t h e s e r e s u l t s which are opposed t o his intuitions.
Beside d e s i r e f o r c o n t r o l o v e r t h e planning p r o c e s s o r o v e r t h e c h o i c e , t h e t h i r d most important f a c t o r motivating t h e decision m a k e r ' s u s e of decision a i d s i s d e s i r e f o r t h e c o n t r o l o v e r t h e i m p l e m e n t a t i o n p r o c e s s . F o r t h e decision maker, i t i s important in t h i s p h a s e (i) t o understand t h e opinions and t h e f u t u r e behaviour of t h o s e c o n c e r n e d with t h e implementation, and (ii) t o e n s u r e t h a t t h e p r e s c r i b e d solutions are a c c e p t e d .
Understanding and considering t h e opinions of t h o s e c o n c e r n e d with t h e im- plementation of a decision may obviously i n c r e a s e 'the c h a n c e of selecting a feasi- b l e solution. One way of e n s u r i n g t h i s is t o u s e decision s u p p o r t methods which in- volve t h e s t a k e h o l d e r s ' full p a r t i c i p a t i o n . Facilitating discussions and t h e p a r t i c i -
pation in t h e formulation of t h e p r o p o s a l may e n s u r e t h e p a r t i c i p a n t s ' internali- sation of t h e solution.
Another way of promoting a c c e p t a n c e d o e s n o t n e c e s s a r i l y involve t h e p a r t i - cipation of t h o s e c o n c e r n e d with t h e implementation in t h e whole decision making p r o c e s s . I t s u f f i c e s to e x p l a i n t o them t h e final p r o p o s a l which may possibly h a v e b e e n p r o d u c e d with t h e a i d of c o n t r i b u t i o n s from e x p e r t s a n d a n a l y s t s as well as from institutions of high s t a t u s . I n t h i s c a s e t h e a c c e p t a n c e of t h e r e s u l t s c a n b e promoted t h r o u g h a s e c o n d well known psychological mechanism f o r adjustment, i.e. t h e ictentzfication of t h o s e c o n c e r n e d with t h e implementation with t h e p e r - s o n s who g e n e r a t e d t h e solution.
In summary, i t must b e emphasized t h a t d i f f e r e n t motivations c a n usually b e s e r v e d by d i f f e r e n t methods which
-
if a p p l i e d simultaneously-
c a n complement e a c h o t h e r , b u t c a n b e c o n t r a d i c t o r y a s well. Given aTrrareness of s u c h conflicting r e q u i r e m e n t s , a d e c i s i o n a n a l y s t o r DSS d e s i g n e r h a s t o d e c i d e in e a c h p a r t i c u l a r c a s e (i) which a c t o r in which r o l e on which l e v e l with which motivations should b e s e r v e d , (ii) which a c t i v i t i e s in t h e decision making p r o c e s s should b e a i d e d , a n d (iii) which methods a n d r e l a t e d computer-based s o f t w a r e modules if a n y should b e a p p l i e d a s a n a i d to e a c h a c t i v i t y .VI.
C A S E S T U D I E S ORGANIZED WITHINTHE
CONCEPTUAL FRAME;LN REAL WORLD HIGH LEXEL DECISION
MAKING
Testing t h e r e l e v a n c e of t h e c o n c e p t u a l f r a m e w o r k s d e s c r i b e d a b o v e r e q u i r e s a r e a l world d e c i s i o n making l a b o r a t o r y f o r t h e i r successful implementation. By t h i s w e mean a f a c i l i t y w h e r e t h e a c t u a l decision m a k e r s c a n meet to work t o g e t h e r o n t h e a c t u a l d e c i s i o n problem t h e y a r e c u r r e n t l y facing s u p p o r t e d by d e c i s i o n a n a l y s t s a n d decision aiding systems. In aiding decision making d i c t a t e s a b o v e l e v e l 3 , t h e p r e s e n c e of a s p e c i a l i s t decision a n a l y s t i s e s s e n t i a l , a n d problem formula- tion a n d solution at t h e s e l e v e l s r e q u i r e s t h e u s e of g r o u p p r o c e s s e s which c a n n o t b e formalized i n t o automated systems. However, t o b e e f f e c t i v e , high l e v e l decision making must include a p p r o p r i a t e s u p p o r t a t a l l lower l e v e l s , implemented h e r e t h r o u g h t h e s e l e c t i o n of a p p r o p r i a t e decision aiding systems from a l i b r a r y of problem s t r u c t u r i n g methods. The n a t u r e of t h e r e s e a r c h w e h a v e d e s c r i b e d r e - q u i r e s also t h a t k e y a s p e c t s of t h e i n t e r a c t i o n s between t h e decision m a k e r s , deci- s i o n a n a l y s t s a n d decision aiding systems must a l s o b e monitored in forming a n evaluation of t h e u s e of any p a r t i c u l a r decision aiding t e c h n i q u e o n DSS. However, t h i s monitoring p r o c e s s must n e v e r become i n t r u s i v e or a p p e a r r e s t r i c t i v e t o t h e decision m a k e r s while working on t h e problem f o r r e a l r a t h e r t h a n a c t i n g as "sub-
jects" o r guinea pigs, t r y i n g to handle what would quickly become toy problems.
Decision c o n f e r e n c e s p r o v i d e a ' n a t u r a l l a b o r a t o r y ' in which t h e problem solving p r o c e s s c a n be studied, while still meeting t h e r e q u i r e m e n t s s e t o u t above.
The r e s e a r c h plan f o r t h e IIASA c o l l a b o r a t i v e p r o j e c t on Evaluation of Deci- sion S u p p o r t Systems p r o v i d e s f o r t h e p a r t i c i p a n t s in t h i s collaboration to develop c a s e studies on t h e i r own development a n d application of decision s u p p o r t systems and decision aiding methods, making u s e of t h e conceptual framework we h a v e out- lined a b o v e in ways a p p r o p r i a t e f o r e a c h case. The review and r e v i s i o n s in t h e light of t h e insight gained and comments made in subsequent g r o u p discussions of meetings of a l l t h e major p a r t i c i p a n t s in t h e p r o j e c t , t h e r e v i s e d c a s e s t u d i e s will f o r m c h a p t e r s within a book on Experiences i n DSS c o n s t r u c t i o n for Problem S t r u c t u r i n g t o b e published t h r o u g h IIASA. The book will a l s o contain a n initial c h a p t e r providing a g e n e r a l introduction and s u r v e y , and a c h a p t e r identifying im- plications f o r development and application of decision aiding techniques and deci- sion s u p p o r t systems.
W. A SUPPORTING CATALOGUE OF METHODS FOR DECISION PROBLEM STRUCTURING
A s u p p o r t i n g aim of t h i s a s p e c t of t h e r e s e a r c h within t h e f r a m e is t o assem- ble a catalogue of specifications of methods which could b e implemented as modules in a l i b r a r y of problem-structuring methods, f o r two p u r p o s e s : (i) t o i n c r e a s e t h e f r e q u e n c y with which high-level decision m a k e r s utilize t h e available problem- s t r u c t u r i n g methods in t h e solution of ill-defined decision problems, and (ii) t o gen- erate DSSs t h a t c a n help t h e a n a l y s t o r consultant to s t r u c t u r e novel problems in s u c h a way t h a t t h e decision m a k e r i s a b l e t o function effectively.
This catalogue will b e published t h r o u g h IIASA as a n information r e s o u r c e f o r DSS d e v e l o p e r s a n d decision a n a l y s t s . The e n t r i e s in t h e catalogue will b e based on s e l f - r e p o r t by t h e method d e v e l o p e r s in r e s p o n s e to a questionnaire developed in a form consistent with t h e of t h i s p a p e r . This means t h a t members of t h e p r o j e c t team f o r t h e c o l l a b o r a t i v e r e s e a r c h and IIASA will not b e a b l e t o t a k e responsibili- t y f o r t h e a c c u r a c y of t h e d e s c r i p t i o n s in t h e catalogue. Instead, t h e c a t a l o g u e will s e r v e t o i n c r e a s e a w a r e n e s s of potential methods and t o establish liaisons between method d e v e l o p e r s and method u s e r s .
Ideally, methods included in t h i s c a t a l o g u e (and s o f t w a r e implementing them) will meet t h e r e q u i r e m e n t t h a t t h e y h a v e c a p a b i l i t i e s a t e a c h of t h e following lev- els:
Level 3: r e s t r u c t u r i n g capability within a p a r t i c u l a r s t r u c t u r a l v a r i a n t o r
"frame!' (establishing new c r i t e r i a )
Level 2: assessing judgement o n a v a r i a b l e within a fixed s t r u c t u r e (e.g. "what if"
models)
Level 1 : judgement within fixed s t r u c t u r e (e.g. with information r e t r i e v a l s e r - vice)
Crucial o b j e c t i v e s which must b e achieved in assembling a n y decision problem s t r u c t u r i n g l i b r a r y consist of knowing how t o specify c h a r a c t e r i s t i c s of modules included in, o r r e q u i r e d f o r inclusion within t h e l i b r a r y at e a c h of t h e s e t h r e e lev- els. Existing modules which might b e included in t h e l i b r a r y a l s o need t o b e as- s e s s e d in terms of t h e i r c a p a b i l i t i e s at e a c h of t h e s e levels, identifying s t r e n g t h s and weaknesses, and possibilities f o r f u t u r e development.
The catalogue will a l s o contain guidelines specifying t h e c h a r a c t e r i s t i c s of modules which could potentially b e included in a n a p p r o p r i a t e l y specified l i b r a r y . These guidelines should e n a b l e a n applications r e q u i r e m e n t t o b e i n t e r s e c t e d with modules a p p r o p r i a t e l y specified.
REFERENCES
Berkeley, D . and Humphreys, P.C. S t r u c t u r i n g Decision Problems a n d t h e "Bias Heuristic". Acta P s y c h o l o g i c a , 1982, 50, 201-252.
Checkland, P . S y s t e m s t h i n k i n g , s y s t e m s p r a c t i c e . Chichester: Wiley, 1981.
Hogarth, R.M., Michaud, C, and Mery, J.L. Decision behaviour in u r b a n develop- ment: A methodological a p p r o a c h and s u b s t a n t i v e consideration. Acta P s y c h o - l o g i c a , 1980, 4 5 , 95-117.
Humphreys, P.C. Levels of r e p r e s e n t a t i o n in s t r u c t u r i n g decision problems. Jour- n a l o f A p p l i e d S y s t e m s A n a l y s i s , 1984, ll, 3-22.
Humphreys, P.C. Intelligence in decision s u p p o r t . Major P a p e r . Tenth R e s e a r c h Conference on S u b j e c t i v e Probability Utility and Decision Making, Helsinki, 1985.
Humphreys, P.C. a n d B e r k e l e y , D. Problem s t r u c t u r i n g calculi and levels of knowledge r e p r e s e n t a t i o n in decision making. In R.W. Scholz (Ed.), Decision Making U n d e r U n c e r t a i n t y . Amsterdam: North Holland, 1983.
Humphreys, P.C. a n d Berkeley, D. Handling u n c e r t a i n t y : Levels of analysis in deci- sion problems. In G.N. Wright (Ed.), B e h a v i o u r a l Decision Making: Theory a n d A n a l y s i s . New York: Plenum, 1984, in p r e s s .
Hurnphreys, P.C., Vari, A. and Vecsenyi, J. Methods f o r analysing t h e e f f e c t s of ap- plication of Decision S u p p o r t Systems in R and D decisions. Collaborative pa- p e r CP-82-69, Laxenburg, Austria: International I n s t i t u t e f o r Applied Systems Analysis, 1982.
Humphreys, P.C., L a r i c h e v , O . , Vari, A. and Vecsenyi, J . Comparative anaiysis of use of Decision S u p p o r t Systems in R and D decisions. In H. Sol (Ed.), Processes a n d Tools for Decision S u p p o r t . Amsterdam: North Holland, 1983.
J a q u e s , E.A General T h e o r y o f B u r e a u c r a c y . London: Heinemann, 1976.
L a r i c h e v , 0.1. Psychological validation of Decision Methods. J o u r n a l of A p p l i e d S y s t e m s A n a l y s i s , ll, 1984.
Lock, A. Applying decision a n a l y s i s in a n o r g a n i s a t i o n a l c o n t e x t . In Humphreys, Svenson & Vari (eds.), A n a l y s i n g a n d A i d i n g D e c i s i o n P r o c e s s e s . Amster- dam: North Holland, 1983.
P f e i f f e r , J . Power i n D r g a n i s a t i o n s . London: Pitman. 1981.
Phillips, L.D. Requisite decision modelling: A c a s e study. J o u r n a l of t h e Opera- t i o n a l R e s e a r c h S o c i e t y , 1982, 33, 301-311.
Phillips, L.D. A t h e o r y of r e q u i s i t e decision models. Acta P s y c h o l o g i c a , 1984, 56, 29-48.
Phillips, L.D. Decision s u p p o r t f o r s e n i o r e x e c u t i v e s . D a t a m a t i o n , 1985, in p r e s s . S a v a g e , L.J. The F o u n d a t i o n s o f s t a t i s t i c s . London: Wiley, 1954.
Vari, A. a n d Vecsenyi, J . Decision analysis of i n d u s t r i a l R a n d D problems: Pitfalls a n d lessons. In P.C. Humphreys, 0 . Svenson a n d A. Vari, A n a l y s i n g a n d Aid- i n g D e c i s i o n P r o c e s s e s . Amsterdam: North Holland, 1983.
Vari, A. a n d Vecsenyi, J . Selecting decision s u p p o r t methods in organizations.
J o u r n a l of A p p l i e d S y s t e m s A n a l y s i s , 1984, ll, 23-36.
APPENDIX UASil C 3 W O R A T F . T PROJXCT
ON FJALUATION O F DECISION SUPPORT SYSTEMS
A.
ORGLWSATICN CH
THE PIiOJECTThe IIASA c o l l a b o r a t i v e p r o j e c t o n Evaluation of Decision S u p p o r t systems i s designed t o p r o v i d e a c o n s t r n c t i v e f r a m e w o r ~ f o r internatiofiai c o i l a b o r a t i o n between r e s e a r c h e r s of t h e E a s t a n d West. I t i s b a s e d o n a c o n t r a c t between IIASA a n d t h e Hungarian il'ational member o r g a n i z a t i o n , t h e B u r e a u of Systems Anaiysis of t h e Hungarian S t a t e Office f o r Technical Development.
This c o n t r a c t p r o v i d e s s u p p o r t f o r t h e i n t e r n a t i o n a i c o l l a b o r a t i o n involved in t h e work d e s c r i b e d below. P a r t i c i p a n t s in t h e p r o j e c t h a v e a r r a n g e d t o fund t h e components of t h e work c a r r i e d o u t in t h e i r own institutions, (development a n d p r e p a r a t i o n of c a s e s t u d i e s , compiling t h e c a t a l o g u e , e t c . ) t h r o u g h p r o j e c t sup- p o r t e d s e p a r a t e l y within t h e i r institutions. The r e s p o n s i b i l i t y f o r a r r a n g i n g t h e nationai components of t h e work l i e s with t h e s e individual institutions a n d n o t with IIASA.
This p r o j e c t a t t e m p t s to p r o v i d e a framework f o r designing a n d s e l e c t i n g decision s u p p o r t systems (DSSs) f o r s t r u c t u r i n g ill-defined d e c i s i o n problems b a s e d on a c r o s s - c u l t u r a l c o m p a r a t i v e s t u d y e v a l u a t i n g DSSs in d i f f e r e n t coun- t r i e s . The i n t e r n a t i o n a i network of t h e p r o j e c t w a s c r e a t e d with t h e p a r t i c i p a t i o n of t h e following c o l l a b o r a t i v e g r o u p s in H u n g a r y , UK, USA a n d USSR.
In t h e S o v i e t Union a t VNIISI, P r o f e s s o r 0 . L a r i c h e v , D r . H: Moskovich a n d t h e i r c o l l e a g u e s are working o n a p p l i c a t i o n a n d development of multidimensionai scaling a n d semi-ordering methods in decision-making, a n d on t h e psychological vaiidation of DSSs.
In t h e UK at t h e LSE, P . Humphreys, L. Phiilips, S . Wooler a n d t h e i r col- l e a g u e s a r e developing a n applying i n t e r a c t i v e d e c i s i o n aiding method which p r o - vides e f f e c t i v e s u p p o r t f o r decision maiting at all levels. Lower l e v e l s u p p o r t modules h a v e b e e n programmed as i n t e r a c t i v e s o f t w a r e , while h i g h e r l e v e l s u p p o r t r e l i e s upon a n a l y s i s of decision m a ~ e r s n a t u r a l problem s t r u c t u r i n g language a n d t h e u s e of d e c i s i o n c o n f e r e n c i n g t e c h n i q u e s . At t h e M a n c h e s t e r Business School, P r o f e s s o r A. NIcCosh i s using DSS s o f t w a r e modules in s t r a t e g i c pianning applica- tions.
In t h e USA, P r o f e s s o r D. Gustafson at t h e University of Wisconsin a n d W.
Cats-Baril at t h e U n i v e r s i t y of Vermont a r e developing s e v e r a l decision a i d s , impiementing a n a n a l y s i s w h e r e t h e c o m p u t e r would guide p e o p l e i n t o thinking
a b o u t how t h e y would plan f o r implementation e f f o r t s .
In Hungary at t h e B u r e a u f o r Systems Analysis (OMFI3 REI), A. V a r i , Z. P a p r i k a and J. Vecsenyi are investigating t h e conditions u n d e r which t o u s e new decision technology in ill-defined decisions finds s u c c e s s f u l a p p l i c a t i o n s .
The g r o u p s identified a b o v e h a v e a g r e e d t o c a r r y o u t c o l l a b o r a t i v e r e s e a r c h t h r o u g h IIASA in t h e collection, development, field t e s t i n g , a n d publication of d e t a i l s of decision s u p p o r t methods a n d a s s o c i a t e d guidelines, as a c o n t r i b u t i o n t o t h e solution of s t r a t e g i c decision problems by top-level decision m a k e r s .
The r e s e a r c h e r s from LSE, OMFB-RE1 a n d VNIISI h a v e p r e v i o u s l y made com- p a r a t i v e a n a l y s i s of t h e Application of Decision S u p p o r t Systems in R a n d D Deci- sions, as p a r t of IIASA's Management a n d Technology A r e a , Task 2: 1981-1983; t h i s work i s d e s c r i b e d in a s e r i e s of IIASA C o l l a b o r a t i v e P a p e r s a n d t h e a n a l y s i s i s summarized in a book d e v e l o p e d from t h e IFIP WG8.3 Working C o n f e r e n c e o n Processes a n d Tools for Decision Support, held at IIASA in 1 9 8 2 (Humpnreys, Vari a n d Vecsenyi, 1982; Humphreys et al., 1983).
B. OBJECTIVES
AND
SCOPE OF THE PROJECTThe p r o j e c t o n Evaluation of Decision S u p p o r t Systems i s a continuation a n d development of (i) t h e IIASA p r o j e c t o n Comparative Analysis of t h e Application of decision S u p p o r t Systems in R a n d D decisions: 1981-1983; a n d of (ii) t h e r e s e a r c h a c t i v i t i e s of members of t h e i n t e r n a t i o n a l network of t h e c o l l a b o r a t i v e g r o u p in Hungary, UK, USA, USSR c o n c e r n i n g t h e u s e of decision t h e o r y a n d methodology in o r g a n i z a t i o n a l c o n t e x t .
a a s e d o n t h e a p p a r e n t i n c r e a s i n g demand f o r using decision s u p p o r t systems in solving complex d e c i s i o n p r o b l e m s a n d a l s o o n t h e accumulated e x p e r i e n c e s in decision a n a l y s i s a n d DSSs development t h e following o b j e c t i v e s will b e a c h i e v e d : 1. Reviewing of methods f o r s u p p o r t i n g t h e s t r u c t u r i n g a n d solving of ill-defined
unique decision problems. The methods will include both t h o s e t h a t c a n b e used b y t h e d e c i s i o n m a k e r s independently of e x t e r n a l c o n s u l t a n t s a n d t h o s e designed f o r use by decision a n a l y s t s working t o g e t h e r with t h e decision mak- ers.
2. Providing a framework f o r designing a n d s e l e c t i n g decision s u p p o r t systems (DSSs) f o r s t r u c t u r i n g ill-defined decision problems.
3. Providing a b a s i s f o r a c r o s s - c u l t u r a l c o m p a r a t i v e study o n e x p e r i e n c e s in problem solving DSSs c o n s t r u c t i o n a n d application in d i f f e r e n t c o u n t r i e s t h r o u g h a n i n t e r n a t i o n a l network of c o l l a b o r a t i n g g r o u p s .
C. EXPECTED
FINAL
RESULTSA t t h e end of t h e p r o j e c t t h e final r e s u l t s will b e published t h r o u g h IIASA a n d p r o b a b l y t h r o u g h o t h e r publishing channels. The published r e s u l t s will comprise:
1. A book of e x p e r i e n c e s on DSS c o n s t r u c t i o n f o r problem s t r u c t u r i n g compris- ing a n i n t r o d u c t o r y c h a p t e r on frameworks useful in DSS c o n s t r u c t i o n a n d application, a n i n t e r n a t i o n a l s e r i e s of c a s e s t u d i e s developed by t h e p a r t i c i - p a n t s in t h i s p r o j e c t , a n d a final summary c h a p t e r making comparisons a c r o s s t h e case s t u d i e s (methodological, c u l t u r a l , r e s u i t s ) a n d pointing towards t h e development of t h e o r y DSS f o r problem s t r u c t u r i n g .
2. A comprehensive review of t h o s e methods (existing s o f t w a r e implementing t h o s e methods which p r o v i d e a r e s o u r c e b a s i s f o r decision problem s t r u c t u r - ing l i b r a r i e s . This will b e s u p p o r t e d by a c a t a l o g u e of methods and guiaeiines f o r t h e i r s e l e c t i o n a n d u s e in p r a c t i c a l applications.
D.
PHASES OF THE PROJECTThe p r o g r a m of t h e p r o j e c t consists of a p r e p a r a t o r y p h a s e , followed by a main p h a s e ending o n 31 December 1985. I t comprises a t o t a l of 15 t a s k s .
In t h e preparatory phase a c o n c e p t u a l f r a m e f o r t h e p r o j e c t will b e formu- l a t e d and t h e i n f r a s t r u c t u r e of t h e i n t e r n a t i o n a l c o l l a b o r a t i o n will b e e s t a b l i s h e d f o r t h e main p h a s e . The p r e p a r a t o r y p h a s e w a s completed at a t a s k f o r c e meeting in Budapest, 11-13 F e b r u a r y 1985.
In t h e main phase, r e s e a r c h e r s are studying t h e h i s t o r y of t h e solution of a g r o u p of decision problems t h r o u g h o u t t h e i r v a r i o u s s t a g e s : recognition of t h e problem situation, calling f o r decision analysis, s e l e c t i o n of a problem-solving p r o - c e d u r e , definition a n d s t r u c t u r i n g of t h e problem within t h e n a t u r a l language of t h e p a r t i c i p a n t s in t h e decision making p r o c e s s a n d t h r o u g h t o r e c o r d i n g t h e problem, r e p r e s e n t a t i o n a c t u a l l y used as a b a s i s f o r d e s c r i b i n g p r o c e d u r e s ( i n t e r - viewing t h e decision a n a l y s t s , t h e c h o i c e a n d use of confidential q u e s t i o n n a i r e s with t h e p a r t i c i p a n t s , and s o f o r t h ) .
This means t h a t a p p r o a c h i n g t h e problem by monitoring t h e decision s t r u c t u r - ing phenomenon as a whole. Based on t h e r e c o r d i n g , c0dir.g a n d analysis of t h e information g a t h e r e d , models s u p p o r t i n g i l l - s t r u c t u r e d decision s i t u a t i o n s in a n organizational c o n t e x t c a n b e f u r t h e r developed. The case s t u d i e s will b e developed t h r o u g h discussion a n d c o m p a r a t i v e analysis within t h e g r o u p of p a r t i c i - p a n t s with a view t o f u t u r e publication. New t h e o r e t i c a l findings will need t o b e developed t h r o u g h t h i s work, and t h e s e will form a major f e a t u r e of t h e book t o b e w r i t t e n summarizing t h e r e s u l t s of t h e case studies.
E.
TASKSOF
THE PROJECT1. Planning meeting f o r t h e p r o j e c t .
2. Establishing t h e i n f r a s t r u c t u r e of t h e p r o j e c t . 3. C r e a t i n g t h e c o n c e p t u a l framework of t h e p r o j e c t .
4. Pilot r e c o r d i n g and a n a l y s i s b y e a c h p a r t i c i p a n t of u s e of a n i n t e r a c t i v e deci- sion aiding method o r DSS, meeting t h e c r i t e r i a outlined a b o v e a n d p r e p a r i n g a case study f o r t h e t a s k - f o r c e meeting.
5. Organizing t h e F e b r u a r y 1 9 8 5 t a s k - f o r c e meeting.
6. Finalizing t h e q u e s t i o n n a i r e a n d guidelines f o r (i) t h e decision problem s t r u c - t u r i n g method c a t a l o g u e , (ii) case s t u d y book a n d c o m p a r a t i v e analysis.
7. Forming t h e s p e c i f i c a t i o n of a decision s t r u c t u r i n g p r o g r a m c a t a l o g u e , b a s e d upon method-developer s e l f - r e p o r t s t o t h e specifications q u e s t i o n n a i r e . 8. Conducting (by individual p a r t i c i p a n t s ) decision a n a i y s e s using decision aiding
o r decision s u p p o r t methods including t h e s e l e c t i o n and u s e of modules identi- fied in 7.
9. E l a b o r a t i n g t h e e x p e r i e n c e s r a i s e d t h r o u g h t h e decision a n a l y s e s c a r r i e d o u t in 8.
10. Exchanging case s t u d i e s within t h e network, a n d providing f e e d b a c k t o a n a l y s t s , f a c i l i t a t e d t h r o u g h a meeting of t h e p r o j e c t team in Helsinki, August 1985.
11. Evolving a framework f o r reviewing a n d d e s c r i b i n g t h e case s t u d i e s with emphasis o n t h e r o l e of problem s t r u c t u r i n g language and i t s r o l e within deci- sion analysis.
12. Revising a n d completing t n e s p e c i f i c a t i o n of t h e decision problem s t r u c t u r i n g method c a t a l o g u e .
13. Completing t h e c a s e s t u d y book a n d c o m p a r a t i v e s t u d y . The s c h e d u l e of t h e t a s k s i s shown in Table 2.
Table 2. S c h e d u l e f o r t h e t a s k s of t h e p r o j e c t .
i
T a s k s S e p t . 1984 Dec. Feb. Apr. 1 9 8 5 Jul. Oct. D e c . 1 1I
Planning meetings f o r t h e p r o j e c t
Establishing t h e i n f r a s - t r u c t u r e
C r e a t i n g t h e c o n c e p t u a l framework
P i l o t r e c o r d i n g a n d c a s e s t u d i e s
Design of method ques- t i o n n a i r e a n d guidelines Task f o r c e meeting in Budapest, 11-13 Feb.
Building a c a t a l o g u e of methods
Decision a n a l y s i s case s t u d y p r e p a r a t i o n
E l a b o r a t i n g tine e x p e r i - e n c e s of d e c i s i o n a n a l y s i s
Exchanging case s t u d i e s
Evolving a framework
f o r problem reviewing a n d d e s c r i b i n g case s t u - d i e s
Completing s p e c i f i c a t i o n of t h e c a t a l o g u e
Completing t h e case s t u d y book
