Uncertainty Assessment in the Decision Analysis Process
4.3 Uncertainty Analysis
The process of uncertainty analysis using error propagation techniques is illustrated in Figure 4.2.
In the first part of this section, the background of error propagation techniques is introduced.
Robustness measurement using Signal-to-Noise Ratio (SNR) is presented in the second part.
Uncertainty characterized by percentage uncertainty
with confidence level
Numbers of standard deviations
Uncertainty expressed by mean and standard deviation
Calculation of propagated uncertainty using error propagation techniques
Transform into
Transform into
Input
Figure 4.2: The Process of Uncertainty Analysis using Error Propagation Techniques
4.3.1 Background of Error Propagation Techniques
Error propagation techniques answer the question: how the uncertainties of input variables will be propagated to some predefined functions involving these variables and lead to the final result [17]. There are two classes of error propagation techniques: analytical and simulation-based numerical error propagation techniques.
The analytical error propagation technique relies on a linearized Taylor series expansion of the function about the mean of each variable, the total error of the function is obtained by
4.3 Uncertainty Analysis
combining the linearized individual error in quadrature. For a function
y=f(x1, x2, ..., xn) (4.7) wherex1, x2, ..., xn are input variables,δx1, δx2, ..., δxn refer to the relatively small uncertainties inx1, x2, ..., xn, respectively. The small uncertainties can be identified as Gaussian distribution provided that their magnitudes are not too large [17]. Small uncertainties of the variables δx1, δx2, ..., δxncan be used with their standard deviationσx1, σx2, ..., σxninterchangeably. Based on Taylor series expansions, the propagated errors of input variablesx1±δx1, x2±δx2, ..., xn±δxn can be analytically described by Equation 4.8 [17].
σ2y = whereσ2y is the total variance of the function, ∂x∂f
j is a partial derivative of the functionf with respect to variablexj, when treating other variablesx1, x2, ..., xj−1, xj+1, ..., xnas constants,σ2xj is the variance of variable xj, and σx2jxi is the cross-product covariance when variables xj and xi are correlated. If the variables x1, x2, ..., xn are independent, we can omit the cross-product covariance term, Equation 4.8 reduces to
σy2=
The contribution due to the uncertainties in x1, x2, ..., xn is considered separately through Equation 4.9, provided that the errors of the input variables could be seen as normally distributed and there is no strong nonlinearity associated with the function in its evaluation range.
While analytical error propagation technique is appropriate for simple calculation processes, simulation-based numerical error propagation technique is more suitable for dealing with com-plex models, where trade-off has to be made between results accuracy and computation time.
4.3.2 Robustness Measurement using Signal-to-Noise Ratio
Robustness is an important performance measurement when uncertainty exists. Taguchi pi-oneered the application of robust design methods in product design and manufacturing cess [120]. Robustness reflects product’s ability to withstand uncontrollable variations in pro-duction and usage. The Signal-to-Noise Ratio (SNR) is one way to measure the robustness in Taguchi’s method. The SNR in terms of mean and standard deviation is defined as Equa-tion 4.10.
SNR = 20log10(µ
σ) (4.10)
The SNR is expressed in decibel (dB). For instance, 40 (dB) means that the magnitude of mean is 104020 = 100 times the magnitude of standard deviation. A larger SNR value indicates more robustness against uncertainty.
Moreover, linearity also influences the SNR value. When the relationship between the input and output of a system is not linear, deviation from linearity is taken as the error after the decomposition of variation and the SNR becomes smaller [120].
Uncertainty Analysis for an Aircraft Selection Example
Uncertainty analysis for an aircraft selection example, as described in Subsection 2.3.4, is con-ducted in this subsection. The decision matrix is repeated in Table 4.1 for the convenience of calculation.
Table 4.1: Decision Matrix of an Aircraft Selection Example for Uncertainty Analysis Criteria
C1: Comfort C2: Cost C3: Environmental friendliness Alternatives w1: 0.3 w2: 0.4 w3: 0.3
Aircraft A 8 7 10
Aircraft B 9 6 5
Aircraft C 6 7 8
Assume that the DM states that there are 15% uncertainties existing in criteria values with 80% confidence level, and there are 30% uncertainties existing in weighting factors with 90%
confidence level. Following the uncertainty analysis process shown in Figure 4.2, percentage uncertainties with confidence levels are transferred into means and standard deviations, Monte Carlo-based error propagation technique is used to calculate the propagated uncertainties.
When SAW is used to solve the aircraft selection example, the probabilistic ranking of the three candidate aircraft is summarized in Table 4.2. The largest number in each row indicates the most likely ranking. It can be observed that Aircraft A has the highest probability to be ranked first, Aircraft B is most likely to be ranked second, and Aircraft C has the highest probability to be ranked in the last place.
Table 4.2: The Probabilistic Ranking in an Aircraft Selection Example Alternatives
Ranking Aircraft A Aircraft B Aircraft C
1st 72.00% 26.00% 2.00%
2nd 25.00% 56.00% 19.00%
3rd 3.00% 18.00% 79.00%
4.4 Local Sensitivity Analysis via Iterative Binary Search Algorithm
0 20 40 60 80 100
[A3 A2 A1]
[A3 A1 A2]
[A2 A3 A1]
[A2 A1 A3]
[A1 A2 A3]
[A1 A3 A2]
All possible ranking permutations
Simulation runs
20% 40% 60% 80% 100%
Figure 4.3: The Probabilistic Ranking Permutations in an Aircraft Selection Example
Table 4.3: Robustness Measurement using Signal-to-Noise Ratio in an Aircraft Selection Example Alternatives Mean Standard deviation SNR (dB)
Aircraft A µ= 0.9041 σ= 0.1036 18.8148 Aircraft B µ= 0.8506 σ= 0.1096 17.7978 Aircraft C µ= 0.7741 σ= 0.0973 18.0144
In addition to the probabilistic ranking of each alternative, the likelihood for alternatives permutation is also calculated and demonstrated in Figure 4.3, where the vertical axis represents all possible alternatives permutations, the lower horizontal axis stands for simulation runs, and the upper horizontal axis corresponds to the occurrence probability of each permutation. It can be seen that the alternative permutation [A1 A2 A3] ([Aircraft A Aircraft B Aircraft C]) has the highest probability of occurrence.
In order to compare the robustness of the three alternatives against uncertainties in weighting factors and criteria values, SNR for each alternative is calculated using Equation 4.10 and summarized in Table 4.3. Considering that a larger SNR value indicates more robustness against uncertainty, we can observe from Table 4.3 that Aircraft A is most robust against uncertainties in weighting factors and criteria values among the three alternatives.