7.2 Resource Demand Model
7.2.2 Comparison to Known Approaches
The modeling approach presented in this thesis was compared to known ones in a first analysis.
Two month of the resource demand time series were used to characterize the model (∆tp2 = 61 days). The remaining ten month were used to compare the forecasts to the actually required resources (∆tp3 = 192 days).
Selected Approaches
A common approach known from classical time series analysis was applied for comparison.
The seasonal trend was modeled by a deterministic function derived using moving averaging.
The residuals were treated as random noise that was modeled using the ARIMA approach.
The noise was pessimistically overestimated in a first version of this approach to derive required resources from the models the same way like suggested in [17, 129, 104]. This approach will be called M A+ in the following. The saving potential the approach provides is expected to be very low, since M A+ pessimistically overestimates the noise. Hence, a modified version of this approach, calledM A−, was additionally analyzed that (very optimistically) completely ignores the noise. Only the trend function is used for forecasting, which is expected to result in many and strong underestimates but also should strongly increase the saving potential.
Finally, the result achieved by M A− andM A+ can be regarded as upper and lower limits for the approach presented in this thesis. The new approach should provide significantly higher
2NOWIS Nordwest-Informationssysteme GmbH & Co. KG
136
7.2 Resource Demand Model
saving potential compared to M A+, while the underestimates ofM A− are not exceeded.
Discussion of Results
Accuracy and saving potential achieved by the different modeling approaches when they were used for dynamic resource management are presented in Figure 7.4 and 7.5 once for CPU time and once for memory. In both cases, the new model was analyzed once with and once without performing error correction at runtime the way presented in Section 6.5.3. The respective cases are annotated byEC andno EC respectively.
P(δdyn<0)=20.9%
P(δdyn>0)=34.6%
MA-new model ECno EC
-0.4 0 0.5δdyn1.0 0.5
0.2
0.2
0.5
0 0.5 1.0
1.0
1.0
1.0
1.0
MA+
πmin
usable: Πmax=91.0%
provided:Πmax=94.9%
usable potential provided potential )
( dyn
Pδ ~tsav
P(δdyn<0)=2.6%
P(δdyn>0)=96.5%
P(δdyn<0)=0.9%
P(δdyn>0)=98.7%
P(δdyn<0)=0.1%
P(δdyn>0)=99.9%
usable: Πmax=86.3%
provided:Πmax=54.4%
usable: Πmax=86.3%
provided:Πmax=43.0%
usable: Πmax=91.0%
provided:Πmax=10.2%
CPU time
Figure 7.4: Accuracy and provided saving potential of different modeling approaches, when they are applied to forecast required CPU time for dynamic resource management.
P(δdyn<0)=50.35%
P(δdyn>0)=46.5%
MA-new model ECno EC
-0.4 0 0.5 1.0 0.2
0.2
0.2
0.2
0 0.5 1.0
1.0
1.0
1.0
1.0
MA+
πmin
usable: Πmax=74.2%
provided:Πmax=76.3%
usable potential provided potential )
( dyn
Pδ ~tsav
P(δdyn<0)=1.8%
P(δdyn>0)=98.1%
P(δdyn<0)=0.4%
P(δdyn>0)=99.5%
P(δdyn<0)=0.1%
P(δdyn>0)=99.9%
usable: Πmax=74.2%
provided:Πmax=39.2%
usable: Πmax=74.2%
provided:Πmax=36.1%
usable: Πmax=74.2%
provided:Πmax=18.2%
RAM
δdyn
Figure 7.5: Accuracy and provided saving potential of different modeling approaches when they are applied to forecast required memory for dynamic resource management.
7 Experimental Assessment
A probability distribution ofδdyn(t) is presented on the left side of both figures for each ap-proach. The cumulated probabilities of over- and underestimates are given as well. The saving potential determined the way introduced in previous section is presented for each approach on the right side of the figures. The bar plots show the overall time period, in which savings higher than a certainπmincould be provided to the scheduling approach related to the overall simulation time. The yellow bars represent the savings that theoretically could be provided in case of an exact forecasting method. The green ones are the savings that the analyzed methods actually provided. Furthermore, the overall resource saving potentials Πmax provided by the exact forecasting approach and by the real ones are presented as well.
One can clearly see that the forecasting accuracy is already very close to the one ofM A+ even without error correction at runtime. The estimated resources were lower than the actual ones only in up to 2.6% of time. Error correction performed at runtime can further reduce underestimates to below 1.0%.
The new approach could significantly increase the savings potential ofM A+. Only very low savings π could be provided to the scheduling approach byM A+. A reason is that not the trend of the resource demand time series but the noise performance is varying very strong over time. M A+ assumes a constant noise performance and hence could not take any advantage of varying noise performance. This fact becomes further apart with respect to the results of M A−. Neglecting the noise leads to even more provided saving potential than actually present.
Underestimates in up to 50% of time are the consequence.
The forecasting accuracy of the models for static resource management was analyzed as well.
The results are presented in the table in Figure 7.6 a). Amax needs not to be estimated for memory using the models, sinceAmax=Rmaxis valid for memory capacity (cf. Section 5.4.2).
Rmax is determined in phase one of the concept using benchmarks and thus can be directly used asAmaxin phase two and three as well. Hence, only the accuracy of the forecastedAmax for CPU time was analyzed.
) 0 ( stat<
Pδ
MA- new model MA+ )
min(δstat
0.1%
-44.3%
0.2%
-37.9%
0.1%
-44.3%
CPU time
a) b) tp2 tp3 time
CPU time Amax A(t)(new model)
tp3+∆tp3 δstat≈-38%
δstat≈-38%
Figure 7.6: a) Accuracy of the modeling approaches when they are applied for static resource management b) An exemplary time series and the forecastedAmax
All three analyzed approaches underestimated the actually required CPU time in only up to 0.2% of time. One can see in Figure 7.6 b) that only single spikes of the actually required resources A(t) exceed the provided resource capacityAmax. But these spikes can be
signifi-138
7.2 Resource Demand Model
cantly higher thanAmax, which can lead to strong performance slowdowns in these cases. The reason for such violations is mainly a changed long term trend in phase three. The slope of the trend slightly increases so that Amaxwas estimated to low.