Comparison to Known Approaches

The following four different static resource management approaches were compared against each other concerning the required server count and the frequency of resource shortages in a first analysis:

• KnownPess: Pessimistic not statistic resource management as described in Chapter 4 (baseline for savings),

• KnownStat: Classical statistic resource management as suggested in [117, 58, 45],

• NewStatPess: The new statistic resource management that pessimistically overestimates correlations as described in Section 5.4.2, and

• NewStatOpt: The new statistic resource management that additionally uses correlations for resource management as described in Chapter 5.4.5.

146

7.3 Statistical Static Resource Management

The first one distributes VMs to servers according to required resources that are determined using benchmarks. Enough resources are reserved for each VM to meet its demand even in case of the maximally expected workload. No SLO violations caused by resource shortages will occur. The second one has been shortly described in Section 5.4.1. Resource demand is modeled using one single random variable. Required resource capacity is determined as percentile of this variable. The second and third one are the approaches that were developed within this thesis. They have been presented in Chapter 5.

All four approaches were used to distribute the set of 50 VMs to servers. The same SLO was assigned to all of them in each simulation run. The fine grained specification that has already been introduced in Section 7.1.1 (Figure 7.1 a)) was selected for both of the new statistic approaches. Three different percentile based SLO specifications (Figure 7.1 b)) were used with KnownStat. All of them guarantee the same performance like the fine grained specification as described in Section 7.1.1. The normal server configuration (4 cores, 32 GB) was used in all simulation runs.

Discussion of Results

The results are presented in Figure 7.9. First of all, one can say that statistical resource management can in general significantly reduce the number of required servers compared to the pessimistic not statistic approach. Savings between 17% and 42% could be achieved.

NewStatOpt KnownStatP₃

∆#srv[%]

a) 50

0 25

25

42 42 33

17

NewStatPess

KnownStatP₂ KnownStatP₁

NewStat

server savings resource shortages

KnownStat Pess Opt P₁ P₂ P₃ max

Ø std

0.15% 0.15%

0.02% 0.02%

0.05% 0.05%

0.00% 0.12%

0.00% 0.06%

0.00% 0.05%

0.90%

0.40%

0.27%

b)

~

rs

P

Figure 7.9: a) Server savings of different statistical static resource management approaches.

b) The respective relative frequencies of resource shortages.

Not all percentile based SLOs led to the same savings comparable to the evaluation result of the fine grained SLO specification. Mainly the minimal performance goalsη^max that must be achieved in any case limited the savings in this evaluation scenario. The less restrictive this performance goal was selected, the more savings could be achieved byKnownStat independent from the assigned probabilities.

NewStatPess could not achieve the same savings likeKnownStat in most cases. This result was expected, since positively correlated resource demand behavior is pessimistically assumed by NewStatPess. The same savings could be only achieved, when correlations are used for a

7 Experimental Assessment

more optimistic planning (NewStatOpt).

An interesting fact concerns the additional savings of NewStatOpt. Although most VMs have more or less positively correlated workload (correlation coefficient: between 0.3 and 0.6), using correlations for resource management could increase the savings from 25% up to 42%. The reason is that mainly the lower resource demand values that are dominating the time series are positively correlated. The high spikes of different VMs are often uncorrelated.

They are even negatively correlated in some cases. Hence, using correlations can in any case significantly increase the savings, since only the higher resource demand values are relevant for static resource management.

The resulting distribution of VMs to servers hardly led to actual resource shortages despite the comparable large amount of underestimates of the models (cf. Section 7.2). The reason mainly was unused resource capacity that remained at the servers. Most of the VM required far more than 50% of CPU time (A^max_i >0.5). Hence, a significant amount of CPU time capacity remained unused, since only 400% of CPU capacity (4 cores) was available. The remaining capacity could not satisfy the demand of one further VM in many cases. Furthermore, VMs require their maximal resource demand rarely at the same time as discussed before. Hence, such negative correlations can prevent resource shortages, even if one VM exceeds the expected demand.

The disadvantages of KnownStat have been discussed in Section 5.4.1. The assumption of uncorrelated resource demand behavior was pointed out as the major weakness. Such weakness did not lead to significant resource shortages in this evaluation scenario. The reason is that mainly the minimal performance goal η^max that must be achieved in any case. This performance goal determined the required resource capacity A^max_i . Providing such minimal resource capacity A^max_i to a VM results in far less resource shortages than specified by the limitsP_η^min_i . Hence, the incorrectly calculated probability distribution of the overall resource demand had no effect at all.

One could now believe that performing any statistics to ensure meeting P_η^min

i does not make any sense at all for realistic values ofP_η^min_i andη^max and for realistic resource demand behavior. But these result are again an artifact of the low sample interval of the resource demand. Shorter resource demand time series (with a duration of 21 days) that had a sample rate of one sample per minute were analyzed as well. These time series already showed quite a stronger deviation of the noise performance. As a consequence, the resulting percentile is much higher compared to the one obtained when the strongly averaged time series were used.

Analyses with such more fine sampled time series were presented in [58]. The authors showed that indeed neglecting correlations can lead to significant SLO violations.

It has been already pointed out in the previous section that sample intervals of 5min are far to long to guarantee any performance goal at all. A sample rate of several samples per

148

7.3 Statistical Static Resource Management

second would be more appropriate. Neglecting correlations between such fine sampled resource demand time series are expected to lead to significant larger errors according to the results presented in [58]. Such analysis could not be performed within this evaluation due to missing data.