Response Time Decomposition of a Multi-Tier System

When there is only a single request processed in the multi-tier system withmtiers{T₁,· · ·,T_m}, the response time can be written as the sum of the service timesS_iof all involved tiersT_i:

R¹_T =

m i=1

∑

Si. (4.2.1)

Such a situation is very common for testing systems, which are used by developers to test mod-ifications in their systems. However, this situation does usually not cover the workload of the productive system, which is often subjected to multiple requests originating from multiple differ-ent requesting applications.

With an increasing concurrent request count, the individual service times get usually superposed with queuing times. To model the response time of a multi-tier Internet system, there are various different approaches. The state-of-the-art approach is to model the entire multi-tier system as a (complex) queuing network composed of multiple simple queuing systems and to apply the MVA algorithm to calculate the mean response time for a particular request population.

The basic approach behind the MVA algorithm is to consider the individual queues of the queuing network independently from each other. The overall network solution then results in the product of the individual solutions obtained from the individual queues. Such queuing networks are therefore also calledproduct-form queuing networks(cf. Section 2.7.1).

The determination of the number of queues per tierTi is one of the important aspects in the of modeling multi-tier systems. This, however, is beyond the scope of this thesis. Instead, the goal in this chapter is to provide insight into the effect of an improvement in an individual tier onto the overall system. This is important to understand how a planned or already implemented opti-mization will affect the overall systems’ behavior. This is also important to determine whether a costly improvement effort is worthy to be tackled. Otherwise, this can often lead to a waste of valuable time and money. Therefore, to simplify the discussion onto the most relevant aspect, in this chapter we focus on multi-tier systems, which comply with the (Ti≡Q_i) property, such as depicted in Figure 4.3.

Our approach is fundamentally based on the load distribution analysis of the established MVA algorithm. Therefore, in the following, we provide insight into several aspects of its asymptotic behavior.

4.2.2.1 The Basic MVA Algorithm

Algorithm 1 represents a basic form of the MVA algorithm, which was also used by Urgaonkar et al. [79]. Note that its background was already explained in Section 2.7.1. In this section, the given algorithm serves the purpose of clarity.

If we model each tier as a single queue, this algorithm can be used to predict a multi-tier system’s response time for an increasing concurrency leveln.

Algorithm 1Basic MVA

Since the MVA algorithm is a recursion, to determine the response timeRⁿ_T for a concurrency level n, the algorithm always starts with the calculation ofR¹_T and terminates at the response timeRⁿ_T for the concurrency level in question.

The lines 8, 10 and 12 of Algorithm 1 constitute the fundamental recurrence relation of the MVA algorithm, which can also be written as:

Rⁿ_i =Si(1+τⁿ⁻¹Rⁿ⁻¹_i ), (4.2.2) whereRⁿ_i denotes the processing time as defined above. The variable

τⁿ⁻¹= (n−1)

∑^m_i=1Rⁿ⁻¹_i (4.2.3)

denotes the throughput of the multi-tier system for concurrency leveln. The overall response time Rⁿ_T is then the sum of the per tier processing timesRⁿ_i.

The recurrence relation (4.2.2) only shows how the processing time at concurrency level n is composed from the processing time forn−1. To reveal all the terms incorporated from previ-ous concurrency levels, the following theorem provides a compact representation ofRⁿ_T. It also provides insight into the development of the response timeRⁿ_T calculated by means of the MVA algorithm.

Theorem 1. For the response time Rⁿ_T, calculated by means of the MVA algorithm for a queuing network composed of m iterative queues, such as illustrated in Figure 4.3, it holds:

R¹_T=

Proof. Since the overall response time is a sum of the processing times, it holds Rⁿ_T =

m i=1

∑

Rⁿ_i.

Hereby, the relation (4.2.4) is trivially proven. The proof for (4.2.5) in turn, is then accomplished by deducing a compact representation for exactly these processing times:

Rⁿ_i = 1+L_iⁿ⁻¹

By continuing with the successive substitution of the emerging terms, for the processing times Rⁿ⁻³_i ,Rⁿ⁻⁴_i ,· · ·R¹_i we finally get:

Theorem 1 reveals that the response timeRⁿ_T calculated by means of the MVA, is basically com-posed of power series of the services timesS_iwith complex coefficients depending on the response times from previous steps, which therefore again results in a recursion.

However, this theorem is not appropriate for practical situations. First of all, it is numerically highly unstable due to floating point errors. More importantly, it is not suitable to reveal specific asymptotic behaviors of the response time. Yet, to understand the overall system behavior for increasing concurrency this asymptotic behavior is essential.

Therefore, we proceed with a reformulation of the recurrence relation in equation (4.2.2):

Rⁿ_i =Si(1+τⁿ⁻¹Rⁿ⁻¹_i ) =Si 1+(n−1)Rⁿ⁻¹_i

with

ω_iⁿ⁻¹:= Rⁿ⁻¹_i

∑^m_j=1Rⁿ⁻¹_j , 0<ω_iⁿ⁻¹<1 (4.2.9)

and m

i=1

∑

ω_iⁿ⁻¹=1. (4.2.10)

The termω_iⁿ, also called weighting factor, is a relative measure of how many of then requests in the entire system are processed at tier T_i. The relation (4.2.9) is true, because an individual weighting factor is a ratio between the corresponding processing time and the overall response time (composed of all processing times).

Note thatLⁿ_i ≤n−1, sinceLⁿ_i denotes the queue length ofT_i at the moment when another request is going to populate the system while there are already(n−1) requests in processing. With the recurrence relation (4.2.8), the term for the queue lengthLⁿ⁻¹_i is expressed as a per tier weighting factor, distributing the overall load among the individual tiers.

The advantage with these weighting factors is that they enable to analyze the asymptotic behavior of the response time calculated with the MVA algorithm. The analysis is based on the characteristic of the weighting factors that they are always bounded by 0<ωi<1. Our analysis of the MVA algorithm behavior is essentially based on the development of these weighting factors. This is dedicatedly examined in the following section.