2.3 Responsiveness
2.3.4 Challenges of responsiveness
Responsiveness or, more precisely, the response set and response time distribution, are very attractive metrics owing to their concise representation of a service’s probabilistic behavior in real time and in the presence of faults. However, it does have its own particular set of challenges as well.
The complexity challenge
Computing a response time distribution is usually a very complex undertaking. The complexity or even possibility of solving this problem heavily depends on the chosen modeling technique and on the desired precision of the results. A plethora of modeling techniques is available; examples include, but are not limited to, Markov models and queueing theory (in particular, real-time queueing [170]), Petri nets, methods based on temporal logic, or combinatorial approaches. A common problem is that for all of these methods, deriving random distributions for any non-trivial problem is difficult; often, only means and perhaps some stochastic moments are derivable. Sometimes it is possible to find expressions for random distributions using such a technique; however, this expression might only be solvable numerically. Sometimes, only simulation results are obtainable, and sometimes, it is possible to characterize a service execution by using actual measurements.
The definitions given above are orthogonal to the time of their actual calculation or estimation. This calculation can—in theory—happen on-line or off-line. In an off-line calculation the state of the system (as far as load and/or faults are concerned) is only known probabilistically. In an on-line calculation, such as the calculation of the responsiveness function for a service request, the state of the system at this point in time is known (to a certain degree) and this knowledge can be used in the responsiveness estimation. Therefore, such an on-line calculation is potentially more precise than an off-line one. From a theoretical point of view, this corresponds to the computation of conditional probabilities.
An on-line calculation also has the possibility to monitor the actual progress of a service execution and use this information, e.g., to make decisions about resource allocation, fault-tolerance protocols, admission tests for additional load, or to take into account the impact of various system modes. For an accurate on-line monitoring, the service might need to cooperate and provide status information in a specified manner. Also, rapid estimation techniques forRSare necessary.
The comparison challenge
Often, people are interested in a comparison of different alternatives to implement a service or in the relative quality of two systems. More precisely, the question is how to compare the response sets or, somewhat simpler, two response time distributions of a given service on two different systems.7
Non-stationary services with a meaningful deadline8 can be compared using their request timest0 and
RF(t
0
). If the service is additionally stationary (on both systems), the responsivenessRF(0) is the natural basis for comparison: it is just a number. If no deadline is given for a stationary service, one has to compare the response time distributions as functions. Formally, how can an order relation be defined on the set of response time distributions so as to reflect an intuitive notion of “better (or more responsive) service”? While the function space has a partial order relation9 it is by no means clear how to judge two functions that are non-comparable with respect to this partial order.
A simple metric to compare functions with this generality would be the mean of theRTD. But since responsive service execution is the objective, the mean alone is insufficient. An obvious supplement is the variance or standard deviation. Yet even with these two numbers the ugly fact of incomparability rears its
7Note that this is not precisely identical to comparing two systems, since it is not necessarily the case that two different systems allow the same set of services. On an end-to-end level, however, this makes sense nonetheless, since available services can be regarded as a user requirement; similarly, both systems must allow the same service request times.
8A meaningful deadline is one that reflects some actual requirement imposed by the system’s environment. Any deadline used only internally in a system is purely artificial and should not be considered as a basis for comparison of a system’s or a service’s level of merit.
9For two functionsf1,f2that both map a domainDto, say, the real numbersR,f1
<f
2if and only if8x2D:f1 (x)<f
2 (x). It is only a partial order, since there are functionsf1,f2such that9x1;x22D:f1(x1)<f2(x1)^f1(x2)>f2(x2).
head: is a small mean with large variance or a larger mean with a smaller variance preferable? In such a situation, the more predictable service might be preferable.
What characterizes a predictable service in the first place? Ideally, it would be a service with a precisely determined runtime t0, i.e., with a response time distribution of the form RTD (t) = H(t t0
).10 Such a service is obviously unrealistic, but a tractable definition for “more predictable” can be generalized from it—intuitively, the fewer possibilities for runtimes there are, the better:
Definition 2.6. Letr,r1,r2be response time distributions. DefineD(r)def= fxj8>0:9x12(x ;x+):
r(x)6=r(x
1
)gas the differential set ofr, the set of points whererchanges its value.
The response time distributionr1is called more predictable thanr2if and only if:
D(r
2
)is uncountably infinite,D(r1
)is uncountably infinite, andD(r1
)D(r
2 ), or
D(r
2
)is uncountably infinite,D(r1
)is at most countably infinite, or
D(r
2
)is countably infinite,D(r1
)is countably infinite, andD(r1
)D(r
2 ), or
D(r
2
)is countably infinite,D(r1)is at most finite, or
D(r of the set of changes ofrwith
(intuitively, a service with a large jump and a few small ones is more predictable than one with many jumps of the same size).11
Unfortunately,Dis very difficult to assess for a real service executed on a real system. Therefore, often the standard deviation, and perhaps the variation coefficient (the standard deviation divided by the mean), are the only practical metrics of predictability. Yet these characterizations have to be used with care: it is possible to construct two response time distributions r1, r2 such that r1 is more predictable than r2 in the sense of Definition 2.6, but the variation coefficient ofr1is larger than that ofr2. Nevertheless, the variation coefficient is a suitable approximation for a predictability metric for practical purposes. A precise metric for comparing two arbitrary services is still a question of future work.
Note also that predictability in the sense of Definition 2.6 or in the sense of a small standard deviation is a different concept than responsiveness, since no concept of a deadline is considered in the definition of predictability. A service might be highly predictable, but also miss a deadline; it might meet all deadlines, but have a large standard deviation (or variation coefficient) or a largeD.
The composability challenge
An important desideratum for abstractions of computer systems is composability. Today’s system are hardly ever completely designed from scratch, but are rather composed of pre-existing components. Given some properties of these components and the way they are composed, what are the properties of the new, composed system?
This question touches on some critical points. One is to identify a relevant set of properties, another is to characterize the process of putting systems together so that it is both effectively and efficiently possible to argue about some properties of the composed system, a third is to see if the new system has some properties
10
His the Heaviside functionH(t)=1fort0andH(t)=0fort<0.
11For this last case, note that the finiteness ofD(r1
)implies that all points werer1changes its values are points of discontinuity of
r1(and similar forr2).
2.3. RESPONSIVENESS
that were not present in its individual parts (an example for such an emerging property would be fault tolerance in a system based on potentially faulty components).
For responsiveness as an example for such a property, the question of composability can be made more concrete. A service is commonly composed of a number of lower-level services. Given the responsiveness of these services, what is the responsiveness of the composed service? If an efficient technique can be found to compute the responsiveness of composed services under reasonable assumptions about the interactions of the lower-level services, responsiveness can be considered as an example of a composable abstraction. The search for such a technique is the objective of current research in the Computer Architecture and Communications group at Humboldt-University Berlin and not the topic of this dissertation.
The challenge of Commercial Off-The-Shelf systems
Closely related to the question of composability is the relationship of responsiveness and COTS systems. By definition, COTS systems consist of existing components, but they aggravate the problems of responsiveness since often only insufficient knowledge about both system and components is available (e.g., scheduling poli-cies of a COTS operating system are not specified, making any analysis difficult). Moreover, COTS systems are often open systems, making fixed assumptions about the load or the way individual services interact with each other impossible.
This very openness requires some mechanisms to tame the interactions of services in a manner consistent with the COTS nature of the system. This disallows, e.g., any changes to the existing system (be it hardware or operating system) and requires middleware solutions put on top of these systems. One example for such a middleware is controlled arbitration of resources, for which a solution is described in Chapter 8; Chapter 7 introduces a concept for a replication-based fault-tolerance mechanism that conforms to this necessity for interface-respecting middleware solutions. However, responsive COTS systems are a problem area that still requires a lot of research and, owing to its enormous complexity and number of interacting factors, does not promise any simple, final solutions in the foreseeable future.
“The time has come”, the walrus said,
“To talk of many things:
Of shoes—and ships—and sealing wax—
Of cabbages—and kings—
And why the sea is boiling hot—
And whether pigs have wings.”
– Lewis Carroll
Chapter 3
Related Work
Distributed systems in general and cluster-based systems in particular have been an object of research for a considerable time. Some notable projects are described in this chapter, mainly considering projects that focus on performance, fault tolerance, real time, or Quality-of-Service aspects. The more interesting ones of these projects have more than a single focus; in particular, some examples for systems combining fault-tolerance and real-time capabilities in the sense of responsiveness are discussed.
3.1 Kinds of clusters
Cluster-based systems have been built for a number of different purposes. Perhaps the historically first are systems that are mostly concerned with executing computation-bound, large-grained parallel applications.
Such applications rarely communicate between concurrent parts and are therefore ideally suited for execution even on loosely coupled clusters. Examples for such systems include Condor [181] or Utopia [312]. They excel at achieving throughput for the entire system, but often do not provide exceptional performance or guarantees on performance for an individual program
Dedicated high-performance clusters are build to execute a parallel program as fast as possible. The main challenge here is communication performance; but the lack of a single system image (compared to supercomputers) adds other problems as well. Examples for these kind of projects truly abound—an overview can be found in, e.g., [111, 237]. Some notable examples for these two kinds of cluster organization (high throughput and high performance) are discussed in Section 3.2 along with some systems that extend beyond clusters to wide area environments.
Another kind of cluster-based systems is concerned with providing high availability, but do usually not target parallel applications. Here, a user should be able to rely upon the permanent availability of the system to execute user requests. Traditionally, this availability can be achieved by hot or cold standby techniques such as primary-backup, often implemented with custom networks and operating systems. Some of the commercial systems from Tandem, Stratus or Sequoia (cp., e.g., [260]) belong to this category, although one would hesitate to classify these systems as COTS clusters since they do heavily modify the basic systems or a truly custom-designed. More in the spirit of COTS are systems like Wolfpack [257] that extends Windows NT yet runs on typical COTS clusters; details can be found in Section 3.3.
Research in real-time systems has, to an even larger degree than fault-tolerance research, focused on custom-built systems. This focus is dictated by the complexities of commodity systems with regard to their behavior in real time. A few projects are discussed in Section 3.4, including one that addresses real-time
capabilities for high-performance computing. Addressing both real time and fault tolerance is the objective of projects described in Section 3.5.
The notion of Quality of Service is intimately linked with the idea of a guarantee (in whatever form) on the service quality, and as such includes real time as a special case. Much research has been performed in the context of Quality of Service; a brief overview of this area is given in Section 3.6, considering network Quality of Service and endsystem Quality of Service in turn.
All these kinds of cluster organizations (or custom-designed architectures) can be regarded as first-generation approaches. For a second generation, the combination of high availability with high performance and guar-anteed Quality of Service exhibits the union of all the problems mentioned before: A collection of (possibly heterogeneous) machines should be usable under an abstraction of an ideal, reliable, predictable machine.
Such a system provides reliable and guaranteed execution of (multiple) programs. No current system has all these capabilities, but it is the ultimate goal of many research efforts.