A closer inspection of the behaviour ofIGNOREandSMARTSTART, resp., versus the behaviour of REPLANandAVGFLOWREPLAN, resp., shows:REPLAN is unstable under∆-reasonable load because of infinite deferment of requests, which can not happen inIGNORE, sinceIGNOREdoes not replan often enough to defer requests.
On the other hand: reoptimizing less frequently means leaving out opportunities to improve, and thus, on average,IGNOREis empirically worse thanREPLAN. The key idea to combine the advantages of both policies is to constrain the reoptimization thatREPLANperforms. The result is the following online algorithmDELTAREPLAN, so far unpublished, which works as follows:
Whenever a new request becomes available,DELTAREPLANcomputes a prelim-inary transportation schedule for the setRof all available requests by solving the problem of minimizing the total completion time of Roffline under the restriction that no request in the transportation schedule has predicted flow time more than 2∆. If the makespan of the optimal transportation schedule is at most∆, the new schedule is accepted and becomes the active schedule. The new schedule is rejected otherwise, whence the previous schedule is kept active. It then moves the server ac-cording to the active schedule until a new request arrives or the schedule is done.
Note, that the new requests that trigger the reoptimization are not rejected. It is the new schedule that is rejected. Thus, since we do not allow rejection of requests,
DELTAREPLANis only feasible if each request is in an accepted schedule, sooner or later.
Summarized, we define:
Definition 9 (Algorithm DELTAREPLAN). Algorithm DELTAREPLAN (∆,ρ) has parameters∆ >0,ρ>1 (indicating that it aims at(∆,ρ)-reasonable request sets) and works with an internal buffer holding an active schedule and possibly some requests. It may assume the following states (initially it is IDLE):
IDLE Wait for the next point in time when requests become available. Goto PLAN.
PLAN Produce a preliminary transportation schedule for all currently available requestsR(taken from the buffer) minimizingcomp forRoffline under the con-straint that no request has a predicted flow time exceeding 2∆, possibly by a
ρ-approximation algorithm. If the problem is infeasible or the computed comple-tion time exceeds∆reject the new schedule and keep the old one active, thereby buffering the new requests. Otherwise replace the active schedule by the new one. Goto BUSY.
BUSY Serve requests according to the active schedule. If a new requests is re-leased or the active schedule is done, goto PLAN.
The result is:
Theorem 6.Let∆>0andρ≥1. For all instances with(∆,ρ)-reasonable request sets, AlgorithmDELTAREPLAN(∆,ρ) employing aρ-approximation algorithm for reoptimization yields a maximal flow time of no more than2∆.
Proof. As long as all new schedules are rejected,DELTAREPLAN (∆,ρ) works in the same way asIGNORE. Whenever a new schedule is accepted the constraints on the flow times of the scheduled requests guarantee the bound by construction. Since no schedule of length larger than∆ is accepted, rejection of all optimal schedules thereafter yields a maximal release span for buffered requests of at most∆. The buffered requests can therefore theoretically be served in time at most∆/ρ. Because
DELTAREPLAN(∆,ρ)employs aρ-approximation algorithm, it computes a schedule of length at most ∆. Since all requests during the work on a schedule have been ignored, the flow times of them are exactly the flow times IGNORE would have produced. Thus, the flow time constraints are satisfied for all of them. Therefore, the first computed schedule after the work on the active schedule has finished will be accepted. Consequently, every request will be in an accepted schedule at some point. Thus, the claim holds. ut
What happens if we do not know, how reasonable the request sets are going to be, i.e., if we do not know(∆,ρ)in advance? Let us restrict to the case with approx-imation factorρ=1 in order to concentrate on the core aspect. IfDELTAREPLANis run with a∆0<∆ on a∆-reasonable request set then still all schedules that would be rejected withDELTAREPLAN(∆) would also be rejected byDELTAREPLAN(∆0).
A problem may occur that when the active schedule is done, the new schedule has makespan larger than∆0so that we have to reject it; but then we are stuck. We can then modifyDELTAREPLANin three ways to by-pass this problem:
IGNORE-DELTAREPLAN Accept all schedules that are computed because the old schedule is done.
DOUBLE-DELTAREPLAN Take∆00:=2∆0as a new estimate of∆and runDELTARE
-PLAN(∆00). This is often calleddoubling techniquefor parametrized algorithms [4].
DELTAREPLAN Take the makespan∆00of the new schedule (which is at most∆) as a new estimate of∆and runDELTAREPLAN(∆00).
The first option usesIGNOREas a back-up wheneverDELTAREPLAN(∆0) fails to produce a schedule. This way, we obtain the same bound 2∆ on the flow times but we may lose some efficiency due to too many rejected schedules.
Theorem 7.Let∆>0andρ≥1. For all instances with(∆,ρ)-reasonable request sets, Algorithm IGNORE-DELTAREPLAN employing aρ-approximation algorithm for reoptimization yields a maximal flow time of no more than2∆. ut
The estimate for∆ in the doubling technique will at some point surpass the true∆. Then, we still get a bound on the flow times, but only with respect to the over-estimated∆, i.e., a bound of 4∆ in the worst case.
Theorem 8.Let∆>0andρ≥1. For all instances with(∆,ρ)-reasonable request sets, AlgorithmDOUBLE-DELTAREPLAN employing aρ-approximation algorithm for reoptimization yields a maximal flow time of no more than4∆. ut
Since forDELTAREPLANthe estimates for∆ never exceed∆ and the reoptimization problems as well as the acceptance of new schedules are at least as constrained as for DELTAREPLAN(∆), we conclude that DELTAREPLANhas flow times bounded by 2∆, and the loss of efficiency is decreasing as the estimate of∆ gets closer and closer to∆. We obtain the following result:
Theorem 9.Let∆>0andρ≥1. For all instances with(∆,ρ)-reasonable request sets, AlgorithmDELTAREPLANemploying aρ-approximation algorithm for reopti-mization yields a maximal flow time of no more than2∆. ut
This basic DELTAREPLAN-technique can be applied in much more general situa-tions (see [20] for a sketch). We arrived at an algorithm very much in the spirit of MPC with ingredients from online-optimization and queuing theory: For a classi-cal problem in online optimization, estimate the characteristic difficulty of the input stream in terms of∆, the definition of which was inspired by queuing theory, and use cleverly constrained reoptimization model with a suitable objective to obtain a strongly stable system.
13 Conclusion
We have shown how naive reoptimization policies in the control of elevators may lead to unstable systems. Moreover, via the notion of (∆,ρ)-reasonable load we found a modification of the usual reoptimization policies that achieves strong sta-bility, a new notion aiming at stability in worst-case analysis in a queuing system.
The new notions and the policies emerge as a combination of paradigms from basic online-optimization, queuing theory, and model predictive control. We conjecture that closing the gap between these fields will lead to interesting, sometimes surpris-ingly simple but yet useful innovations.
The analysis under reasonable load is valid in much larger generality. Essen-tially, every system in which servers have to serve requests can be captured. This encompasses also general dial-a-ride problems. A generic formulation of the princi-ple based on a generic integer linear programming formulation of the offline version of some online problem is presented in [20]. We did not present this here for the sake of a less abstract exposition.
There are a couple of open questions in this area:
• DoesMAXFLOWREPLAN produce bounded flow times in terms of∆ under∆ -reasonable load?
• The policies in this chapter are all based on the computation of higher-level in-formation, namely a precomputed schedule. On this higher level, there is no im-mediate notion of a “terminal state”. Is there any version of “terminal state con-straints” or “terminal costs” for the snapshot problem that can guarantee stability of the corresponding replan policy?
• Of course, since the reasonability∆ is a worst-case measure, performance may benefit if∆ is considered as a dynamically changing property of the request set which should be estimated in a time-dependent fashion in order not to use a too large∆ most of the time; especially, when there are traffic peaks. Can one rigorously quantify the benefits of such a dynamic approach?
• We have no non-trivial theoretical guarantees for the expected average flow-times over a distribution of request sets. Does DELTAREPLAN have provably better average flow times thanIGNORE, as it seems empirically?
• Experience shows that minimizing the averagequadraticflow times in the snap-shot problem leads to empirically stable systems. Can one guarantee strong sta-bility for them?
The LCCC theme semester revealed that quite a few types of logistic control prob-lems are attacked by more than one mathematical community; up to now rather in isolation than in cooperation. We would be very happy if this volume—and, in par-ticular, this chapter—motivated a thorough performance comparison. More specif-ically: what can be achieved, in theory and practice, by the various techniques in queuing theory, model predictive control, stochastic dynamic optimization, and on-line optimization ona common set of problems?
∗
We thank two anonymous referees for helpful comments on the presentation of this chapter. The second author is grateful for the opportunity to participate in a very inspiring theme semester at LCCC and the financial support by LCCC.
References
1. Ascheuer, N., Krumke, S.O., Rambau, J.: Online dial-a-ride problems: Minimizing the com-pletion time. In: Proceedings of the 17th International Symposium on Theoretical Aspects of Computer Science, vol. 1770, pp. 639–650. Springer (2000)
2. Atallah, M.J., Kosaraju, S.R.: Efficient solutions to some transportation problems with appli-cations to minimizing robot arm travel. SIAM Journal on Computing17, 849–869 (1988) 3. Ausiello, G., Feuerstein, E., Leonardi, S., Stougie, L., Talamo, M.: Competitive algorithms
for the traveling salesman. In: Proceedings of the 4th Workshop on Algorithms and Data Structures (WADS’95),Lecture Notes in Computer Science, vol. 955, pp. 206–217 (1995) 4. Borodin, A., El-Yaniv, R.: Online Computation and Competitive Analysis. Cambridge
Uni-versity Press (1998)
5. Crites, R.H., Barto, A.G.: Improving elevator performance using reinforcement learning. In:
S. Touretsky D. C. Mozer M. E. Hasselmo M. (eds.) Advances in Neural Information Pro-cessing Systems 8. MIT Press, Cambridge MA (1996)
6. Crites, R.H., Barto, A.G.: Elevator group control using multiple reinforcement learning agents.
Machine Learning33(2–3), 235–262 (1998)
7. Fiat, A., Woeginger, G.J. (eds.): Online Algorithms: The State of the Art,Lecture Notes in Computer Science, vol. 1442. Springer (1998)
8. Frederickson, G.N., Guan, D.J.: Nonpreemptive ensemble motion planning on a tree. Journal of Algorithms15, 29–60 (1993)
9. Frederickson, G.N., Hecht, M.S., Kim, C.: Approximation algorithms for some routing prob-lems. SIAM Journal on Computing7, 178–193 (1978)
10. Gross, D., Harris, C.: Fundamentals of queueing theory. Wiley Series in Probability and Statistics. Wiley (1998)
11. Gr¨otschel, M., Hauptmeier, D., Krumke, S.O., Rambau, J.: Simulation studies for the on-line dial-a-ride-problem. Preprint SC 99-09, Konrad-Zuse-Zentrum f¨ur Informationstechnik Berlin (1999). URL http://opus4web.zib.de/documents-zib/401/SC-99-09.pdf. Extended ab-stract accepted for presentation at Odysseus 2000, first workshop on freight transportation and logistics, Crete, 2000
12. Hauptmeier, D., Krumke, S.O., Rambau, J.: The online dial-a-ride problem under reasonable load. In: Proceedings of the 4th Italian Conference on Algorithms and Complexity,Lecture Notes in Computer Science, vol. 1767, pp. 137–149. Springer (2000)
13. Hauptmeier, D., Krumke, S.O., Rambau, J., Wirth, H.C.: Euler is standing in line—dial-a-ride problems with FIFO-precedence-constraints. Discrete Applied Mathematics113, 87–107 (2001)
14. Hiller, B., Vredeveld, T.: Probabilistic analysis of online bin coloring algorithms via stochastic comparison. In: D. Halperin, K. Mehlhorn (eds.) ESA,Lecture Notes in Computer Science, vol. 5193, pp. 528–539. Springer (2008)
15. Kellerer, H., Tautenhahn, T., Woeginger, G.: Approximability and nonapproximabiblity results for minimizing total flow time on a single machine. In: Proceedings of the Symposium on the Theory of Computing (1996)
16. Klug, T., Hiller, B., Tuchscherer, A.: Improving the performance of elevator systems using exact reoptimization algorithms. Tech. Rep. 09-05, Konrad-Zuse-Zentrum f¨ur Information-stechnik Berlin (2009)
17. Krumke, S.O.: Competitive analysis and beyond. Habilitationsschrift, Technische Universit¨at Berlin (2002)
18. Krumke, S.O., Rambau, J., Torres, L.M.: Realtime-dispatching of guided and unguided auto-mobile service units with soft time windows. In: R.H. M¨ohring, R. Raman (eds.) Algorithms – ESA 2002, 10th Annual European Symposium, Rome, Italy, September 17–21, 2002, Pro-ceedings,Lecture Notes in Computer Science, vol. 2461. Springer (2002)
19. Putermann, M.L.: Markov Decision Processes. Wiley Interscience (2005)
20. Rambau, J.: Deferment control for reoptimization – how to find fair reoptimized dispatches.
In: S. Albers, R.H. M¨ohring, G.C. Pflug, R. Schultz (eds.) Algorithms for Optimization with Incomplete Information, no. 05031 in Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum (IBFI), Schloss Dagstuhl, Germany (2005). URL http://drops.dagstuhl.de/opus/volltexte/2005/66 [date of citation: 2010-12-28]
21. Shmoys, D.B., Wein, J., Williamson, D.P.: Scheduling parallel machines on-line. SIAM Jour-nal on Computing24(6), 1313–1331 (1995)
22. Tar˘au, A.: Model-based control for postal automation and baggage handling. Ph.D. thesis, Technische Universiteit Delft (2010)