Chapter 5
Chapter 5 CrowdNav: A Sample Application
is based on SUMO (Simulation of Urban Mobility)[15] andTraCi(Traffic Control Interface)[27]. CrowdNav consists of a number of cars traveling in a virtual city following the itineraries of their drivers and of a centralized navigation service (see5.2).
A fraction is guided by the smart navigation service, whilst the others are controlled by the standard built-in routing algorithm of SUMO.
The smart routing service is controllable via multiple hyperparameters, which are summarized in table5.1For instance, randomness is introduced by route randomization in order to avoid giving the same routes to a large number of drivers. The data freshness threshold determines when dynamic traffic information is regarded expired.
Figure 5.2: The graphical user interface of CrowdNav: Cars are driving in a virtual city, some of them are guided by the smart navigation system (red).s
Name Description Range
route randomization
degree to which random noise is introduced introduced to avoid giving same routes
[0,0.3]
exploration percentage controls the ratio of smart
cars used as explorers [0,0.3]
static info weight controls the importance of
static information on routing [1,2.5]
dynamic info weight controls the importance of
dynamic information on routing [1,2.5]
exploration weight controls the degree of
exploration of the explorers [5,20]
data freshness threshold
threshold for considering observed traffic-related data as stale and disregard it
[100,700]
re-routing requency controls how often the route would be
invoked to re-route a smart car [10,70]
Table 5.1:CrowdNavhyperparameter space.
The goal is to optimize the quality of CrowdNavwith respect to the average trip overhead. The overhead for each trip is defined as
y= actual duration of a trip
theoretical duration of a trip ≥1. (5.1) The system has the character of a noisy, expensive black-box: the simulation is complex and the relationship of input parameters xand trip overheady is not known in advance. Further, each simulation has to run some time which makes evaluations expensive. Ultimately, function evaluations are noisy: the trip overhead doesn’t only depend on the route proposed by the router, which itself has stochastic components (e. g. the rout randomization factor), but particularly on other cars and the traffic which is generated stochastically. Noise can be smoothed out the longer we run the simulation.
This application raises again the question of how much budget in terms of simulation runtime should be spent on each configuration.
For observing data produced by CrowdNav for different configurations, we use the Real-Time Experimentation (RTX) tool [23]. In this thesis, existing work by Gerostathopoulos et al. [10] was extended by the integration of SMBO with inten-sification, which is employed to find the optimal hyperparameters for the smart navigation system. As surrogate model, a random forest is fitted on the non-aggregated data as random forests turned out to be more stable in the scope of this application.
New points were proposed by the lower confidence bound criterion. In order to get good results in a reasonable amount of time we tune over two hyperparameters only:
the route randomization factor and the data freshness threshold. In this example, one evaluation corresponds to one trip. We use an initial Latin hypercube design of size 10. Every time a new parametric configuration is set in the router, the first 2000 trips are omitted from analysis as it takes some time until the system adapts (burn-in rate). Every configuration is evaluated 1000 times. After 80 optimization iterations the identification phase is started with bOCBA = 10000 evaluations. The algorithm terminates, if a minimum P(CS) of 0.7 or a maximum number of 20 identification iterations is reached.
Figure5.3 shows the 20 best configurations and the 90% percent confidence intervals around the mean trip overhead for the corresponding configuration. Though it seems that the identification step in the end allows a clear distinction between the best point and inferior points, this is questionable when looking at the configurations in the 2-dimensional hyperparameter space (see figure5.4). We see that good points are very close which can have two reasons: either the effects observed are random and appear because noise has not been smoothed out sufficiently or the true underlying mechanism
Chapter 5 CrowdNav: A Sample Application
in fact very unsmooth. In future work, it still needs to be clarified if higher-level hyperparameters like the burn-in rate or the initial evaluation budget of 1000 are sufficient to capture the noise level. The performance of sequential-based optimization in this application needs to be validated in a more extensive experimental study.
Figure 5.3: Resulting top 20 configurations after using SMBO with identification as optimizer for CrowdNav. The figure shows the average trip overhead for the top 20 configurations. 90% confidence intervals for ¯y(i) were calculated based on assuming normality of simulation outputs.
Figure 5.4: The two-dimensional hyperparameter space that was tuned over in this application and the configurations evaluated in the course of hyperparameter optimization.
Chapter 6 Conclusion
In this work we formulated the general SMBO with intensification algorithm which extends sequential model-based optimization by a replication strategy. We have identified four replication strategies from literature: the simple fixed strategy, replication budget allocation through ocba and inc and inc+ that iteratively perform races against the incumbent configuration.
In an experimental study, the replication strategies have been compared: whereas experiments on artificial test functions have shown that replication strategies can improve the optimization result considerably in the 20-dimensional case, the SKO algorithm that employs no replication strategy at all could not been outperformed by those methods employing replication. Based on the benchmark on artificial test functions, theocbareplication strategy together with a noisy Gaussian process surrogate learner was the best performing instance of SMBO with intensification.
Results of deploying SMBO with intensification in machine learning tuning problems suggest that replication strategies can improve the tuning outcome, but results vary between datasets and thus do not allow clear conclusions or recommendations for practical purposes. In future work, these methods will be investigated on more machine learning problems.
From experiments performed on the artificial test functions, we analysed the final error made by the respective optimization algorithms in more depth: often, a fraction of the overall error is attributable to false identification of the final best point in the end.
Motivated by the OCBA theory, we proposed a new version of sequential model-based optimization that tackles this type of error in a final identification step and provides the user with a sort of confidence in this returned solution: the probability of correct selection. While a first experimental investigation of the performance of this algorithm for optimization of artificial test functions and in machine learning hyperparameter tuning yields promising results, this method needs to be evaluated in more depth in future work.
We emphasize that the point in time, when optimization is stopped, and the identifica-tion phase is started has an impact on the final outcome. In our experiments, we simply defined a budget for the optimization phase. When it was exceeded, the identification phase was started. Two suboptimal scenarios can occur: First, the optimization phase
Chapter 6 Conclusion
is quit too early and the identification phase is triggered when the function was not optimized sufficiently. In this case, multiple suboptimal configurations are compared and the identification phase won’t yield useable results. Second, the optimization phase takes too long. In this case, the identification phase could have been entered earlier and budget could have been saved. In future work, we will investigate how to find this
“sweet spot” of switching from optimization to identification. Furthermore, we might look into in better notions of confidences in solutions.
Finally, we employed the new proposed method on a black-box optimization problem for parametric traffic simulation optimization to tackle the question of how much simulation budget should be spent on single configurations. In future work, this we will analyse the functioning of SMBO with identification in a more extensive benchmark study.