16. How well can the proposed algorithm approximate an orcale-based optimal solu-tion that would require a priori knowledge about worker performance?
17. How well does the proposed algorithm perform compared to conventional algo-rithms which either do not learn or which learn in a simpler fashion?
3. We prove the equivalence of the energy minimization problem to a multi-dimensional knapsack problem and thereby derive the complexity of the opti-mization problem.
4. Based on analytical and numerical evaluation, we derive conditions with respect to the topology, system parameters, and task context under which computation offloading in multi-hop networks is beneficial.
5. We find in numerical simulations that the proposed context-aware greedy heuris-tic algorithm yields near-optimal results under various network settings and task contexts.
6. We study the computational complexity of the proposed context-aware greedy heuristic algorithm and the overhead of the proposed centralized architecture of decision making with respect to its communication requirements.
Chapter 4 addresses the problem of context-aware caching at the edge for cache hit maximization, and answers Questions 7-12 by the following contributions:
7. We propose a model for context-aware proactive caching in a local cache at the edge of the wireless network. The model explicitly allows different content to be favored by different users and includes that content popularity depends on the user’s context.
8. We use a decentralized architecture of decision making and take a machine-learning-based approach. Based on a contextual MAB framework, we present an online learning algorithm for context-aware proactive caching that incorpo-rates diversity in content popularity across the user population, takes into account the dependence of the users’ preferences on their contexts, and supports service differentiation. Using this algorithm, the controller of a local cache can learn context-specific content popularity online by regularly observing context infor-mation of connected users, updating the cache content, and observing cache hits subsequently.
9. We study the computational complexity of the proposed context-aware proac-tive caching algorithm and its overhead in terms of memory and communication requirements.
10. We show possible extensions of the proposed context-aware proactive caching algorithm. Specifically, we consider its combination with multicast transmissions, the incorporation of caching decisions based on user ratings, the inclusion of asynchronous user arrivals, and the extension to multiple local caches.
11. We analytically bound the loss of the proposed context-aware proactive caching algorithm compared to an oracle that has a priori knowledge about content pop-ularity. We derive a sublinear upper regret bound, which characterizes the learn-ing speed and proves that the proposed algorithm converges to the optimal cache content placement strategy that maximizes the expected number of cache hits.
12. We numerically evaluate the performance of the proposed context-aware proactive caching algorithm based on a real world data set. A comparison shows that by exploiting context information in order to proactively cache content for currently connected users, the proposed algorithm outperforms reference algorithms.
Chapter 5 investigates the problem of context-aware worker selection for performance maximization in mobile crowdsourcing (MCS) with non-spatial tasks, and answers Questions 13-17 by the following contributions:
13. We propose a model for context-aware worker selection in an MCS application.
The model allows different task types by using the concept of task context to describe the features of a task. The model describes worker performance as a possibly non-linear function of the task context and of the worker context.
14. We use a hierarchical architecture of decision making and take a machine-learning-based approach based on a contextual MAB framework. We propose a context-aware hierarchical online learning algorithm for worker selection in MCS applications with non-spatial tasks. The algorithm learns online without requiring a training phase. By adapting and improving the worker selection over time, the algorithm can hence achieve good results already during run time. The proposed algorithm is split into two parts, one part executed by the MCSP, the other part by local controllers (LCs) located in each of the workers’ mobile de-vices. An LC learns its worker’s performance online over time, by observing the worker’s personal contexts and her/his performance. The LC learns from its worker’s contexts only locally, and personal context is not shared with the MCSP. Each LC regularly sends performance estimates to the MCSP. Based on these estimates, the MCSP takes care of the worker selection. This hierarchical coordination approach enables the MCSP to select suitable workers for each task based on what the LCs have previously learned.
15. We study the computational complexity of the proposed context-aware hierar-chical online learning algorithm and its overhead in terms of local memory and communication requirements. Moreover, we analyze how many times the perfor-mance of each worker has to be observed. Keeping this number low is crucial
since observing worker performance requires quality assessments, which may be costly.
16. By establishing an analytical upper regret bound, we provide performance guar-antees for the learned worker selection strategy and prove that the proposed context-aware hierarchical online learning algorithm converges to the optimal worker selection strategy.
17. We numerically evaluate the performance of the proposed context-aware hier-archical online learning algorithm based on synthetic as well as real data using different worker performance models. A comparison shows that by exploiting context information for worker selection, the proposed algorithm outperforms reference algorithms.
Finally, the main conclusions of this thesis and a brief outlook on future research directions are presented in Chapter 6.
Chapter 2
Context-Aware Decision Making in Wireless Networks
2.1 Introduction
Along with the new paradigm of understanding wireless networks as networks of dis-tributed connected resources, new techniques are envisioned that jointly consider and leverage different types of resources in order to improve the system performance. In order to optimize resource usage, these techniques requirecontext-aware decision mak-ing [BWL18, MSS13], as motivated in Section 1.2. A context-aware decision-making framework for such a technique essentially consists of the following two parts:
(i) A system model needs to be formulated, consisting of five components, of which an overview will be given in Section 2.2.1. In particular, to allow for context-aware decision making, the designer needs to define a context model, specifying which context is needed for decision making and which sources should acquire the context by using their data collection resources [Hen03, PZCG14, MSS13].
Moreover, one or several adequate decision agents need to be identified and an appropriate architecture of decision making needs to be designed within which the decision agents are responsible for decision making [Lun92, KB97, FCGS02].
(ii) The decision agents need to be properly designed, which requires to model and classify the problem to be solved by the decision agents and to velop an appropriate method according to which the decision agents take de-cisions [KAC+15, SNHH15, JZR+17, MH16].
Context-aware decision making for new techniques that jointly consider and exploit different resources in wireless networks can be understood asan interaction between de-cision agents and the environment [KAC+15,SB98]. By designing (i) the system model and (ii) the decision agents, the specific properties of this agent-environment interac-tion are determined. Figure 2.1 shows an illustrainterac-tion of a general agent-environment interaction and connects the components of a context-aware decision making framework with this general agent-environment interaction. More specifically, Figure 2.1 shows a set of decision agents interacting with the environment by taking actions based on
Figure 2.1. General agent-environment interaction and overview of context-aware de-cision making. Five components of system model for context-aware dede-cision making shown in brown and design of decision agents shown in yellow.
observations [KAC+15] and interacting with each other via communication. Moreover, Figure 2.1 depicts how the five components of the system model for context-aware de-cision making in wireless networks, shown in brown, and the design of dede-cision agents, shown in yellow, relate to the general agent-environment interaction. These relations will be explained in the following sections.
In Section 2.2, we introduce the considered system model for context-aware decision making in wireless networks, by first giving a brief overview of its five components in Section 2.2.1, and then discussing in more detail two of the components, the context model in Section 2.2.2 and the architecture of decision making in Section 2.2.3. Finally, in Section 2.3, we discuss the design a decision agents by pointing out different methods for decision making. In this regard, we also give a short introduction to two specific types of decision-making methods which are relevant for the remainder of this thesis, namely, optimization-based approaches and machine-learning-based approaches using multi-armed bandit frameworks.