5.4.1 Formal Problem Statement
In this section, based on the models presented in Section 5.3, we formulate the prob-lem of context-aware worker selection for maximizing the worker performance in MCS applications with non-spatial tasks to be solved in hierarchical fashion by the MCSP and the LCs. As stated above, we assume that tasks t = 1, ..., T arrive sequentially.
Consider now an arbitrary sequence of T task and worker arrivals, i.e., consider a sequence of tasks t = 1, ..., T with arbitrary task budgets {bt}t=1,...,T, arbitrary task contexts {ct}t=1,...,T, arbitrary worker availability {Wt}t=1,...,T and arbitrary worker contexts{xt,i}i∈Wt,t=1,...,T. The goal of the system of MCSP and LCs is to select work-ers for each task in such a way that the expected cumulative worker performance up
to task T is maximized. Based on the action model in Section 5.3.6, the problem of selecting, for each task, a subset of workers which maximizes the sum of expected performances given the task budget is given by
max
T
X
t=1
X
i∈Wt
θi(xt,i,ct)yt,i (5.6)
s.t. X
i∈Wt
yt,i ≤mt ∀t= 1, ..., T
yt,i ∈ {0,1} ∀i∈ Wt, ∀t= 1, ..., T.
with yt,i of (5.3), θi(xt,i,ct) as defined in Section 5.3.4 and the constraints from (5.4).
Problem (5.6) includes the expected context-specific worker performances θi(xt,i,ct), i.e., the expected performances of available workers as functions of their joint personal and task contexts.
5.4.2 Oracle Solution
First, we analyze Problem (5.6) under the assumption that there would be an entity that had a priori knowledge about context-specific worker performances and access to the current personal worker contexts. Hence, only in this section, suppose that there exists an entity which (i) is an omniscient oracle, knowing the expected performance of each worker under each joint personal and task contexta priori, and that this entity (ii) is centrally informed about the current personal contexts of all available workers for each arriving task.
For such an entity, Problem (5.6) corresponds to an ILP problem, cf. Section 2.3.2.2. As the sub-problems in Problem (5.6) for the different tasks are not coupled, Problem (5.6) can be decoupled into T independent sub-problems, one for each arriving task. For a task t, if fewer workers are available than required, i.e., Wt ≤ mt, the trivial optimal solution of the sub-problem associated to task t is to request all available workers to complete the task. In contrast, if for a task t, Wt > mt holds, the sub-problem associated to task t corresponds to a knapsack problem, cf. Section 2.3.2.3, with a knapsack of capacity mt and with Wt = |Wt| items, where item i ∈ Wt has a unit weight and a non-negative profitθi(xt,i,ct). Due to the unit weights, the sub-problem in this case actually is a special case of the knapsack problem that may be solved efficiently. Indeed, the optimal solution of the sub-problem can be easily computed in a running time of at most O(Wlog(W)) as follows. The optimal solution is given by ranking the available workers in Wt according to their context-specific expected performances and by selecting themt highest ranked workers.
For a task t∈ {1, ..., T}, we denote an optimal subset of workers to select for the task bySt∗ :={s∗t,1, ..., s∗t,min{m
t,Wt}}. Formally, these workers satisfy s∗t,j ∈ argmax
i∈Wt\Sj−1 k=1{s∗t,k}
θi(xt,i,ct) for j = 1, ...,min{mt, Wt}, (5.7)
whereS0
k=1{s∗t,k}:=∅. Note that several workers may have the same expected perfor-mance and hence the optimal set of workers may not be unique, which is also captured here. Moreover, note that an optimal set St∗ of workers for task t depends on the task budget bt, task context ct, price et, the set Wt of available workers and their personal contexts {xt,i}i∈Wt, but we write St∗ instead of St∗(bt,ct, et,Wt,{xt,i}i∈Wt) for brevity.
Let
S∗ :={St∗}t=1,...,T (5.8)
be the collection of optimal subsets of workers for the collection {1, ..., T} of tasks.
We call this collection thecentralized oracle solution, since it requires an entity with a priori knowledge about expected context-specific worker performances and with access to personal worker contexts to make optimal decisions.
5.4.3 Contextual Multi-Armed Bandit Formulation
Now, we characterize Problem (5.6) under the conditions actually faced by the MCSP and LCs. Namely, the set of MCSP and LC do not have a priori knowledge about expected performances, and the workers’ personal contexts are only locally available in each mobile device, but may not be shared with the MCSP.
If for an arriving task t, fewer workers are available than required, i.e., Wt ≤ mt, by simply requesting all available workers (i.e.,St=Wt) to complete the task, the MCSP automatically selects the optimal subset of workers. Otherwise, ifWt> mtholds for an arriving taskt, the MCSP cannot simply solve the sub-problem for taskt appearing in Problem (5.6) like the centralized oracle. This is because on the one hand, it does not know the expected performances θi(xt,i,ct) and on the other hand, the MCSP cannot access the workers’ personal contexts. Hence, in this case, a machine-learning-based approach, cf. Section 2.3.1, is needed since the system of MCSP and LCs can only learn the workers’ performances by selecting different workers over time and observing their instantaneous performances.
Considering the problem statement in Section 5.4.1, under the conditions actually faced by the MCSP and LCs, Problem (5.6) can be understood as a contextual MAB problem
as follows, cf. Section 2.3.3.4. The MCSP and LC can be understood as a set of agents, where one of them (the MCSP) needs to sequentially select from a set of actions. In the considered MCS problem, the set of actions is given by the set W of workers. There is a sequence of tasks t = 1, ..., T that corresponds to a sequence of rounds faced by the agents. At the arrival of a task t, only a subset Wt ⊆ W of workers may be available and hence the set of actions may be different in each round. For each arriving task, the following events happen sequentially. First, the MCSP receives the task and especially observes the task contextct. Moreover, the LC of each available workeri∈ Wtobserves its worker’s personal context xt,i. This corresponds to several contexts revealed to the agents in the beginning of a round. Secondly, the MCSP selects a subset of min{mt, Wt} workers from set Wt and requests them to complete the task. This corresponds to an agent selecting a subset of available actions. Thirdly, each LC of a requested worker observes the instantaneous performance of the worker. This corresponds to the agents receiving a reward for each selected action. Taking into account the assumptions about the arrival processes of the tasks, workers and their performances in Sections 5.3.2 – 5.3.4, Problem (5.6) corresponds to a contextual MAB problem with a similar model as the one presented in Section 2.3.3.4. The main differences between these two models are as follows:
• In Problem (5.6), the agent may select several actions per round instead of only one and the number of actions to be selected may be different in each round.
Therefore, formally, Problem (5.6) is a contextualcombinatorial MAB problem, cf. Section 2.3.3.2. However, since neither the objective function nor the con-straints in Problem (5.6) are combinatorial, Problem (4.4) is more accurately a contextual MAB problem with several action selections per round, but not of combinatorial nature.
• In Problem (5.6), actions may be unavailable in arbitrary rounds, whereas in the model in Section 2.3.3.4, actions are always available. Therefore, Problem (5.6) is a contextual MAB problem with sleeping arms, cf. Section 2.3.3.2.
• Instead of one agent as in the model in Section 2.3.3.4, Problem (5.6) has to be solved cooperatively by several agents, where one coordinating agent (i.e., the MCSP) selects a subset of actions in each round based on the estimates of a set oflearning agents (i.e., the LCs), where each learning agent observes the context of one particular action and learns the rewards of this action.
Consequently, a coordination mechanism between the MCSP and LCs needs to be de-signed in order to enable the LCs to learn their workers’ context-specific performances
over time and to enable the MCSP to select suitable workers for each task to maximize the worker performance on this task given its task budget. Specifically, over time, the system of MCSP and LCs has to use a suitable trade-off between exploration and ex-ploitation, by, on the one hand, selecting workers about whose performance only little information is available and, on the other hand, selecting workers who are likely to have high performance. For each arriving task, the selection of workers depends on the history of previously selected workers and the corresponding observed performances.
Since observing worker performance requires quality assessments that may be costly, the number of performance observations should be limited in order to keep the cost for quality assessment low. An algorithm which maps the history of previously selected workers and observed performances to the next selections of workers is called alearning algorithm. The performance of such a learning algorithm can be evaluated by com-paring its loss with respect to the centralized oracle solution given in (5.8) in terms of the achieved cumulative worker performance. Formally, for an arbitrary sequence ofT task and worker arrivals, the regret of learning with respect to the centralized oracle solution is given by
R(T) =E
T
X
t=1
min{mt,Wt}
X
j=1
ps∗
t,j(xt,s∗
t,j,ct, t)−pst,j(xt,st,j,ct, t)
. (5.9)
where pst,j(xt,st,j,ct, t) denotes the instantaneous performance of the selected worker st,j ∈ St, j ∈ {1, ...,min{mt, Wt}}, with personal worker context vector xt,i for task t with task context ct. Here, the expectation is taken with respect to the se-lections{St}t=1,...,T made by the learning algorithm and the randomness of the workers’
performances.
Equivalently, one can write the regret R(T) as R(T) =
T
X
t=1
min{mt,Wt}
X
j=1
θs∗
t,j(xt,s∗
t,j,ct)−E[θst,j(xt,st,j,ct)]
. (5.10)