Planning

of running several agents toward the ball at once. To our knowledge, the cur-rent formal model of hybrid automata and their tools do not provide means for modeling these types of situations. Therefore, it seems to be useful to extend hybrid automata in a way that allows them to combine both reactive and deliberative decision making. This combination can avoid catastrophic failures and provide better quality of decisions in time constrained dynamical environments. Consequently, the formal verification of hybrid automata, by means of reachability analysis can be used as planning-problem solver where a plan can be achieved, iff the final plan is reachable. Hence the trajectory from the initial state to the reachable goal will accommodate the solution of this plan.

This chapter contributes to use hybrid automata as a conceptual model for planning and it goes toward enhancing the decision making of the hy-brid automata in order to improve the future outcomes of models. This can be accomplished by letting discrete transitions occur on the basis not only of reactive decisions of the continuous evolution of the variables, but also of particular preference functions. The expressiveness of the CLP prototype presented in Chapter 4 facilitates to implement this extension. To demonstrate the idea of this chapter, we present an example taken from supply chain man-agement in continuous dynamic environment. As far as we know, this is the first attempt to use hybrid automata for planning multi-agent systems whose decisions rely on a performance measurement.

The rest of this chapter is organized as follows: In Sec.7.2 we first review the planning, show its relation to model checking framework, and show the planning using deliberation. Sec.7.3 Introduces the scenario, which illustrates the approach of this chapter. Then formal definitions of extended hybrid au-tomata are discussed in Sec.7.4. Finally, Sec.7.5 shows how to specify and analyze the planning requirements.

7.2 Planning

Planning in artificial intelligence is decision making about actions to be taken.

Generally, the classical planning problem can be formulated as follows: given

• a description of the known part of the initial state of the world denoted by I,

• a description of the goal, denoted by G, and

• a description of the possible actions that can be performed,

then, the solution of the planning determines the sequence of actions in order to reach G from I under achievement a certain objective.

In the last decade, the term multi-agent planning [de Weerdt et al., 2005]

has been introduced as an approach to the planning problem with complex goals that divides the problem into sub-problems and allows each agent to deal with each sub-problem. The solutions to the sub-problems have to be combined and coordinated afterwards to achieve a coherent and feasible solu-tion to the original problem. According to de Weerdt et al., multi-agent plan-ning is defined as: given a description of the initial state, a set of global goals, a set of at least two agents and for each agent a set of its capabilities and its private goals, find a plan for each agent that achieves its private goals, such that these plans are jointly coordinated and the global goals are met as well.

7.2.1 Planning as Model Checking

In the last few years, several research has approached planning with formal methods based on model checking [Giunchiglia and Traverso, 2000]. The key idea behind this approach refers to the strong relation between the framework of model checking and planning. The framework of model checking consists of a formal model M of a system, an initial state s₀of the system, and a formal specification of a propertyψ to be verified in this system. The model check-ing aims at verifycheck-ing ifψis satisfied in M, i.e. M,s0ψ. Basically, the model checker is an algorithm that takes(M,s₀,ψ)as input and systematically vis-its the states of the model M, in order to verify if the propertyψ holds. The model checkers returns success if M satisfies the property ψ; otherwise, it returns a counter-example, that is a state in the model M where the prop-ertyψ is violated. In this framework, the planning problem can be formally described in a way that the model M describes the planning environment’s dynamics, s₀describes the initial state of the environment and the propertyψ describes the goal to be achieved. So, if M,s₀ψ, the planner returns a plan, i.e. the behavior which allows the systems to achieve its goal; otherwise, the planner returns failure. Fig. 7.1 shows the relation between model checking and planning.

Using the framework of model checking, the solution of the plan is the tra-jectory holding the sequences of reached states (actions) from a starting state to a goal state. Another way to find the plan is to use the counter example—

generated from a model checker—as a solution of the planning problem. The negation of the goal is stated as safety property and introduced to the model checker. If the problem is found to be reached, one of the powerful points of model checkers is to generate a counter-example, which can be used to provide a solution to the problem. This idea has been adopted by several

re-7.2 Planning 101

search, such as [Giunchiglia and Traverso, 2000; Pistore and Traverso, 2001;

Pereira and Barros, 2008].

The classical way to solve the planning problem has been focused on find-ing any solution plan without careful consideration of quality of the plan. For many practical problems, the problem is not only to find a plan, but also to achieve a certain objective at the end of the plan. The objective of the planning, according to [Nau et al., 2004], can be specified in several different ways as follows:

• The simplest specification consists of a goal state G, and the objective is achieved by any sequence of state transitions that reaches the goal states. In a logistic scenario, for example, the objective to have a ship-ment reached to its final destination.

• The objective is to satisfy some condition over the sequence of states fol-lowed by the system. For example, one might want to require states to be avoided during the planning, e.g. reaching after deadline .

• The specification of objective based on a utility function with penalties and rewards. The goal is to optimize some function of these utilities, e.g.

sum, maximum, minimum, over the sequence of the states followed during the planning.

Thus, the assessment of the planning objective is crucial to determine the quality of the plan.

Model checker

Planner initial state s₀

initial state s₀ model M

propertyψ

model M

propertyψ

plan or failur

success or counter example

Fig. 7.1. Planning versus Model checking.

7.2.2 Deliberative Actions

During planning, if things do not work as expected, agents must be able to react and reconsider the plan. For instance, if an agent runs into unexpect-edly high traffic on its chosen route through the city, then it must be able to consider changing the plan. When there are alternative actions to react to the

unexpected changes during the plan, the agent should deliberate to select the best alternative way. The deliberation process generally focuses in the ways to achieve a goal and the decisions of which goal to be achieve. Deliberation is particularly useful in hazardous environments where the correct action se-lection is crucial. Decker and Lesser [1998] have stated that an agent should deliberate, if one of the following conditions is met:

• The agent has a choice of actions and the choice affects performance.

• The order in which activities are carried out affects performance.

• The time at which actions are executed affects performance.

If the agent deliberates rationally, it will try to find the best ways it can per-form the actions. In other words, when the agent perper-forms a certain action A, it will try to do so in a way that maximizes the expected utility of A. So to de-cide whether to perform A, the agent should assume that it will be performed in the best way, i.e. the value of expected utility of A should be the maxi-mum value for all the ways in which the agent can think of for performing A. The expected value of an action is defined, according to the decision the-ory [c.f Berm ´udez, 2009] to be the expected value of the environment when the action is performed. In the decision theory, standard models of decision-making involve calculations of the expected utility of each available action.

Starting with each possible outcome, multiplying the utility of that outcome by the probability of the condition of the environment in which it will come about. Summing of the values that obtained for each of the possible outcomes of a certain action, gives the expected utility of that action. Standard models of decision-making identify the rational resolutions of decision problems as those that maximize expected utility.