We have argued that a representation of socially adequate behavior is essentially based on qualitative concepts of spatial configurations. While qualitative constraint networks provide the means to represent aconfiguration, the representation of aconventionalso involves inscribed behavior. Conventions inscribe behavior within a certain context, typically triggered by an event or associated with a process (e.g., keep left when standing on an escalator). Among the different options to represent behavior, we choose to apply a temporal logic that links spatial configuration knowledge (snapshots) to temporal sequence. This approach has several nice implications:
• Employing a temporal logic to connect qualitative representations of snapshots allows static as well as dynamic spatial knowledge to be represented with the same vocabulary of spatial concepts.
• Like any standard logic, it allows non-spatial and non-temporal knowledge to be repre-sented aside the spatio-temporal knowledge using propositional symbols.
• Last but not least, temporal logic has been integrated with motion planning and robot control (Kress-Gazit and Pappas, 2010; Ding et al., 2011; Kloetzer and Belta, 2010) and recognition of processes (Kreutzmann et al., 2013).
Of course, this approach has limitations. A technical limitation are missing quantifiers, thus the approach requires a predetermined maximum number of distinct objects to consider. A potential knowledge modeling limitation can be seen in binary truth evaluation of classic logics, e.g., if handling graduated, fine-grained concepts.
6.5.1 QLTL: Linear Temporal Logic with Qualitative Spatial Primitives
For representing social conventions we require temporal sequence knowledge, i.e. which situation occurs before/after another. This can be achieved using a lean logic like Linear Temporal Logic (LTL) (Pnueli, 1977). LTL is a modal logic that extends propositional logic with modal operators which connect statements about snapshots (calledworldsin modal logic) using dedicated modal operators. In order to integrate this logic with spatial primitives, we choose to encode qualitative spatial relations in terms of propositional symbols and we extend the semantics to also include a spatial semantic.
Since conventions also depend on the types of objects involved in a certain configuration (person, obstacle, robot, etc.), we also introduce sorts to retain category information in logic formulae. This leads to the following syntax definition for our logic QLTL. The ingredients are:
• a set of spatial symbolsS. Let s be a number of sorts, then Si = {si,1, si,2, . . .}, i = 1, . . . , sare sets of spatial symbols andS :=S
i=1,...,sSi
6.5 Formalization of Social Conventions
• R={r1, . . . , rn}is a set of qualitative relation symbols
• F ={f1, . . . , fl}be a set of function symbols
• G={g1, g2, . . .}is a set of propositional symbols for representing general, non-spatial knowledge
• The set of propositionsP is defined asP :=G∪ {r(s, t)|r∈ R, s, t∈(S ∪ {f(si)|f ∈ F, si ∈ S})}.
The idea underlying this definition is to use natural notation of qualitative relations so that it is possible to represent spatial knowledge by a single propositional symbol. Propositions are either describing non-spatial facts (G) or some spatial relationrbetween two objectssandt which can either be sorts or some sort dependent aspectf(si). For example,NTPP(h,sec(r)) that a humanhis standing in the security rangesecof the robotr.
Formulae in QLTL are then defined recursively:
• pis a formula for everyp∈P
• Ifφis a formula, so is¬φ
• Ifφ, ψare formulae, so isφ ⊗ ψwith⊗ ∈ {∧,∨,→,↔}
• Ifφis a formula, so isM φwithM ∈ {◦,2,}
• Ifφ, ψare formulae, so isφ N ψwithN ∈ {U, R}
The semantics of QLTL are similar to LTL, i.e., an interpretation establishes an ordered sequence of worlds. Within each single world, all propositional symbols are mapped to truth valuestrueandfalse, inducing the interpretation of formulae composed with logic conjuncts (∧,∨, . . .). For convenience of notation QLTL includes function symbols for relating the individual regions established by an agent, e.g., social space, personal space, etc. The semantics of a function f is a mapping S → S of spatial symbols. In other words, functions are used as shorthand notations for the respective spatial symbols. In QLTL we further require interpretations within all worlds to bespatially consistent, i.e., sensor interpretations must not be contradictory. This defines thespatial semanticsof QLTL. Within one world, the interpretation of all (spatial) propositionsr(s1, s2)withsi ∈ Sorsi =fj(s)for somefj ∈F, s∈ Sinduces a qualitative constraint network with variablesSand according constraintsri(x, y)wherex, y are either the spatial symbols s1, s2 or the symbols obtained by application offj(si). The spatial semantics of a relationr is defined by the respective qualitative calculus. Functions F are also assigned with a respective spatial semantic, e.g., mapping the position of a human to its estimated personal space. We say that an interpretation is spatially consistent if, first, all induced constraint networks are consistent and, second, if all mappings inF respect their spatial semantics, e.g., personal(h)is the region that determines the personal space of has defined by the functionpersonal.
The semantics of modal operations connect distinct worlds within a sequence, identical to the sematics of LTL:
◦φ(next) φholds in the following world
2φ (always) φholds in the current and in all future worlds φ(eventually) φholds in a future world,(φ↔ ¬2¬φ)
φ U ψ(until) φholds at least until eventuallyψ holds, but they dont have to hold at the same time
φ R ψ (release) ψ holds until and including the world in whichφfirst becomes true
In this work, we define the set of qualitative relation symbolsQto be the union ofOPRA4 andRCC-8relations. This allows us to represent the social conventions. Since we will obtain an interpretation from the perception of the robot, we can assume it to be spatially consistent.
6.5.2 Conventions as QLTL formulae
We introduce a convenient notation for representing conventions as QLTL formulae. Con-ventions considered in this work essentially come in an “if-then-until” flavor in the sense that observing a certain event or process triggers (start) a sequence of required configurations (effects) to achieve a behavior in accordance with the regulation until an end state or a break criteria is reached. The end criteria is reached if everybody behaves as expected while the break criteria prevents the system from getting stuck in a rule if something unexpected happens, e.g. a person involved does not behave as expected by turning around and moving away. Break criteria might be a timeout or if involved persons leave the public space of the agent. For reasons of space we do not consider break criteria in further detail. Conventions can easily be represented as QLTL formula if we pursue a declarative description of regulation-compliant behavior using the pattern
φstart → ◦ φeffectU (φend∨φbreak)
(6.1) We note that QLTL is expressive enough to allow conventions to be represented which are not effective. If, for example, the sub-formula specifying the trigger condition would refer to a future situation, than it may not be possible to decide whether the trigger condition is satisfied.
Roughly saying, we want to exclude conventions of the kind “if this will turn out to be wrong, don’t do it”. Deciding effectiveness of a convention is beyond the scope of this paper—we assume that the conventions are stated in a way that allows trigger conditions to be detected at the time they apply.
We point out an inconvenience of directly using modal logic for knowledge representation, namely its lack of variables. The pattern introduced in (6.1) can involve propositional symbols only. As a consequence, for a convention that says how to avoid an obstacle, we require separate formulae, one for each individual obstacle. To this end, we introduce a shorthand notation for conventions that supports variables. In the following we writex1 :s1, . . . , xn :sn : φwith variablesx1, . . . , xnof respective sortss1, . . . , sn, meaningV
xi∈si,i=1,...,nφ0 withφ0 obtained by substitution ofxifor the respective spatial or propositional symbol.
6.5 Formalization of Social Conventions
6.5.3 QLTL Representation of Social Conventions
We now exemplarily formalize social conventions from Class 1 and 2 (see Section 6.3.2) in QLTL. Throughout this section we userto denote the robot (r : robot), andhfor humans (h: human). In the following we usexandyto denote objects of any of these sorts.2 To refer to the social spaces constituted by a humanhwe use the functional notationintimate(h),personal(h), social(h), public(h) respectively. On the syntactical level these functions denote special symbols for referring to the regions, whereas on the semantic level the specific region must be interpreted based on the physical object specified byh. In order to improve readability of formal conventions we define some macro relations to abbreviate complex relations. First, describing that two agents are in a head on situation: HEAD ON(x, y) := x4∠15−115−1y or are moving in the same direction: SAME DIR(x, y) :=x4∠15−17−9 y∨x4∠7−915−1y. Next we definexbeing on the left or right side of y: ON LEFT(x, y) := y4∠∗2−6x,ON RIGHT(x, y) := y4∠∗11−13xand x being in front or behind y: IN FRONT(x, y) := y4∠∗15−1x, BEHIND(x, y) := y4∠∗5−11x.
Finally, by OVERLAPwe define that there is a partial or complete overlap: OVERLAP(x, y) :=
PO(x, y)∨TPP(x, y)∨NTPP(x, y)∨TPPI(x, y)∨NTPPI(x, y). Using these primitives we can formalize convention 1a as follows:
φ1astart := OVERLAP(r,social(h))∧HEAD ON(r, h)
φ1aeffect := ¬PO(r,personal(h))∧ (6.2)
ON LEFT(h, r)RBEHIND(h, r) φ1aend := BEHIND(h, r)
This means, if the robot enters or is in the social space of another agenthand they are head on, the robot must not move into the personal space ofh. Furthermore,hhas to be on the left ofr, thusrneeds to turn right untilrhas passedh, i.e.,ris behindh. Convention 1b can be modeled similar except that they are inSAME DIR(r, h)instead ofHEAD ON(r, h)andrhas to overtake on the left (ON RIGHT(h, r)).
All conventions of class 2 are covered by the following:
φ2start := OVERLAP(r,social(h))∧
(ON LEFT(h, r)∨ON RIGHT(h, r)) φ2aeffect := ¬PO(r,personal(h))∧ IN FRONT(r, h) φ2beffect := ¬PO(r,intimate(h))∧ BEHIND(r, h)
(6.3) φ2ceffect := stop(r)U BEHIND(r, h)
φ2effect := φ2aeffect∨φ2beffect∨φ2ceffect φ2end := BEHIND(h, r)
2Remark: these symbols need different interpretations regarding the relation they are used for. ForRCC-8they need to be interpreted as regions, e.g. the space covered by an object (obj(x)), and forOPRA4as oriented point (opos(x)). For brevity we omit this distinction in the presented formalizations.
If the robot enters the social space ofhon the left or right,rhas three options. Either he passes hin the front with not entering the personal space, pass behindhwith a smaller distance, i.e. it is allowed to enter personal but not the intimate space, or he can stop until the human has passed.
For brevity we only sketch the main aspects to consider formalizing the three remaining classes. In case of a narrow passage (class 3) we need to represent an overlap of obstacles with a space of the agent to interact with, e.g. a wall to the left in the social space of h:
OVERLAP(w,personal(h))∧ON LEFT(w, h). For dealing with conventions regarding groups (class 4) we need to redefine the spaces with respect to the individuals involved. One approach is to define the group space as the union of all individual spaces, e.g.social(g) :=∪h∈gsocial(h).
Dealing with class 5 is straightforward. If the robot is in a specific context, e.g. a libraryl (φstart = OVERLAP(r, l)), he has to adapt his behavior, e.g., switch to quiet mode (φeffect = q mode(r)), until he is not in the context anymore (φend =¬φstart).