Knowledge-Based Systems and Deductive Databases

Wolf-Tilo Balke Philipp Wille

Institut für Informationssysteme

Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de

Knowledge-Based Systems

and Deductive Databases

6.1 Implementation of Datalog in DBs 6.2 Top-Down-Evaluation

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- D-

Next Lecture

•  The Datalog semantics are given by Herbrand interpretations

– A Datalog program ! is a set of Horn clauses – Any Herbrand interpretation that satisfies ! is a

model

– Unfortunately, it is not quite that easy to compute a Herbrand model for !

– Also, multiple models exists per program – which conveys the intended semantic?

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- E-

Semantics of Datalog

•  Datalog^f

– Datalog^f is computationally complete

– The intended semantic of a Datalog^f program is given by the least Herbrand model

• For the least Herbrand model "#$$"$%$"&$for any other Herbrand model "&$holds

• This leads to "$'($)$*#$$whereas * is the set of all #$$whereas * is the set of all Herbrand models

• Informally: The least model is a model for ! and does not contain superfluous statements

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- F-

Semantics of Datalog

•  Operational semantics for Datalog^f

– To compute the least Herbrand model, a fixpoint iteration approach can be employed

• Start with an empty set of ground atoms

• Iteratively refine set (by adding more atoms)

• Fixpoint iteration is monotonous (set is only expanded in each iteration)

• As soon as the fixpoint is reached, set becomes stable (i.e.

no changes)

• The method is finite for Datalog^f

• The stable result is equivalent to the least Herbrand model

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- G-

Semantics of Datalog

•  Iterative Transformation step:

– Elementary production rule +_!

– Idea: Apply all given rules with premises contained in the set of the previous step

• For ,_-(./# this puts all atoms into the result

• For following steps, everything which can be followed by a single application of any rule is added

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- H-

Semantics of Datalog

•  Datalog^neg is more difficult

– Datalog^neg does not provide more expressiveness, but allows for more natural modeling

– Problems:

• Datalog^neg is potentially unsafe (i.e. generates infinite or excessively large models)

• Datalog^neg is potentially ambiguous (i.e. multiple distinctive models possible)

– In general, no least Herbrand model possible – Instead, multiple minimal Herbrand Models with

0$"$12342$567$839385:$";<7:'$=$"&$>?42$@25@$"&A"$B – Intersection of minimal models is not a model itself…B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- I-

Semantics of Datalog

•  Datalog^neg problems can be addressed by restricting possible programs

– Ambiguity: Assume negation as failure

• A non-provided fact is assumed to be false

– Safety: Enforce positive grounding

• Each variable appearing in a negative clause needs to appear in a positive clause

• Variable is positively grounded

• Evaluation can thus be restricted to known facts,

examination of the whole (potentially infinite) universe not necessary

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- J-

Semantics of Datalog

– These restrictions allow a deterministic choice of models

• Negative dependencies of ground instances induce a preference on models

• “Best” model wrt. that preference is called perfect model and is also a minimal model

• Perfect model is the intended semantics of Datalog^neg

– Operative semantics of Datalog^neg is given by iterated fixpoint iteration

• Take advantage of positive grounding and work along program partitions representing the program strata

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- K-

Semantics of Datalog

– For each strata partition, consider only rules which are positively grounded in a previous strata

– On the union of those rules and the previous ground instances, apply normal fixpoint iteration

• i.e. iterate a fixpoint iteration along the program strata

•  Both fixpoint iteration and iterative fixpoint iteration are very inefficient

– Better algorithms in the next lectures….

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LM-

Semantics of Datalog

•  In the previous week, we have seen the elementary production operator +_!$B$B

– But how can we put this operator to use?

– Many deductive DBMS do not choose to implement everything “from the scratch”

•  Especially implementations in Prolog and Lisp are very common

– However, for reliably storing huge amounts of data (e.g.

the facts in the extensional DB), there is already a wonderful technology:

Relational Databases

•  Also, most applications already use RDBs and SQL

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LL-

6.1 Datalog

in Algebra

•  In this section, we will map Datalog^neg to Relational Algebra

– This will allow us an implementation of Datalog concepts within a RDB

– Idea:

•  Take datalog program

•  Translate to relational algebra

•  Evaluate the algebra statement

•  Return results

– Also, this will allow us to take advantage of established features of databases

•  Query optimization

–  Indexing!

•  ACID properties

•  Load balancing

•  etc…

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LD-

6.1 Datalog

in Algebra

•  When using the Relational Model and

Relational Algebra, we assume the following:

– Data (i.e. facts) is stored in multiple relations

– A relation C over some sets D_E, …, D₉ is a subset of their Cartesian product

• C$%$D_E ! … ! D₉

• The sets D_E, …, D₉ are finite and are called domains

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LE-

6.1 Relational Algebra

•  Relational algebra operations available

– Base operations of relational algebra

– Derived operations

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LF-

6.1 Relational Algebra

F$B G56@7>359$H6;<?4@B

IB J7:74@3;9B

KB H6;L74@3;9B

M$B J7@$N93;9B

O$B J7@$"39?>B

PB Q;39>$R$C$P$J$S$I_T$RC$F$JU$$UB

V#$WB X7Y@$Z$C3[2@$J783$Q;39>$R$C$V$J$S$K_5@@RCU$RC$P$JU$$UB

•  In the following, we will use variants of normal relational algebra

– Attribute are referenced by their number instead by their name, e.g. \E or \]B

– When using references to relations in binary operations, e.g. joins, we may also refer to them as

^:7Y@_$or ^63[2@_B

• RC$F$JU$P^:7Y@_`\a(^63[2@_`\E$bB

– We distinguish two types of relational algebraB

• C7:c:[^d excluding the set minus operator

• C7:c:[$$including the set minus operator

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LG-

6.1 Relational Algebra

•  Examples:

– Name of hero with id=1

• K_\eI_\E(E$RfU$$B

– All powers of hero with id=2

• K_\g$RRI_\E(e$fUPRf`\E(fH`\EU$fH$PR^:7Y@_`\e(^63[2@_`\EU$H$U$$

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LH-

6.1 Relational Algebra

!"# $%&'#

L- >?#&"<N-

D- >C#:&,,#C-O-

(!"# )!"#

L- D-

D- L-

D- E-

L- E-

L- F-

!"# )*+',#

L- >,<#"<4-P+"<@3%+5#"- D- ;&%&=<"&,<,-

E- ;&%&@+0?/-

F- Q#C4&-Q<&%'-R&"&C+5#"-

-#

.#

-.#

•  In the following, we will implement a simple fixpoint iteration with relational algebra

– We will only consider safe Datalog^neg programs, i.e. negative literals and head variables are positively grounded

•  Given is a safe Datalog^neg program ! and a relational database

– Task:

• Store extensional DB in tables

• Encode intensional DB in a customized relational algebra elementary production operator

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LI-

6.1 Implementation

– Each predicate symbol 6_E#$h#$6₈ of the extensional database is assigned to a relation C_E#$h#$C_8B

•  i.e. those predicates provide the facts, each fact has its own relation

– Each predicate symbol i_E#$h#$i₈ of the intensional database is assigned to a relation j_E#$h#$j₈

•  i.e. those predicates are defined by rules

– For ease of use, we restrict each predicate to be defined either in the intensional or the extensional DB

•  i.e. each predicate which was used to define facts is not allowed to occur in the head of a rule

•  This does not limit the expressiveness of Datalog programs

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LJ-

6.1 Implementation

– The predicate symbols k#l#$m#$n#$(#$o$are assigned to the hypothetical relations p'($.X+#$q+#$X+r#$q+r#$rj#$

srj/ B

• Those relations are of infinite size and thus, of course, not stored the RDB

• We will see later that they can be removed

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- LK-

6.1 Implementation

•  Just a short consideration:

How could we map relational algebra to Datalog?

– I_\e(g$CBB tB CRu#$gU`B

– K_\E$CB B t$B C&RuU$'v$CRu#$wU`B

– CFJB B tB CJRb#$u#$w#$xU$'v$CRb#$uU#$JRw#$xU`B – C$P^:7Y@_`\E(^63[2@_`\e$J$B $ty

B B CJRb#$u#$w#$xU$'v$CRb#$uU#$JRw#$xU#$b(x`B – C$V^:7Y@_`\E(^63[2@_`\e$J$B $ty

B B CJRb#$uU$'v$CRb#$uU#$JRw#$xU#$b(x`B – C$M$JB B tB C&Ru#$wU$'v$CRu#wU`y

B B B B C&Ru#$wU$'v$JRu#wU`B

– C$z$JB B tB C&Ru#$wU$'v$CRu#$wU#${JRu#$wU`B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DM-

6.1 Implementation

•  Now, how can we translate from Datalog to relational algebra

– Some pre-processing is necessary!

•  Transform all rules of the intensional DB such that the head contains only variables

– This can be achieved by replacing any head constant with a new variable and adding a literal binding that variable to the old value

– e.g. iRu#$5#$|U$'v$X_E#$h#$$X₉$$$

t iRu#$w#$xU$'v$X_E#$h#$$X₉#$w(5#$x(|$B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DL-

6.1 Implementation

•  Change the order of the variables such that their safety is ensured by previous body literals

– A literal is unsafe, if it is potentially infinite

– e.g., CRu#wU$'v$u(w#$}RuU#$iRwU$is not in correct order as the safety u(w is not ensured by previous literals

• There are infinite possibilities for X being equal to Y

– t CRu#wU$'v$}RuU#$iRwU#$u$($wB

• is in correct order as }RuU and iRwU limit the possible values of u and wB

– We also sort positive literals before negative ones

• …for positive grounding

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DD-

6.1 Implementation

•  Each rule $C$'v$X_E#$h#$X₉$ is now transformed to relational algebra as follows

–  For each literal X_E#$h#$X₉, the respective atomic component c_3$S$}₃R@_E#$h#$@₈U$is transformed into a relational expression r_3B

•  r₃$S$I_TRH₃U$with H₃ being the relation corresponding to }_3B

•  The selection criterion θ is a conjunction of conditions defined as follows:

For each @₃, a condition is added

–  \L$($@_L$ if$@_L is a constant symbol

–  \L$($\~$$if$@_L and @_~ are the same variables

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DE-

6.1 Implementation

– Example:

• }Ru#$eU$'v$iRu#$u#$w#$eU#$6Ru#$EU$$t$RC7}:547$4;9>@59@>U$y }Ru#$xU$'v$iRu#$u#$w#$eU#$6Ru#$EU#$x$($e$t$$R+659>:5@7$@;$Cv c:[UB

B r_E$'($IR\E$($\e$$\Ä$(eU$jB B r_e$'($I_R\e$(EU$CB

B r_a$'($I_R\e$(eU$rjB

•  After treating the single literals, we will compose the body expression Å$$from left to right

–  Initialize the temporary expression Å_E'($r_EB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DF-

6.1 Implementation

•  Depending on the variables in the literals, the following expressions Å_e - Å_~$$are generated differently:B

–  Å₃'($Å_3vE$F$r_3$ iff X₃ does not contain any variables of the previous body literals,

i.e. Ç56>RX₃U$)$Ç56>R.X_E$#h#$X_3vE/U$($ÉB

•  CRu#$w#$xU$'v$iRu#$eU#$6RwU#$x(a$$$t$$$y

r_E$'($Å_E$($I_R\e$(eU$j$Ñ$$r_e$($CÑ$$$r_a$($I_R\E$(aU$rj$$$$$$ty Å_e$'($RI_R\e$(eU$jU$F$CÑ$$$$Å_a$'($RI_R\e$(eU$jU$F$C$F$I_R\E$(aU$ rjB

•  In short: Conjunctions of unrelated literals result to computing the Cartesian ProductB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DG-

6.1 Implementation

–  Å₃'($Å_3vE$P_T$$r_3$ iff X₃ is positive and shares variables with previous body literals

•  T$forces the columns representing the shared variables to be equal B

•  CRu#$wU$'v$iRa#$uU#$6RwU$#$ukw$t$$$y

r_E$'($Å_E$($I_R\E(aUj$Ñ$r_e($CÑ$r_a$($X+Ñ$t$y Å_e$'($I_R\E(aUj$F$CÑ$$y

Å_a$'($RI_R\E(aUj$F$CU$PR^:7Y@_`\e$($^63[2@_`\E$$^:7Y@_`\a(^63[2@_`\e$U$X+Ñ$B

•  In short: Conjunctions of related positive literals result in generating a join, using the related variables as join

conditionB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DH-

6.1 Implementation

–  Å₃'($Å_3vE$z$RÅ_3vE$V_T$$r_3$U$iff X₃ is negative and shares variables with previous body literals.

•  T$forces the columns representing the shared variables to be equal B

•  CRuU$'v$iRuU#${6RuU$$$t$$$y r_E$'($Å_E$($j$#$$$$r_e$($C$$$$ty Å_e$'($j$z$Rj$VRj`\E$($C`\EU$C$UB

•  In short: Conjunctions of related negative literals result to generating a set-minus, removing those tuples which are related to the negative literalB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DI-

6.1 Implementation

•  Now,$we still have the infinite hypothetical relations p'($.X+#$q+#$X+r#$q+r#$rj#$srj/ in our expressions

– Each join r$P_T$p₃$$or Cartesian product$$r$F$p₃ for any “normal” expression r and p₃Öp$is replaced by a suitable expression of the form KRI$RrUU, e.g.B

• rPr`\E(X+`\Er`\e(X+`\e$X+$$$t$$I_\Ek\e$RrU_B

– This expression was created by, e.g.: rRu#$w#$h`U#$ukw

• rP_r`\E(rj`\Erj$$$t$$K5@@63|?@7>ÜYRrU#$rj`\ERrU_B

– This expression was created by, e.g.: rRu#$h`U#$$u(w

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DJ-

6.1 Implementation

• rFRI_\e(4$rjU$$$t$$K5@@63|?@7>ÜYRrU#$4$RrU_B

– This expression was created by, e.g.: rRh`U#$$u(4B

•  Examples:

– CRu#$wU$'v$iRa#$uU#$6RwU$#$ukw$tB

– Å$'($RI_R\E(aUj$F$CUPR^:7Y@_`\e(^63[2@_`\E^:7Y@_`\a(^63[2@_`\e$U$

X+B

– Å$($I_\ek\a$RI_R\E(aUj$F$CUB

– By algebraic optimization, this will later result to

• Å$($RI_R\E(aUjU$P_\ek\a$CB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- DK-

6.1 Implementation

•  Finally, the whole rule G$S$C$'v$X_E#$h#$X₉ is now

transformed to the expression 7Ç5:RGU$'($K_275<RCU RÅU$$B

– i.e. to evaluate the rule G, we project all variables appearing in its head from its body expression ÅB

•  For evaluating one iteration step for given intensional predicate i₃, all related

rules have to be united

– 7Ç5:Ri₃U$'($MG$Ö$<7YRi3U$RGU_B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EM-

6.1 Implementation

•  Now, the elementary production rule +_! corresponds to evaluating all 7Ç5:Ri₃U_B

•  Queries á$S$}R@$S$}R@_EE#$h#$@#$h#$@₉₉U$can be transformed to U$can be transformed to relational algebra likewise

•  Also note that D5@5:;[ can be translated to

C7:c:[^d while D5@5:;[^97[ has to be translated to full C7:c:[B

– Negation requires the highly inefficient setminus operator B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EL-

6.1 Implementation

•  For actually performing the fixpoint iteration, the following is performed

1.  Create tables for each intensional predicate i₃

2.  Execute the elementary production +_! (i.e. run (i.e. run 7Ç5:Ri₃U for each intensional predicate) and store results temporarily

a.  If result tables are of the same size as the predicate tables, the fixpoint has been reached and we can continue with step 3 b.  Replace content of intensional predicate tables with

respective temporary tables c.  Continue with step 2

3.  Run the actual query on the tables to obtain final result

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- ED-

6.1 Implementation

•  Example

– 7<[7RE#$eU`$7<[7RE#$aU`$7<[7Re#$ÄU`$y 7<[7Ra#$ÄU`$7<[7RÄ#$gU`B

– }5@2Ru#$wU$'v$7<[7Ru#$wU`$$B

– }5@2Ru#$wU$'v$7<[7Ru#$xU#$}5@2$Rx#$wU`B – }5@2Re#$uUàB

– The facts all go into the extensional table r<[7#$y an intensional table H5@2 is createdB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EE-

6.1 Implementation

#1 #2

1 2

1 3

2 4

3 4

4 5

edge

#1 #2

path

– }5@2Ru#$wU$'v$7<[7Ru#$wU`$$B

• Å$'($K_\E#$\e$I_@6?7$r<[7$y

$$$$($r<[7B

– }5@2Ru#$wU$'v$7<[7Ru#$xU#$}5@2$Rx#$wU`B

• Å$'($K_\E#$\e$RI_@6?7$r<[7$P^:7Y@_`\e(^63[2@_`\E$I_@6?7$H5@2U$$y

$$$$($r<[7$P^:7Y@_`\e(^63[2@_`\EH5@2B

– }5@2Re#$uUà$($}5@2Rw#$uU#$w(eB

• Å$'($I_\E(e$H5@2B

– 7Ç5:R}5@2U$'($r<[7$M$$r<[7$P^:7Y@_`\e(^63[2@_`\EH5@2B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EF-

6.1 Implementation

•  Execute elementary production on current tables

– 7Ç5:R}5@2U$'($r<[7$M$$r<[7$P^:7Y@_`\e(^63[2@_`\EH5@2B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EG-

6.1 Implementation

#1 #2

1 2

1 3

2 4

3 4

4 5

edge

#1 #2

path

#1 #2

1 2

1 3

2 4

3 4

4 5

temp_path

•  Replace path table and repeat

– 7Ç5:R}5@2U$'($r<[7$M$$r<[7$P^:7Y@_`\e(^63[2@_`\EH5@2B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EH-

6.1 Implementation

#1 #2

1 2

1 3

2 4

3 4

4 5

edge

path

#1 #2

1 2

1 3

2 4

3 4

4 5

1 4

2 5

3 5

temp_path

#1 #2

1 2

1 3

2 4

3 4

4 5

•  Replace path table and repeat

– 7Ç5:R}5@2U$'($r<[7$M$$r<[7$P^:7Y@_`\e(^63[2@_`\EH5@2B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EI-

6.1 Implementation

#1 #2

1 2

1 3

2 4

3 4

4 5

edge

path

#1 #2

1 2

1 3

2 4

3 4

4 5

1 4

2 5

3 5

temp_path ^{#1 #2}

1 2

1 3

2 4

3 4

4 5

1 4

2 5

3 5

1 5

•  Replace path table and repeat

– No change – fixpoint is reachedB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EJ-

6.1 Implementation

#1 #2

1 2

1 3

2 4

3 4

4 5

edge

path

temp_path ^{#1 #2}

1 2

1 3

2 4

3 4

4 5

1 4

2 5

3 5

1 5

#1 #2

1 2

1 3

2 4

3 4

4 5

1 4

2 5

3 5

1 5

•  Run query to obtain final resultB

– I_\E(e$H5@2

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- EK-

6.1 Implementation

#1 #2

1 2

1 3

2 4

3 4

4 5

edge

path

#1 #2

1 2

1 3

2 4

3 4

4 5

1 4

2 5

3 5

#1 #2

2 4

2 5

result

•  Given an extensional database and a query,

there are two general strategies for evaluation

– Bottom-Up: Start with given facts in the EDB and generate all new facts. Then discard those which don’t match the query

• e.g. fixpoint iteration

• Performs well in restricted and smaller scenarios

• “forward-chaining”

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FM-

6.2 Strategies

– Top-Down: Start with query and generate proofs down to the EDB facts

• Most logical programming environments choose this approach

– e.g. SDL resolution

• Performs well in more complex scenarios where bottom-up becomes prohibitive

• “backward-chaining”

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FL-

6.2 Strategies

•  Scenario

– All facts are contained in extensional database rDâB – All rules are contained in the Datalog program !

•  No facts in !

– Given is a goal query á$S$}R@_E#$h#$@₉U$àB

•  Bottom-up problems

– Generate all deducible facts of ! 䏗$rDâB

– When finished, throw away all facts not matching the query pattern. Especially:

•  All those facts whose predicate is not }B

•  All those facts whose predicate is }, but are more general than the query

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FD-

6.2 Strategies

– Example with constants:

• á$S$}R5#$u#$|U$à

• Why should we generate all facts of } and later discard those which are not subsumed by á?

– In the next lecture, we will explore bottom-up

approaches which avoid generating unnecessary facts

• Magic Sets

• Counting techniques

•  Today, we start with a simple top-down approach

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FE-

6.2 Strategies

•  Basic Idea:

– Start with the query á$S$}R@$S$}R@_EE#$h#$@#$h#$@₉₉U$à U$à

– Iteratively generate all proof trees ending with a ground instance of á and starting with known facts

• Iterate over tree depth

• As a helper data structure create all possible search trees of current depth

• Transform search trees to all possible proof trees

• Stop if no additional search trees / proof trees can be constructed

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FF-

6.2 Top-Down Evaluation

– A search tree is a generic proof tree which is still parameterized to some extent

• Proof trees can be generated from search trees

• Leaf nodes are called subgoal nodes

• Root node is called goal node

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FG-

6.2 Top-Down Evaluation

•  Example:

– 7RE#$eU`$7RE#$aU`$7Re#$ÄU`$7Ra#$ÄU`$7RÄ#$gU`$7Rg#$äU`$7 Rg#ãUB

– }Ru#$wU$'v$7Ru#$wU`$$B B Rule 1B – }Ru#$wU$'v$7Ru#$xU#$}Rx#$wU`$$B Rule 2 – á$S$}Re#$uUB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FH-

6.2 Top-Down Evaluation

L- D-

E- F- G-

I- H-

7-

7- 7-

7-

7- 7- 7-

•  Proof Trees of depth 0

– Which facts are ground instances of á ?

– In our example, this is not the case for any fact…

•  Search Trees of depth 1

– Find all rules C$S$â$'v$c_E#h#c_~$$such that á and â are unifiable

•  Unifiable: There are substitutions such that â matches á

– For each rule C, construct a search tree with á as root

•  Attach a rule node to á containing CB

•  Attach k subgoal nodes representing c_E#h#c_~$in its unified form

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FI-

6.1 Top-Down Evaluation

•  Search Trees of depth 1

– Rule 1: }Ru#$wU$'v$7Ru#$wU`B

– Rule 2: }Ru#$wU$'v$7Ru#$xU#$}Rx#$wU`$B

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FJ-

6.2 Top-Down Evaluation

L- D-

E- F- G-

I- H-

7-

7- 7-

7-

7- 7- 7-

á$S$}Re#$uU--

}Rw#$uU$'v$7Rw#$uU`- 7Re#$uU--

á$S$}Re#$uU--

}Rw#$uU$'v$7Rw#$xU#$}Rx#$uU`- 7Re#$xU-- }Rx#$uU-- +EB

+eB

(#+%-

,37(#+%- C3%&-

•  To generate proof trees from a given search tree, we have to find a substitution å such that for each goal node with clause G, åRGU$Ö$!$M$rDâ$$$B

– By applying this substitution to the whole tree, we obtain a proof tree

– The root node is a result of the query

•  Example:

– Find a substitution for +E (+e does not have one)

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- FK-

6.2 Top-Down Evaluation

á$S$}Re#$uU--

}Rw#$uU$'v$7Rw#$uU`- 7Re#$uU--

å$'($.u$($Ä/-

}Re#$ÄU--

}Rw#$uU$'v$7Rw#$uU`- 7Re#$ÄU-- +E- HEB

•  For any 9lE, all existing search trees of depth n-1 are expanded by treating any subgoal node as a

goal node

– Thus, new rule nodes and subgoals are appended

•  Example: Expanding +e to +e#e$$and$+e#EB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GM-

6.2 Top-Down Evaluation

á$S$}Re#$uU--

}Rw#$uU$'v$7Rw#$xU#$}Rx#$uU`- 7Re#$xU-- }Rx#$uU-- +e#eB

7Rx#$bU--

}Rx#$uU$'v$7Rx#$bU#$}Rb#$uU`- }Rb#$uU--

•  +e#E and some substitutions å$

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GL-

6.2 Top-Down Evaluation

á$S$}Re#$uU--

}Rw#$uU$'v$7Rw#$xU#$}Rx#$uU`- 7Re#$xU-- }Rx#$uU-- +e#EB

}Rx#$uU$'v$7Rx#$uU`- 7Rx#$uU-- å$'($.x$($Ä$u$($g/-

L- D-

E- F- G-

I- H-

7-

7- 7-

7-

7- 7- 7-

}Re#$gU--

}Rw#$uU$'v$7Rw#$xU#$}Rx#$uU`- 7Re#$ÄU-- He#EB

}RÄ#$gU--

}Rx#$uU$'v$7Rx#$uU`- 7RÄ#$gU--

•  +e#e#E and substitutions å_E and å_e

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GD-

6.2 Top-Down Evaluation

á$S$}Re#$uU--

}Rw#$uU$'v$7Rw#$xU#$}Rx#$uU`- 7Re#$xU-- }Rx#$uU-- +e#e#EB

7Rx#$bU--

}Rx#$uU$'v$7Rx#$bU#$}Rb#$uU`- }Rb#$uU--

}Rb#$uU$'v$7Rb#$uU`- 7Rb#$uU-- å_E$'($.x$($Ä#$b(g#$u$($ä/-

L- D-

E- F- G-

I- H-

7-

7- 7-

7-

7- 7- 7-

å_e$'($.x$($Ä#$b(g#$u$($ã/-

}Re#$äU-- }Re#$ãU--

S- S-

He#e#EREUB

He#e#EReUB

•  Please note:

– By applying this type of backward-chaining, not all possible proof trees for the query can be generated – Only proof trees having ground facts in all leaf

nodes are possible

• Those trees are called full proof trees

• However, for each proof tree matching the query, there is also a respective full proof tree

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GE-

6.2 Top-Down Evaluation

•  We can see that the backward chaining proof trees can reach arbitrary depth

– The backward chaining method is sound and complete – But consider the iterated use of rule 2

– The tree is of infinite depth

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GF-

6.2 Top-Down Evaluation

á$S$}Re#$uU--

}Rw#$uU$'v$7Rw#$xU#$}Rx#$uU`- 7Re#$xU-- }Rx#$uU--

7Rx#$bU--

}Rx#$uU$'v$7Rx#$bU#$}Rb#$uU`- }Rb#$uU--

7Rb#$çU--

}Rb#$uU$'v$7Rb#$çU#$}Rç#$uU`- }Rç#$uU--

/#

•  When do we stop building trees?

– A-priory, we have no idea which recursion depth we will need

• ?path (a,X)

• Obviously, the more nodes we have, the deeper the recursion depth will be

– Still the number of sensible combinations of rDâ facts and predicates in !$is limited since

• Both the database and the datalog program are finite

• We can only substitute any constant symbol from some fact in any predicate symbol at any position of a variable

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GG-

6.2 Top-Down Evaluation

+- 7- 4- '- &- /#

•  Theorem: Backwards chaining remains complete, if the search depth is limited to

\}67<345@7>$é$\4;9>@59@>85èR56[>U$B

– #predicates is the number of predicate symbols used – #constants is the number of constant symbols used – max(args) is the maximum number of arguments, i.e.

the arity, of all predicate symbols

– With this theorem, we can stop the backward chaining process after the last sensible production

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GH-

6.2 Top-Down Evaluation

•  Proof sketch:

– \}67<345@7>$é$\4;9>@59@>85èR56[>U is an upper limit for the number of distinct ground facts derived from

! and rDâ$(purely syntactical)B and rDâ$(purely syntactical)B

– We can limit the production process to full proof trees, where at least one new fact is added in each depth level (otherwise the new level is useless…)

– Since we only have a limited number of ground facts, also the number of levels has to be limited…

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GI-

6.2 Top-Down Evaluation

•  Consider an example: a finite number of facts .}5@2R5#|U#$}5@2R|#4U#h#$}5@2R8#9U/$and a rule }5@2Ru#wU$'v$}5@2Ru#xU#$}5@2Rx#wU`B

– Worst case

• Longest possible deduction chain is path(a,n) of length n-1

– The least determined query is ?path(X,Y), i.e. all paths

• There are 9 constant symbols and a single predicate symbol

• The constants can occur in two places, i.e. 85èR56[>U$($e$B

• That means the maximum number of deducible facts is 9^eB

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GJ-

6.2 Top-Down Evaluation

+- 7- 4- '- &- /# "-

•  Many backward-chaining algorithms rely on the concepts of search trees and proof trees

•  However, the generation strategy may differ

– In the previous example, the search trees have been generated one by one according to their depth

• depth 0, depth 1, depth 2, …

• This is called level saturation strategy and resembles an breadth-first approach

– Alternatively, depth-first approaches are possible

• Rule saturation strategy

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- GK-

6.2 Top-Down Evaluation

•  The previously presented top-down algorithm is extremely naïve

– It generates all possible search and proof trees up to the worst-case depth which are somehow

related to the query

• Performance is far from optimal

– In case of less restricted scenarios (e.g. not only Horn clauses or infinite universes), this approach is

inevitably doomed to failure

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- HM-

6.2 Resolution

•  From the field of “real” logics, we can borrow the concept of resolution

– A technique for refutation theorem proofing

• “Reductio ad absurdum”

– Mainly explored around 1965 by J.A. Robinson – Established itself as THE standard

technique for logical

symbolic computation

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- HL-

6.2 Resolution

•  There are several variants of resolution

– Best known in the field of logical programming is the class of SDL resolution algorithms

• “Linear Resolution with Selection Function for Definite Clauses”

• Most popular among these are the general algorithms employed in languages like Prolog or Lisp

• However, in the next lecture we shall study a simplified SDL resolution algorithm suitable for Datalog

– Be curious – that will be fun!

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- HD-

6.2 Resolution

•  The research and developments in the area of deductive databases successfully provided the ability to perform recursive queries

– And with these, some limited reasoning capabilities

•  However, most applications have been tailored to work with traditional SQL based databases

– When using SQL2 (SQL-92), recursive queries cannot be facilitated without external control and huge

performance penalties

– SQL2 is still the default for most today’s databases

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- HE-

6.3 Recursive SQL

•  SQL3 (SQL-99) is a later SQL standard which mainly aims at widening the scope of SQL

– Contains many features which extend beyond the scope of traditional RDBs

•  Binary Large Objects

•  Limited support for soft constraints

•  Updatable views

•  Active databases

•  Object orientation

•  UDF / UDT / UDM

•  References

•  Recursive Temporary Tables

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- HF-

6.3 Recursive SQL

•  Recursive temporary tables adopt many concepts of deductive databases into the SQL world

– Most vendors developed proprietary implementations of recursive tables

• Nobody cared for the standard…

• Syntax may thus differ

– In DB2 known as Common Table Expressions

!"#$%&'(&)*+,&'-./,0&1,-+"'-2&'3456&-2+0+7+,&,-8--9#%:);<%#-*+%=&-8->?<%<@@-9<%%&-8-A:A.-8-;B-*C+3",4?$&<(- HG-

6.3 Recursive SQL