11. XML storage –details

(1)

XML Databases

11. XML storage – details

Silke Eckstein Andreas Kupfer

Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de

11.1 Introduction

11.2 Node-based encoding

11.3 Path-based XPath Accelerator encoding 11.4 Evaluation in SQL

11.5 Skeleton compression 11.6 Enhancing tree awareness 11.7 Overview and References

2

11. XML storage – details

XML Databases – Silke Eckstein – Institut für Informationssysteme – TU Braunschweig

• Different methods for storage of XML documents –Text-based

•Storing whole XML documentsas string

•Can use full text index or path index –Model-based

•Genericmapping of thetreestructure –Schema-based

•Detect and analyse the structure of the XML documents

•Derive a DB schema from the structure –Hybrid approaches

•A combination of some of those methods

–No algorithm has the optimal solution for all kind of XML documents

–Reasonable solution is heavily dependent on the application

XML Databases – Silke Eckstein – Institut für Informationssysteme – TU Braunschweig 3

11.1 Last Lecture

• If we want to run queries (XQuery) on stored XML documents…

–How do weget backthe documents efficiently?

•Depends on the storage method used

•Shall work withallXML documents – as XQuery does?

–Then we have to use a model-basedapproach!

• Last week we have seen thatefficientqueries arenot for freewith model-based storage

–Will efficient queries be possible?

–Now we will see…

11.1 Introduction

• Exploiting DB technology –Our main objective:

Use as much of existing DB technology as possible.

•XQuery operations on trees, XPath traversals and node construction in particular, should be mapped into operations over the encoded database:

•Obviously, encoding εneeds to be chosen judiciously.

•In particular, a faithful back-mappingε-1is absolutely required.

XML Databases – Silke Eckstein – Institut für Informationssysteme – TU Braunschweig 5 [Gru08]

11.1 Introduction

Our goal: let the database do the work!

• Native XML processorsneed external memory representations of XML documents, too!

–Main-memory representations, such as a DOM tree, are insufficient, since they are only suited for "toy" examples (even with today's huge main memories, you want persistentstorage).

–Obviously, native XML databases have more choices than those offered on top of a relational DBMS.

–We will have to see whether this additional freedom buys us significant performance gains, and

–what price is incurred for "replicating" RDBS functionality.

11.1 Introduction

(2)

• Relational XML processors

–Remember our goal: let the database do the work!

–We aim at a truly (or purely) relational approach here:

•Re-use existing relational database infrastructure – table storage layer and indexes (e.g. B-trees), SQL or algebraic query engine and optimizer – and invade the database kernel in a very limited fashion (or, ideally, not at all).

11.1 Introduction

Database-supported XML processors

Using relational database technology as a highly efficient, scalable processor for XMLlanguages like XPath, XQuery, and XML Schema.

• What makes a good (relational XML-) tree encoding?

–Hard requirements:

•ε is required to reflect document order and node identity.

–Otherwise: cannot enforce XPath semantics, cannot support

<< and is, cannot support node construction.

•ε is required to encode the XQuery DM node properties.

–Otherwise: cannot support XPath axes, cannot support XPath node tests, cannot support atomization, cannot support validation.

•ε is able to encode any well-formed schema-lessXML fragment (i.e., εis "schema-oblivious", see below).

–Otherwise: cannot process non-validated XML documents, cannot support arbitrary node construction.

11.1 Introduction

• What makes a good (relational XML-) tree encoding?

–Soft requirements

(primarily motivated by performance concerns):

•Data-bound operationson trees (potentially delivering/copying lots of nodes) should map into efficient database operations.

–XPath location steps (12 axes)

•Principal, recurring operations imposed by the XQuery semantics should map into efficient database operations.

–Subtree traversal (atomization, element construction, serialization).

–For a relational encoding, "database operations" always mean "table operations" . . .

11.1 Introduction

10

11. XML storage – details

11.2 Node-based encoding

• Several encoding schemes are based on an (appropriate) mapping of XML nodes onto relational tuples.

• Key questions are:

–How to represent node IDs, and

–how to represent XML-structure, in particular, document order.

• Obviously, both questions are related, and - since we deal we treestructures - we might as well think of an edge-based representation scheme (in a tree, each non-root node has exactly one incoming edge!)

• Most representations encode document orderinto node IDs by chosing an appropriately ordered ID domain.

• Node IDs

–Two very common approaches can be distinguished:

•XML nodes are numbered sequentially(in document order).

•XML nodes are numbered hierarchically(reflecting tree structure).

–Observations:

•Node ID numbers are assigned automatically by the encoding scheme.

•Sequential numbering necessarily requires additional encoding means for capturing the tree structure.

•Both schemes represent document order by a (suitable) numeric order on the node ID numbers.

•Both schemes envisage problems when the document structure dynamically changes (due to updates to the document), since node ID numbers and document structure/order are related! (see later)

11.2 Node-based encoding

(3)

• Working with node-based encodings –Obviously, relational representations based on node-

based encoding (traditionally called "edge table encodings") provide support for

•(bi-directional) parent-child traversal,

•name tests,

•and value-based predicates

•using the following kind of table:

11.2 Node-based encoding

nodeID parentID elemname value

· · · ·

edgetable

• ... Working with node-based encodings

–This table wastes space due to repetition of element names.

–Furthermore, to support certain kinds of path expressions, it may be beneficial to:

•store paths instead of element names, so as to –support path queries, while

–introduce even more storage redundancy;

•thus use a separate "path table" to store the paths together with path IDs.

11.2 Node-based encoding

nodeID parentID elemname value

· · · ·

edgetable

• Path table representation

–Element names (or rather paths) can now be represented via path IDs in the edge table, pointing (as foreign keys) to the separate path table:

»

•Notice that the path table entries represent paths of the form /bib/doc/author/name, i.e., they record paths that end in element names, not values. Hence, they are type- and not instance-specific: all document nodes that have identical root-to-element paths are represented by a singleentry in the path table!

11.2 Node-based encoding

nodeID parentID pathID value

· · · ·

pathID path

· ·

edgetable

pathtable

16

11. XML storage – details

• Tree partitions and XPath axes

–Given an arbitrary context node o, the XPath axes descendant, ancestor, preceding, followingcoverand partition the tree containing o .

11.3 XPath Accelerator

• Tree partitions and XPath axes –Context node (here: f) is arbitrary

–NB: Here we assume that no node is an attribute node.

Attributes treated separately (recall the XPath semantics).

(4)

XML Databases – Silke Eckstein – Institut für Informationssysteme – TU Braunschweig 19 [Gru08, Gru02]

11.3 XPath Accelerator

• TheXPath Accelerator tree encoding

–... a relational tree encoding based on this partitioning:

•If we can exploit the partitioning property, the encoding will represent each tree node exactly once.

•In a sense, the semantics of the XPath axes descendant, ancestor, preceding, and followingwill be "built into" the encoding ⇒⇒⇒⇒"XPath awareness".

•XPath accelerator is schema-obliviousand node-based:

each node maps into a row in the relational encoding.

• Pre-order and post-order traversal ranks

11.3 XPath Accelerator

Pre-order/post-order traversal

(During a single scan through the document:) To each node v , assign its pre-orderand post-ordertraversal ranks 〈pre(v ); post(v )〉.

• Pre-order/post-order: Tree isomorphism

11.3 XPath Accelerator

pre(v) encodes document order and node identity

v1<< v2 ⇔pre(v1) < pre(v2)  v1isv2 ⇔pre(v1) = pre(v2)

• XPath axes in the pre/post plane –Plane partitions ≡XPath axes, ois arbitrary!

–Pre/post plane regions ≡major XPath axes

11.3 XPath Accelerator

The major XPath axes descendant, ancestor, following, precedingcorrespond to rectangular pre/post plane windows.

• XPath Accelerator encoding –XML fragment fand its skeleton tree

–Pre/postencoding of f: table accel

11.3 XPath Accelerator

24

11. XML storage – details

(5)

• Relational evaluation of XPath location steps –Evaluate an XPath location step by means of a

window query on the pre/postplane.

•Table accelencodes an XML fragment,

•table contextencodes thecontext node sequence(in XPath accelerator encoding).

11.4 Evaluation in SQL

XPath location step (axis αααα) SQL window query SELECT DISTINCT v'.*

FROM contextv, accelv' WHERE v' INSIDE window(α,v) ORDER BY v'.pre

• XPath axes and pre/postplane windows –Window def's for axis α, name test t( * = don't care)

11.4 Evaluation in SQL

• Pre/postplane window ⇒⇒⇒⇒SQL predicate –descendant::foo, context nodev

–ancestor-or-self::*, context nodev

11.4 Evaluation in SQL

v'INSIDE 〈(v.pre,*), (*, v.post),*, elem, foo〉

≡

v'.pre > v.preANDv'.post < v.post AND v'.kind = elemANDv'.tag = foo

v'INSIDE 〈(*, v.pre], [v.post,*),*, elem, *〉

≡

v'.pre <= v.pre AND v'.post >= v.post AND v'.kind = elem

• (e,f)/descendant::node() –Context & frag. encodings

–SQL query with expanded window() predicate

11.4 Evaluation in SQL

SELECT DISTINCT v1.*

FROM context v, accel v1

WHERE v1.pre > v.pre AND v1.post < v.post ORDER BY v1.pre

• Compiling XPath into SQL

–path: an XPath to SQL compilation scheme (sketch)

11.4 Evaluation in SQL

path(fn:root( )) =

SELECT v'.*

FROM accel v' WHERE v'.pre = 0

path(c/α) =

SELECT DISTINCTv'.*

FROM path(c) v , accel v' WHERE v' INSIDEwindow(α, v ) ORDER BY v'.pre

path(c [ α]) =

SELECT DISTINCTv.*

FROM path(c) v , accel v' WHERE v' INSIDEwindow(α, v ) ORDER BY v.pre

• An example: Compiling XPath into SQL

–Compilefn:root()/descendant::a/child::text()

11.4 Evaluation in SQL

path(fn:root()/descendant::a/child::text())

= SELECT DISTINCTv1.*

FROM path(fn:root/descendant::a)v, accelv1

WHEREv1INSIDEwindow(child::text(),v) ORDER BY v1.pre

= SELECT DISTINCTv1.*

SELECT DISTINCTv2.*

FROM FROMpath(fn:root) v, accel v2

WHERE v2INSIDEwindow(descendant::a,v) ORDER BY v1.pre

accelv1

WHEREv1INSIDE window( child::text(),v) ORDER BY v1.pre

( )

^v,

(6)

• Does this lead to efficient SQL? Yes!

–Compilation scheme path(·) yields an SQL query of nesting depth nfor an XPath location path of nsteps.

•On each nesting level, apply ORDER BYand DISTINCT.

–Observations:

•All but the outermost ORDER BY and DISTINCT clauses may be safely removed.

•The nested SELECT-FROM-WHEREblocks may be unnestedwithout any effect on the query semantics.

11.4 Evaluation in SQL

• Result of path(·) simplified and unnested

–path(fn:root()/descendant::a/child::text())

–An XPath location path of n steps leads to an n-fold self join of encoding table accel.

–The join conditions are

•conjunctions √ of

•rangeorequality predicates √ .

11.4 Evaluation in SQL

SELECT DISTINCT v₁.*

FROM accelv₃,accelv₂,accelv₁ WHERE v₁INSIDEwindow(child::text(),v₂)

AND v2 INSIDEwindow(descendant::a,v3) AND v₃.pre = 0

ORDER BY v1 .pre

}}

multi-dimensional window!

• Path-based encodings –Some observations:

•In many cases, the volume of largeXML documents mainly comes from their text contents (PCDATA); their markup/structure is of moderate size.

•In contrast, most queriestend to focus on structural aspects (XPath navigation, tag name tests, . . . ), with only occasional access to character contents.

•Many document collections – even though of only semi- structuredobjects – share large fractions of structure across individual documents/fragments.

11.4 Evaluation in SQL

–Possible conclusions: try to . . .

•represent structure separatefrom contents,

•keep structural representation in (main) memory,

•identify commonstructure (and possibly contents as well), and store only once

11.4 Evaluation in SQL

35

11. XML storage – details

• Data guides/skeletons

–Separate structure from contents . . .

•Chose representations for XML structure (non-leaf nodes) and text contentsindependently.

•Store the two representations separate from each other, such that structural info ("skeleton" or "data guide")

–can be kept small (and thus, in main memory),

–supports major XQuery functionality (esp., XPath navigation) efficiently,

andtext contentsdata

–can be accessed only on demand,

–directed by structure (hence the term "data guide").

–Often, main memory-oriented data structures are used for the skeleton, while external memory data structures hold text contents.

11.5 Skeleton compression

(7)

• Skeleton extraction

–Conceptually, a skeletonof an XML document can be obtained by

•replacing all text content (leaf) nodes of an XML tree with a special "marker" (e.g., a hash mark "#"), indicating that some textual content has been removed.

•The resulting XML tree is a faithful representation of the structureof the original document, while all actual content has to be stored elsewhere.

11.5 Skeleton compression

• ... Skeleton extraction

–Since the skeleton is small (compared to the whole document), it may even be feasible to represent it as a DOM tree in main memory.

•If we assign (global) node IDsto text contents nodes (as usual), those IDs can be used to access text contents from the skeleton.

•If text contents is stored separately in document order, we may not even need the IDs, since a joint traversal of the skeleton and the list of text contents nodes can bring them together.

11.5 Skeleton compression

• Skeleton compression –Notice the following:

1. the more regular the structure of the XML document (collection), the more identicalsubtrees the skeleton will have,

2. it conserves (memory) space, if we foldidentical, adjacent subtrees in the skeleton,

3. an even more compact representation can be obtained, if we share common subtrees, resulting in a skeleton DAG.

11.5 Skeleton compression

• Example

–Replace text contents by special marker "#" to obtain skeleton.

–Fold identical, adjacent subtrees to obtain first version of a compressed skeleton.

–Share common subtrees obtaining compressed skeleton DAG.

11.5 Skeleton compression

• Discussion (1) – Pros:

•Skeleton extraction/compression follows the (database) idea of separatingtype and instance information.

•(Compressed) skeletons are typically small enough to fit into main memory, while only the (mass) instance data needs to be paged in from secondary storage.

•Experiments reported in the literature prove large performance gains compared to both

–completely disk-based storage schemes (because of skeleton being kept in main memory), and –completely memory-based schemes (because of

capability to handle much larger document collections).

11.5 Skeleton compression

• Discussion (2) –Cons:

•Skeletons do not compress too well in some cases (semi- structured data).

•Compressed skeletons exhibit very clumsy structure (typically implemented in some kind of spaghetti, main memory-only data structure).

•Consequently, if skeleton does not fit into memory, usefulness is unclear.

–Possible ways out . . .

•Improve compression scheme.

•Chose skeleton representation also suitable for secondary storage.

•Combine basic ideas with other representation schemes.

11.5 Skeleton compression

(8)

43

11. XML storage – details

• Enhancing tree awareness

–We now know that the XPath Accelerator is a true isomorphismwith respect to the XML skeleton tree structure.

•Witnessed by our discussion of shredder (ε) and serializer (ε^-1) . –We will now see how the database kernel can benefit

from a more elaborate tree awareness (beyond document order and semantics of the four major XPath axes).

–This will lead to the design of staircase join ⋈⋈⋈⋈sj, the core of MonetDB/XQuery's XPath engine.

11. 6 Enhancing tree awareness

• Tree awareness?

–Document order and XPath semantics aside, what are further tree properties of value to a relational XML processor?

The size of the subtreerooted in node a is 4 The leaf-to-root pathsof nodes b, c meetin node d The subtrees rooted in e and a are necessarily disjoint

[Gru08]

• Tree awareness : Subtree size

–Tree property subtree size( on previous slide) is implicitly present in a pre/post-based tree encoding:

–To exploit property subtree size, we were able to find a means on the SQL language level, i.e., outside the database kernel.

⇒This led to window shrink-wrapping for the XPath descendant axis.

post(v) - pre(v) = size(v ) - level(v)

[Gru08, Gru02]

• Subtree size – example

|(v)/descendant| = post(v) – pre(v) + level(v)

Q ≡ (c)/following::node()/descendant::node()

• Tree awareness on the SQL level –Shrink-wrapping for the descendantaxis

–path(Q)

Q ≡ (c)/following::node()/descendant::node()

[Gru08, Gru02]

SELECT DISTINCT v2.pre FROM context c, accelv1, accelv2

WHERE v₁.pre> c.pre AND v1.pre< v2.pre AND v₁.post>c.post AND v₁.post > v2.post

AND v₂.pre<= v₁.post + h ANDv₂.post>= v1.pre– h ORDER BY v₂.pre

(9)

• Tree awareness : Meeting ancestorpaths –Evaluation of axis ancestor can clearly benefit from

knowledge about the exact element node where several given node-to-root paths meet.

•For example:

For context nodes c₁…..c_n, determine their lowest common ancestorv = lca(c₁…..c_n).

⇒Above v, produce result nodes once only.

(This still produces duplicate nodes below v.) –This knowledge ispresent in the encoding but is not as

easily expressed on the level of commonly available relational query languages(such as SQL or relational algebra).

[Gru08]

• Tree awareness : Disjoint subtrees –An XPath location step cs/αis evaluated for a context

node sequencecs.

•This " set-at-a-time" processing mode is key to the efficient evaluation of queries against bulk data. We want to map this into set-oriented operationson the RDBMS.

(Remember: location step is translated into joinbetween context node sequence and document encoding tableaccel.)

–But: If two context nodes c_{i ,j}∈csare in α-relationship, duplicates and out-of-orderresults may occur.

•Need efficient way to identify the c_i∈∈∈∈cs which are notin α- relationship with any other c_j

(forα= descendant: "ci ,jin disjoint subtrees?").

• Tree awareness: Window overlap, coverage –Location step (c₁, c₂, c₃, c₄)/descendant::node().

The pairs (c1, c2) and (c3, c4) are in descendant- relationship:

•Window overlap and coverage (descendantaxis)

• Staircase Join: An injection of tree awareness –Since we fail to explain tree properties and at the

relational language level interface, we opt to invade the database kernel in a controlled fashion

•Inject a new relational operator, staircase join ⋈⋈⋈⋈sj, , , , into the relational query engine.

•Query translation and optimization in the presence of ⋈⋈⋈⋈_sj continues to work like before (e.g., selection pushdown).

•The ⋈⋈⋈⋈_sjalgorithm encapsulates the necessary tree knowledge.

⋈

⋈⋈

⋈_sjis a local change to the database kernel.

–Remember: All of this is optional. XPath Accelerator is a purely relational XML document encoding, working on top ofanyRDBMS.

Introduction and Basics 1. Introduction

2. XML Basics 3. Schema Definition 4. XML Processing Querying XML 5. XPath & SQL/XML

Queries

6. XQuery Data Model 7. XQuery

XML Updates 8. XML Updates & XSLT

Producing XML 9. Producing XML Storing XML 10. XML storage

11. Relational XML storage 12.Storage Optimization Systems

13. Technology Overview

10.6 Overview

• "Database-Supported XML Processors", [Gru08]

–T. Grust

–Lecture, Uni Tübingen, WS 08/09

• "XML and Databases", [Scholl07]

–M. Scholl

–Lecture, Uni Konstanz, WS07/08

• "Staircase Join: Teach a Relational DBMS to Watch its (Axis) Steps"

[GKT03]

–Torsten Grust, Maurice van Keulen, Jens Teubner.

–In Proc. 29th Int'l Conference on Very Large Databases (VLDB), pages 524- 535, 2003.

–http://www.informatik.uni-konstanz.de/~grust/files/staircase-join.pdf

• " Accelerating XPath Location Steps" [Gru02]

–T. Grust

–ACM SIGMOD 2002, June 4–6, Madison, Wisconsin, USA

–http://www-db.informatik.uni-tuebingen.de/files/publications/xpath-accel.pdf

54

11.7 References

(10)

• Now, or ...

• Room: IZ 232

• Office our: Tuesday, 12:30 – 13:30 Uhr or on appointment

• Email: eckstein@ifis.cs.tu-bs.de

11. XML storage –details

11. XML storage – details

11.1 Last Lecture

11.1 Introduction

11.1 Introduction

11.1 Introduction

11.1 Introduction

11.1 Introduction

11.1 Introduction

11. XML storage – details

11.2 Node-based encoding

11.2 Node-based encoding

11.2 Node-based encoding

11.2 Node-based encoding

11.2 Node-based encoding

11. XML storage – details

11.3 XPath Accelerator

11.3 XPath Accelerator

11.3 XPath Accelerator

11.3 XPath Accelerator

11.3 XPath Accelerator

11.3 XPath Accelerator

11.3 XPath Accelerator

11. XML storage – details

11.4 Evaluation in SQL

11.4 Evaluation in SQL

11.4 Evaluation in SQL

11.4 Evaluation in SQL

11.4 Evaluation in SQL

11.4 Evaluation in SQL

( )

11.4 Evaluation in SQL

11.4 Evaluation in SQL

}}

11.4 Evaluation in SQL

11.4 Evaluation in SQL

11. XML storage – details

11.5 Skeleton compression

11.5 Skeleton compression

11.5 Skeleton compression

11.5 Skeleton compression

11.5 Skeleton compression

11.5 Skeleton compression

11.5 Skeleton compression

11. XML storage – details

10.6 Overview

11.7 References

Questions, Ideas, Comments