Stochastic Approach
3.4 Graphical Methods
Graphical methods have their origin in several scientific areas and they can be con-sidered as a marriage between probability theory and graph theory. They provide a natural tool for dealing with uncertainty and complexity. The basic idea is the no-tion of modularity, so that a complex system can be built by combining simpler parts.
Probability theory provides the glue whereby the parts are combined, ensuring that the system as a whole is consistent and providing ways to interface models and data.
Graph theory provides both an intuitively appealing interface by which humans can model highly-interacting sets of variables as well as a data structure that lends itself naturally to design general-purpose algorithms. Therefore we can define them as a sort of multivariate analysis that uses graphs to represent models.
Probabilistic graphical models are graphs in which the nodes represent random vari-ables and the (lack of) arcs represent conditional independence assumption. Hence they are a compact representation of joint probability distribution.
There are two main kinds of graphical models: undirected and directed (but it is also possible to have a model with both directed and undirected arcs, called chain graph).
Undirected graphical models, also known as Markov networks, are more popular but they are not the most suitable to be applied in the railway system since a timetable is based on a sequence of actions in time and hence of ordered variables. Directed graphical models, also known as Bayesian networks, are then the one that will be con-sidered in this thesis. Note that despite the name, Bayesian networks do not necessary imply a commitment to Bayesian methods; rather they are so called because they use Bayes’ rules for interference.
In a directed graphical model an arc from A to B can be informally interpreted as indi-cating that A "causes" B. Directed cycles are disallowed in the railway systems since they would represent an impossible sequence of precedences, but undirected cycles are still possible.
Example 3.1 (Undirected cycle in the railway system)
We consider two trainst and s. We suppose that sis an Euro City (EC) traveling from Basel to Berlin, and thattis an Inter City Express (ICE) traveling from Münich to Hamburg. Their journeys will lead them to Kassel, where passengers of the traint have the possibility to change to trains. Here, even if trainsarrives aftert, it will leave the station first due to a fixed priority list (e.g. internal decision to give precedence to ICE trains). Between Kassel and Göttingen, both trains have to use the same line, so in Göttingen the passengers of train shave the possibility to change to traintto reach their final destination. The problem is represented in Figure 3.4 both in the form of Public Transport Network and of Activity-on-arc Project Network, which is more intuitive to understand.
C
3.4.1 Graph: notation and terminology
In graphical modeling the dependence pattern between variables is associated with a graph in which vertices encode the random variables and edges encode conditional dependence between the variables. Hence before going deeply into this topic, it is nec-essary to recall some definitions of the Graph Theory (the following notation refers to [45]).
A graphG= (V,E)is defined through a set of verticesVand a set of edgesEthat is a subset ofV×V.
Edges(v,w)∈ E with both(v,w)and(w,v)inEare calledundirected, whereas an
3.4 Graphical Methods 49
Figure 3.1: Example of undirected cycle in the railway system edge(v,w)with its opposite(w,v)not inEis called directed.
The graphs we consider are directed, that is, they contain only directed edges drawn asarrows,v→w, thus we can identify edges withorderedpairs of vertices(v,w).
IfS ⊂ V is a subset of the vertex set, it induces a subgraphGS = (S,ES), where the edge setES = E∩(S ×S)is obtained fromGby keeping all edges with both endpoints inS.
Given two nodes v,w ∈ V, we say that they are adjacent, v ∼ w, if there exists an edge betweenvandw, without distinction betweenv → wandw → v. The set of vertices that are adjacent to a vertex is denoted asne(v). We say thatvis aparentofw if there exists an edgev→w, thenwis called achildofv. The set of parents ofwis denoted aspa(w)and the children ofvis denoted asch(v). If there is a directed path fromvtow, thenvis called anancestorofwandwis called adescendentofv. The set of ancestors ofwis denoted asan(w)and the set of descendents ofvis denoted as de(v).
These definitions can be extended to set of nodes. Given a setS ⊂Vwe define:
• ne(S) = (∪v∈Sne(v))\S;
• pa(S) = (∪v∈Spa(v))\S;
• ch(S) = (∪v∈Sch(v))\S;
• an(S) = (∪v∈San(v))\S;
• de(S) = (∪v∈Sde(v))\S;
Theboundary bd(S)of a subsetS of vertices is the set of vertices inV\S that are adjacent to vertices inS, i.e.bd(S) =ne(S)\S.
Apath(of lengthm) is a sequence of verticesv0,v1. . .vmsuch thatvi ∼ vi+1 ∀i ∈ {0 . . .m−1}.
A path isdirectedifvi→vi+1∀i∈{0 . . .m−1}.
Ifv0 = vmthe (directed) path is called a(directed) cycle. Anm-cycle is a path of lengthmsuch thatv0=vm.
If there is a directed path fromvtowwe say thatvleads towand writev7→ w. If bothv 7→ wandw 7→ vwe say thatvandware connected and writev w. The corresponding equivalence classes[v], where
w∈[v]⇔vw
are the (strong)connectivity componentsofG. Ifv∈S ⊂V, the symbol[v]S denotes the (strong) connectivity component ofvinGS.
A subsetS ⊂V is said to be a(v,w)-separatorif all paths fromvtowintersectS. an-cestral sets are union of connectivity components. The intersection of a collection of ancestral sets is again ancestral. Hence for any subsetS of vertices there is a smallest ancestral set containingS which is denoted byAn(S).
Atreeis a connected, undirected graph without cycles. It has a unique path between two vertices. Aforestis an undirected graph where all connectivity components are trees. The name forest arises from the concept that a forest consists only of (possibly disconnected) trees.
Chain graphsare graphs where the vertex setVcan be partitioned into numbered sub-sets, forming a so-calleddependence chain V=V(1)∪. . .∪V(m)such that all edges between vertices in the same subset are undirected and all edges between vertices in different subsets are directed, pointing from the set with lower number to the one with higher number. Such graphs are characterized by having no directed cycles and the connectivity components form a partitioning of the graph into chain components. A graph is a chain graph if and only if its connectivity components induce undirected subgraphs. An undirected graph is a special case of chain graph. A directed, acyclic graph is a chain graph with all chain components consisting of one vertex.
For a chain graphGwe define itsmoral graph Gmas the undirected graph with the same vertex set but withvandwadjacent inGmif and only eitherv→worw→v or if there are s1, s2 in the same chain component such thatv → s1 andw → s2. If no edges have to be added to form the moral graph, the chain graph is said to be perfect. The name moral graph arises from the connection (marriage) of nodes that have a common child.
In graphical modeling, the focus is on models under which some conditional inde-pendence relations of the formX y Y | (some other variables) holds (see [25]). In particular, we focus on models for which these relations take the formX yY | (the rest), whereas “the rest” means all other variables in the model. For such a model, we can construct an undirected graphG = (V,E)whereV is the set of variables in the model andE consists of edges between variable pairs that are not conditionally independent given the rest. In other words, for all pairs{X,Y}such thatX yY |(the rest), the edge betweenXandYis omitted; for all other pairs, an edge is drawn. Thus if two variables are not adjacent, then they are conditional independent given the rest.
This is known as the pairwise Markov property for undirected graphs.
Graph separationis a very important example of a model for the conditional indepen-dence on undirected graphs. LetS1,S2andS3be subsets of the vertex setVof a finite undirected graphG= (V,E), the global Markov property states
ifS3separatesS1fromS2inGthenS1yS2 |S3.
3.4 Graphical Methods 51 As example we consider four variables W, X, Y and Z for which we know that
W y Z | (X,Y)andY y Z | (W,X). The edges {W,Z}and{Y,Z}must be absent from the graph onV = {W,X,Y,Z}. We obtain the graph shown in Figure 3.2, from which we can infer thatW yZ|XandYyZ|X.
Figure 3.2: Example of the separation concept on a graph
The global Markov property allows a simple translation from a graph-theoretic prop-erty, separation, to a statistical propprop-erty, conditional independence.
That isS1andS2are conditional independent givenS3if every path from (a node in) S1 to (a node in)S2 passes through (at least one node in )S3. An analogous model can be defined also for directed graphs introducing the concept of moral graphs. Given S1,S2andS3subsets of the vertex setVof a finite directed graphG= (V,E), then the direct Markov property is
ifS3separateS1fromS2inGmAn(S1∪S2∪S3)thenS1yS2|S3
whereGmAn(S
1∪S2∪S3)is the moral graph of the smallest ancestral set containingS1∪ S2∪S3.
As a particular case, we can consider S1 = {vi}, S2 = {vj} andS3 = {vk}, sub-sets ofV. So the verticesviandvjare independent givenvk, if every path fromvito vjin the moral graphGmAn({v
i,vj,vk})includes the verticevk. We will writeviyvj|vk. In general the procedure to check whether two nodesviandvjare independent given a setS, i.e.viyvj|S, can be schematized as follows:
1. consider the ancestral set of{vi,vj}∪S, that isAn({vi,vj}∪S) =A;
2. draw the subgraph corresponding to the ancestral setGA ⊂G;
3. construct the moral graph ofGA,GmA:
•connecting with undirected edges all pairs of vertices inpa(v) ∀v∈A;
•replacing all the edges inGAwith undirected edges;
4. check if there does not exist any path connectingviandvjinGmA that does not involve any node ofS (i.e. ifS is separatingviandvjonGmA).
In this thesis, we will consider directed graphs with no directed cycles since in the railway network these would represent a set of impossible precedence constraints. In fact the railway timetable is a list of ordered time events. These can be enumerated in such a way thati< jif and only if eventihappens before event j.