Kalypso 1D2D Grid-WPS Interface

The purpose of the Kalypso 1D2D Grid-WPS interface to be developed is a grid extension of geoprocessing operations for flow simulation provided by the Kalypso Simulation Platform. These operations will define a WPS application profile for flood simulation. The profile will then be implemented using the Grid-WPS framework described in Chapter5.

Figure7.1.:Flood simulation use cases for flood managers.

The Kalypso Simulation Platform

The KalypsoSimulation Platform provides a user interface to numerical simulation models in water resources, among which is Kalypso 1D2D [SA+09] employing the low-level calculation core RMA·Kalypso(see Section7.3). Kalypso is also an OGC-conforming geoprocessing server for simulations. The server is fully integrated into the base of the KalypsoSimulation Platform and provides WPS processes for the different Kalypsosimulation models. The KalypsoSimulation Platform either starts its own, private simulation server or connects to a remote server instance over the WPS interface.

A Kalypsosimulation server runs inside a Jetty web application container¹ embedded into Eclipse².

Kalypso 1D2D can be characterized as a fat-client GIS user interface. This system is appropriate for building a flood simulation service prototype because, to the author’s knowledge, it is the only current service interface to a flow model supporting the WPS standard. The greatest benefit of using the KalypsoSimulation Platform lies in its extensibility based on Eclipse plug-ins, simple integration with an existing spatial

1http://jetty.codehaus.org 2http://www.eclipse.org/jetty

data infrastructure, as well as giving users of Kalypso 1D2D an easy-to-use portal to running their simulations in the grid.

Kalypso1D2D WPS Profile

WPS application profiles are supposed to enable interoperable processing, service discovery, and orchestration in a specific domain, such as two-dimensional spatial overlay operations (e. g. intersection of geographic features) or, as in the topic of this section, hydrodynamic modeling.

Kalypso 1D2D is not restricted to running the RMA·Kalypsohydrodynamic model.

It provides a WPS profile to unstructured, one- and two-dimensional flow models, in general. As a precondition to interoperability with a diversity of numerical models, the flow model needs to be stored as a collection of standardized GML files in accordance with the Kalypso 1D2D GML application schema that describes the unstructured mesh for the study area (discretization), flow resistances, boundary and initial conditions, as well as simulation control parameters and results.

Flow resistances are a GML feature with a polygon geometry referring to a flow resistance class. All available flow resistance classes for a model are stored inside a database. They define eddy viscosities and equivalent sand roughness coefficients. For interoperability with a SOS, boundary condition hydrographs are kept in the form of GML observations¹ at a given point location and specifying the spatial direction of measurement. Discharge hydrographs (describing the amount of flow) typically refer to the total discharge across a profile section. In the terminology of Kalypso 1D2D, these sections are calledcontinuity lines and are part of the mesh definition. Water level hydrographs can also be assigned to all mesh nodes of a continuity line. A flood scenario can, finally, be defined by acalculation unit, which bundles a (sub-) domain of the mesh, which can be either one- or two-dimensional, together with its control parameters, continuity lines, initial flow state, and boundary conditions. A coupled calculation unit for a1D-2D model may be composed from calculation units for the respective subdomains.

The Kalypso 1D2D simulation for RMA·_K^alypsohas been split into three separate WPS processes defining the WPS profile for flow simulation in this thesis (compare Figure 7.1): PreRMAKalypso (for setting up a flood scenario), ExecuteRMAKalypso (for running the numerical model), and PostRMAKalypso (for evaluation of the re-sults). PreRMAKalypso, prepares the RMA·Kalypsoinput files according to the GML data structure. Initial conditions may be provided in the form of previous results.

ExecuteRMAKalypsothen takes the prepared files, starts the simulation, and monitors

1According to the OGC Observations and Measurements v2.0also published as ISO/DIS19156.

its progress. Finally, PostRMAKalypsoevaluates the result files and transforms them back to GML.

Due to this tripartite design, the calculation core can easily be replaced by a different one, under the condition that the new flow model abides by the Kalypso 1D2D WPS profile. In consequence, this core has to define conforming processes acting as an adapter between the numerical model and the Kalypso 1D2D model data structure.

Kalypso1D2D Grid-WPS

The Kalypso 1D2D Grid-WPS is the entry-point for starting a flow simulation in the grid. It fulfills the purposes of configuring a coupled flow model for a given flood scenario, setting up the simulation in the grid, monitoring its execution, and retrieving the results. This service is usable by both flood modelers and flood managers.

There are two different ways simulation processes can be executed. In the original im-plementation, a simulation could just be run on the Kalypsoserver (DefaultProcess).

The Grid-WPS presented in this thesis supplements Kalypsowith a novel approach to submit simulation processes as grid jobs to a Globus Toolkit 4 GRAM service (SimpleGridProcess). This means a simulation can, for instance, be executed on a computing cluster or another grid resource. TheSimpleGridProcessrequires a GRAM endpoint URI and valid grid credentials. These credentials are obtained from the cur-rent Grid-WPS context. In this way, a simulation can use either the credentials of the Kalypsoserver or the credentials a Grid-WPS user has delegated to the server.

Both process implementations run the simulation in a so-called sandbox. The idea behind this concept is that the process is confined to run inside a working directory where all process inputs and outputs, and, usually, also the executable file, are placed (compare Chapter5, Section5.2). The process publishes its sandbox directory to the service that started the process, so that this information can be used, for example, to access intermediate outputs and to monitor the progress of its execution. The execution environment of the process, local or in the grid, remains transparent to the service, as long as it is entitled to access the sandbox directory. ASimpleGridProcessrelies on the assumption that there is a GridFTP service running on the GRAM server and publishes the sandbox directory as a URL with the GridFTP protocol.

As a stand-alone component, the Kalypso 1D2D Grid-WPS already allows to run a simulation in the grid in conformance with the level1parallelism sketched in Section7.3, i. e. submission to a computing cluster. The parallelization on level 1is achieved by submitting an MPI job with multiple processes to the GRAM. The only communication on this level is taking place inside the cluster. From the perspective of the Kalypso

1D2D simulation, the MPI grid job is treated as an atomic process. Distribution of work for a single subregion is achieved on the algebraic level.

For the support of level2parallelism, i. e. execution across multiple computing clusters in a grid, a user can set up his model by defining a coupled calculation unit with multiple adjacent one- or two-dimensional subdomains. Originally, the coupled model could only be run in a single process. The Grid-WPS adds the alternative to run the calculation units in a distributed fashion using separate processes for each subdomain coupled at the interfaces (internal Kalypso 1D2D continuity lines). In this way, the subdomains can now be distributed across different loosely-coupled clusters in the grid.

In case such a distributed simulation of a coupled model has been configured, Pre-RMAKalypsodetermines the adjacency information, internal boundary lines, and the types of boundary condition to be exchanged between adjacent regions automatically from the discretization model. This information is required to coordinate the distributed simulation. All calculation cores may immediately begin with the execution until the first synchronization point is reached.

In the following section, this Grid-WPS will be extended by a grid service component, calledflow model coupling service, which has the task of coordinating the execution of a coupled simulation, i. e. when it is to be distributed across multiple grid resources using the second level of parallelism.