Bridging the Technological Gap

One of the major problems lies in the harmonization of the standard for grid service development, the WSRF (see Chapter 2), and the OGC web service standards, in particular the geoprocessing standard WPS (see Chapter4).

Baranski [Bar08a; Bar08b] coined the term “gridification” as the adaptation of existing geoprocessing applications to the requirements and expectations of a grid environment.

He suggested two different levels of gridification:

1. an OGC interface is used as a wrapper to an implementation using the grid in the back (low-level gridification), or

2. fully integrating an OGC service into the grid middleware as a grid service with a standards-compliant OGC proxy service in front (high-level gridification).

The author only implemented the low-level gridification approach reaching the con-clusion that there is no generic way of providing an OGC grid service conforming to the original OGC standard. The implementation had the disadvantage that data had to be transfered to and from the grid. The second approach was successfully demonstrated by Hobona et al. [HFJ07] and Foerster et al. [FS+11]. Padberg and Kiehle [PK09b; PK09a] focus on the extension of an existing spatial data infrastructure through the use of grid infrastructures. They suggest to implement the WPS standard as a grid middleware service with an OGC-compatible proxy service. Not conforming to OGC interfaces on the grid service level was not considered to be a hindrance.

In this thesis, a high-level gridification of the WPS is sought. Additionally, the grid service interface will be made OGC-compliant in the Grid-WPS framework (see Sec-tion5.2).

Lee and Percivall reported on the collaboration between the two organizations for stan-dardization in the grid world, the OGF, and in the geographic information community, the OGC. They identified the technologies from the geospatial and grid communities that have to be integrated in order to build a distributed geospatial system, i. e. key OGC and OGF standards. According to the authors, a major challenge is to hide the complexity of the grid and to simplify tools for use in the geospatial application domain.

The Global Earth Observation System of Systems (GEOSS) is a community that would benefit from such an integration effort [Lee08].

Krüger and Kolbe [KK08] made the main observation that OGC Web Services are lacking a standard security concept. They summarized many results of the German Spatial Data Infrastructure Grid (GDI-Grid) project, which gave a motivation to this thesis. The solution in [PK09b; PK09a] used a MyProxy service (see Chapter2, Subsection2.3.3) to obtain a grid certificate for access to the grid using user name and password credentials submitted as parameters of the WPS process execution. These allowed the WPS process to submit grid jobs and manage the required storage resources.

[LS+08a; LS+08b; LZ09] used WPS for processing large digital elevation models. They implemented a conventional WPS that was able to run grid jobs using Globus Toolkit4. A grid certificate was also obtained from a MyProxy service. It remained an open question how storage resources are used to publish data in the grid for subsequent calculations. This problem was then addressed in [LK+09], which motivated the development of the terrain discretization service [KP09] that lead to the flow model discretization service (see Chapter6) developed in this thesis. The Grid-WPS framework to be developed in this chapter will provide a solution to the problem of intermediate data storage in the grid.

An extension to the WPS standard, called the Temporary Resource Store (TRS), was suggested by Keens [Kee06]. The author stated that WPS ought to consider adding support for stateful resource management to the WPS specification, such as it is defined in the WSRF. Resource management is, in effect, required for processing large data. He further suggests to introduce the operations putResource, getResource, destroyResource, andsetTerminationTime. The getResourceoperation would be used to retrieve the output of a process andputResourcewould send a resource to the TRS, which returned a resource identifier that could then be used as an input parameter in the ^execute request. Resources in the TRS would be subject to a termination time determining when the resource was going to be destroyed automatically by the server. setTerminationTimeanddestroyResourceallowed to control the lifetime of resources in the TRS. Such functionality, as suggested by [Kee06], could be implemented on top of the Grid-WPS framework to be described in Section5.2.

An Overview of Grid and Geoprocessing Standards

Many of the publications dealing with OGC standards in the context of grids deliver valuable information that can be applied to the gridification of OGC web services. The WPS specification is a very recent development, so only literature newer than about

2006makes a direct reference to it, after the WPS0.4request for comments document [SG+05] was issued.

Krüger and Kolbe [KK08] presented an overview of different options for gridification of OGC web services. In their view, grid services are seen as an orthogonal, supporting technology to WPS. The authors make the implicit assumption that users in a spatial data infrastructure have no knowledge of grid technology. However, this assumption only holds for the first two approaches to gridification. The last option (on 66), on the other hand, which also seems to be the most promising one, requires a more integrative view on SDI and grid infrastructures. WPS and grid standards are going to be harmonized in the Grid-WPS framework that is part of both infrastructures.

It should be noted, however, that there is no standard for processing services in the OGSA issued by the OGF. In fact, most of the specifications mainly deal with grid infrastructure, execution and data management on computing clusters, service security, as well as resource management and information services [GM+09].

The OGSA is defined mainly by the OGSA WSRF Basic Profile [FMS06], the Job Submission Description Language (JSDL) [AB+05; AB+08], the OGSA Basic Execution Service (OGSA BES) [FG+07], the HPC Basic Profile [DH+07], which incorporates the JSDL HPC Profile Application Extension [HS+07] and the HPC File Staging Profile [WH08]. In the following table (Table5.1), a comprehensive overview and comparison of the relevant features and standards of both WPS and OGSA grid services is given.

They provide the requirements for the Grid-WPS framework that can co-exist in both a grid infrastructure and an SDI.

Table5.1.:ComparisonoftheWPSandOGSAWSRFBasicProfilestandards. FeatureWebProcessingService1.0WSRFBasicProfile1.0 InterfaceOGCWebServicesCommonSpecification1.1.0, KVPorXMLencodingoverHTTPWS-IBasicProfile1.1a,WS-Addressing1.0–Core StateWPSstatusdocument(optional)WS-ResourceProperties1.2,WS-ResourceLifetime 1.2,WS-BaseNotification1.3 Security—OGSABasicSecurityProfile2.0b DiscoveryOGCCatalogueServicefortheWeb(CSW)UniversalDescription,DiscoveryandIntegration (UDDI)Version2 MetadataCapabilitiesdocument— aWS-IBasicProfile1.1dictatesaWSDL1.1interfacedescriptionrestrictedtoaSOAP1.1“rpc”or“document”stylebindingwith“literal” messageencodingandtransportrestrictedtoHTTP/1.1 bIncludingWS-IBasicSecurityProfile1.0,SecureAddressingProfile1.0,SecureCommunicationProfile1.0,WS-Security1.0withX.509 CertificateTokenProfile,WS-SecurityPolicy1.2,WS-Policy1.5withAttachment Table5.2.:ComparisonoftheWPSandHPCBasicProfile(withHPCFileStagingProfile)standards. FeatureWebProcessingService1.0HPCBasicwithFileStagingProfile1.0 OperationsgetCapabilities,describeProcess,executecreateActivity,getActivityStatuses, terminateActivities,getActivityDocuments Statemodel—Pending,Running(Stage-in,Executing, Stage-out),Finished,Terminated,Failed Input/OutputIn-lineorbyreferenceURI,resultsinresponse documentorrawdataFile-basedJSDLDataStaging(source/targetURI definition) Credentials—Username/tokenorX.509certificate