RDIF@AWI – Raw Data Ingest Framework

(1)

Polar5/6 Devices

Neumayer Devices

Mya II Devices

Internet

AWI

26.10.2013, Peter Gerchow, peter.gerchow@awi.de

RDIF@AWI – Raw Data Ingest Framework

AWI Research Platforms

BREMERHAVEN

Am Handelshafen 12 27570 Bremerhaven Telefon 0471 4831-0 www.awi.de

Temp. Data Exchange

(/hs/datex)

PANGAEA Archive (/hs/usero)

Centralized near Realtime Data

Metadata Generation

/hs/usero/polarstern/fishsounder/ANT24_2/

ANT24_2-D20090711-T103035.raw ANT24_2-D20090711-T104035.raw ANT24_2-D20090711-T105035.raw

…

Mastertrack (DShip)

Documentation / Manuals

• Summary Description

• Data Storage and Archiving

• Device and System Responsibles

• DShip – Data Acquisition System

AWI internal File based Data Access

/hs/usero/polarstern/fishsounder/ANT24_2/

…

… MAC using smb://hssrv1/usero

… PC Using \\hssrv1\usero

access„readonly“

Web based Internet Access

Simple Pre Processing of

Data Access

Protection by Web Application

Centralized offline Data Transport

Heincke Devices

Polarstern Devices

Shipborne Devices

DShip-System with Stationbook

Multibeam Echosounder

Ferrybox

… Sediment

Echosounder

Weather Station TSG (2x)

External Devices

ADCP CTD

Navigation System

OFOS

Raw Data Storage with Tape Library … Data Storage with Tape Library

Dship

(/hs/dms)

Projects

(/hs/bsys) (/hs/csys) (/hs/gsys)

(…)

Universal Archive

(/hs/usera)

PANGAEA Archive

(/hs/usero)

Access Protection by

Filesystem

Metadata Generation File Structure Validation

Peter Gerchow

Data Access for Validation and Post Processing

v2