Big Da a In pi ed Wa e Managemen Pla fo m fo Sen o Da a
Vik o Dmi i e 1, Jo ge Ma GÂome and Man el O me
Ab ac :The age of he da a gene a ed b he a io pe of en o i add e ed o be one of he p omi ing o ce of imp o emen and one of he co e one of he Ind 4. ini ia i e of he Ge man go e nmen . The domain of en o da a managemen con in o l demon a e i abili
o imp o e c en l e i ing p oce e in a io di ec ion . De pi e heo e icall applicabili of he ame app oach of he en o da a managemen in he domain o he ha a e , c en o k foc e on demon a ing p elimina e l of he e ea ch done i h foc on a e managemen . The p elimina e l demon a ed in he fo m of p incipal pla fo m a chi ec e. The main idea behind he pape i o ho he p incipal a chi ec e in o de o open a f e oppo ni fo b ild- ing a p o o pical implemen a ion of he p opo ed a chi ec e la e on. The p incipal a chi ec e ha an big da a phenomena p incipal like ol me, eloci and a ie b -de ign.
Ke o d : en o , ma da a, big da a, a e managemen , in e ne of hing
1 In od c ion
Wa e e o ce i a a ing poin of li e fo all li ing pecie on o plane . In ca e of he pace mi ion in ea ching e a e e ial life, he a ge n mbe one i ea ching fo a ailabili of a e . Mo life on ea h need f e h a e , b onl . % of he a e i on
he plane calc la ed a d inkable [Gl9 ]. So i i impo an o e o a e ainable and en ible. B de pi e he e a e a lo of di c ion abo he a e all a o nd he globe, he main q e ion ha ill need an an e i Ho e can ppo a e ainable age i h info ma ion and comm nica ion echnologie (ICT)? De pi e he opic of e o ce
ainabili i no a ne one, he a e a mo impo an e o ce fo h man, na e and ind need mo e a en ion f om diffe en ie poin , an e ample of ch i ICT. Wa e i no longe an endle e o ce a he e ha e o hink abo dependable me hod and look fo app oached o enhance ainabili a pec of a e ili a ion.
I i a common and ide p ead idea among people li ing in Ge man ha he ha e no p oblem i h he a e ppl , b i hin he ne fe ea people i h high p obabili ma face ome of he p oblem ch a o he co n ie a e facing.
1Uni e i of Oldenb g, B ine Info ma ion S em VLBA, Amme lÈande Hee . 114-118, 1 9 Old- enb g, ik o .dmi i e @ ni-oldenb g.de
Uni e i of Oldenb g, B ine Info ma ion S em VLBA, Amme lÈande Hee . 114-118, 1 9 Old- enb g, jo ge.ma .gome @ ni-oldenb g.de
Uni e i of Oldenb g, B ine Info ma ion S em VLBA, Amme lÈande Hee . 114-118, 1 9 Old- enb g, man el.o me @ ni-oldenb g.de
E peciall , he ni a e concen a ion in he a e of he No h We egion in Ge man i a p omi ing p oblem, ne o alini a ion and pop la ion g o ing, hich ill de®an l a ac ing fa mo e a en ion in he f e. Some in i ion a e o king on he opic of p e en ion ch p oblem . The a ed o e info ma ion and comm nica ion echnolo- gie o i ali e and comp e he cena io , in hich he im la e he p oblem ha a e abo o appea . B cond c ing diffe en im la ion and i ali a ion he a e ing
ea ching fo applicable ol ion .
The o he ide of he p omi ing ICT ili a ion i hin he con e e o ce managemen , and a e managemen in pa ic la , i he age of o he da a o ce like Open Da a, a
ell a e ice and in e face de eloped on op of he Open Da a [Da1 , MRS1 , G 11].
The Open Da a i eme ging a one of he mo p omi ing al e adding da a-cen ic ap- p oache . Open Da a i a ac ing mo e and mo e a en ion f om he b ine and go e n- men ide . In 11 McKin e p edic ed ha big da a ma ke fo he E opean Sec o b
i going o be a o nde 5 billion pe ea [Ma11], he e he Open Da a i pla ing a eall g ea ole. The f he ep fo he o k a e o combine Open Da a i h p op i- e a da a f om a e pplie and a e a ho i ie in o de o c ea e mo e ainable and al e adding p oce e hea il ppo ed b deci ion made pon da a.
E peciall , he concep of openne i hin he Open Da a app oach i p omi ing o make ci ie ma e . An idea of making ci ie ma e a ac mo e a en ion o he Sma Ci ie [To1 ] phenomena. Follo ing idea and concep behind Sma Ci ie allo e-
o ce ch a ban ci e ice , hei inf a c e ili a ion o being mo e ain- able, hich i e impo an , e en c cial, i h he con in o l g o ing bani a ion [Ki14, Ha1 ]. De pi e age of he a e i e common in almo an loca ion , he po ibili of p ope and mo op imal a e e o ce ili a ion i ho m ch in e - men ( i h j ICT d i e inno a ion ) i e p omi ing in highl pop la ed and ell o gani ed loca ion ch a ci ie .
Main Idea
The age of he da a gene a ed b he a io pe of en o , hich can be a pa of he da - o-da ed ma phone de ice, o ome o he dedica ed de ice ha i able o mea e empe a e, o ga p e e i hin he ca engine, a e ag eed o be one of he p omi ing o ce of imp o emen and one of he co e one in he In e ne of Thing (IoT) [AIM1 , G 1 ] and Ind 4. ini ia i e of he Ge man Go e nmen [Ka1 ]. The domain of en o da a managemen con in o l demon a e i abili o imp o e he c en e i ing p oce e in a io di ec ion .The c en o k i foc ed on demon- a ing p elimina e l of he e ea ch done i h foc on a e managemen . The idea of he o k i ® of all o ho he p incipal a chi ec e and han b ild a p o o pi- cal implemen a ion of he p opo ed a chi ec e la e on. The p incipal a chi ec e ha an big da a p incipal like ol me, eloci and a ie . C en o k i going o demon a e he p incipal a chi ec e ha ill enla ge he po en ial of a e managemen , in pa ic - la , he a chi ec e i add e ing follo ing eq i emen (a) con olida e en o and e en
4
da a, (b) o e da a, (c) offe in e ac i e anal i and anda d epo ing, (d) offe pl ggable componen app oach.
In he a e managemen he e a e n mbe of diffe en mea ed a e ha can be aced b pa ic la en o (e.g. p e e, ¯ id-¯o , le el of a e poll ion, al a e in ion, e c.). B adding on op of he en o da a, a b ine da a f om he companie and he da a f om c ome a e in e e ing cena io fo he a e managemen can be c ea e b
he e pe f om a io domain (e.g. ainabili ). S ch h ge amo n on da a f om man diffe en o ce c ea e a mo l common dimen ion of he big da a: Vol me, Veloci and Va ie [La 1]. On he o he hand, he app oach i de igned fo age of b ine and c ome da a fo economic and ainable goal a ell, fo e ample b inco po a ing hem oge he in o p edic ion model o ainable epo ing. D e o he abo e men ioned he o k ha a ele ance o he En i onmen al Managemen Info ma ion S em (EMIS). An implemen a ion of he e ca e i h EMIS in ide p opo ed a chi ec- e ho ld demon a e he applicabili of idea de c ibed b he c en o k on mo e g an la le el.
The idea i o demon a e abili ie of ICT o help in ol ing man pe of p oblem , hich a e managemen i c en l facing and going o face in he f e. Taking in o con ide a ion add e ed a chi ec e ill help he o gani a ion o be mo e ¯e ible in e m of da a o age and p oce ing, a ell a p epa e he inf a c e, hich i mainl fo- c ed on he da a anal i pa , fo f e change and gene a ion of eno mo amo n of en o da a. The gene al app oach p opo ed b he o k o mee add e ed eq i emen can be di ided in o follo ing ep : (a) con olida e hi o ical a e ela ed da a (b) con-
olida e eal ime a e da a f om en o (c) i ali e a e e e oi i h e pec o hei pa ame e ol me, geo-loca ion, a e con mp ion and a e eco e po en ial (d) appl a i ical me hod (e.g., da a mining, machine lea ning, e c. ).
A chi ec e O e ie
Gene al o e ie of he a chi ec e of he Wa e Managemen Pla fo m fo Sen o Da a i ho n on he Fig.1. The co e componen of he o e all a chi ec e i an In-Memo Da aba e, hich can be de®ned a an da aba e managemen em (DBMS) ha ili e capaci of main memo and e di k a he econda o age [PZ1 ]. U age of main memo a he p ima da a o age allo o o e come ph ical ob acle of da a o age ba ed on he magne ic o a ing di k . The age of In-Memo Da aba e ill igni®can l impli®ca ion da a model (beca e mo of he o k a o nd ch a p e-agg ega ion i hin he anal ical c be a e no mo e eq i ed), i al o ho ld be b ing a igni®can pe fo mance boo o he all pe of q e ie ( epo ing, add-hoc, OLAP, e c.).
The p ima ole of he Wa e Managemen Pla fo m fo Sen o Da a i o offe cen al- i ed da a o age, in eg a ion and anal i facili ie fo each pa ic la a e ela ed ce- na io. Pa of he pla fo m, ha i e pon ible fo he da a o age con i o of 5 main componen and addi ional abili of pl gging c om componen : (1) E en P oce ing Engine ( ) Long-Te m S o age ( ) In-Memo Da aba e (4) Repo ing (5) In e ac i e
5
Fig. 1: A chi ec e Of Wa e Managemen Pla fo m fo Sen o Da a
Anal i . The pla fo m ge en o da a h o gh he Sof a e D i en Sen o Da a H b. The in e nal comm nica ion be een componen i hin he pla fo m a e facili a ed b com- monl accep able anda d and p o ocol (ODa a, ODBC, HTTP S, e c.). In he pa ic la ca e, comm nica ion be een Sof a e D i en Sen o Da a H b and Sen o RTU- ill be ba ed on CIM (Common Info ma ion Model) ha b ing in e ope abili o he a ge a chi ec e and al o ppo comp abili i h a ide ange of de ice .
A i ho n on he Fig.1 of a e componen a e n in ide he p i a e clo d, ch ap- p oach allo o cale he o e all a chi ec e i h acco dance of demand. U ing enhanced clo d ol ion ack ill al o p epa e a f e land cape fo he deplo men of he a ge ed ol ion. In eg a ion of ad anced en o , ch a RTU (Remo e Te minal Uni ), in he a chi ec e h o gh he Sen o Da a H b, a i ho n in Fig. 1, ill b ing ¯e ibili o he a chi ec e in he en e of in eg a ing ne pe of RTU (e i ing on he ma ked and f e one ).
A i a men ioned abo e, e e componen of he pla fo m ho ld n in he clo d ack.
The E en P oce ing Engine can be implemen ed i h he help of f ame o k ha allo p oce ing of con in o da a, fo e ample Od e F ame o k [Ap1 ]. The Long-Te m S o age cane be implemen ed a a di ib ed da aba e o a a di ib ed ®le em i h p oce ing engine on op. In pa ic la , he ole of he Long-Te m S o age can be o e aken b e.g. Apache Hadoop and o he componen f om i Eco em [Va1 ]. An In-Memo Da aba e ill be ed fo he e ca e, in hich eq i emen a e demanding da a p oce -
ing hea ope a ion ch a ad-hoc q e ie , KPI i ali a ion o e en da a mining and machine lea ning. In e ac i e Anal i of da a ill in eg a e diffe ed anal ical capabili ie
ch a highl c omi able anal ical ie . The p edic i e anal ic can be a ell in- eg a ed in o he pla fo m h o gh Pl gable Componen . Repo ing can be done b mean of an openl a ailable ool ch, fo e ample Pen aho BI [LBF14] o e en ome o he in-ho e de eloped of a e ol ion.
4 Rela ed Wo k
The e a e n mbe of p ojec con ib ing o he ainabili of he pa ic la ci and making a ci a ma one b in eg a ing a io of da a eam i hin a ingle em and p oce ing in eg a ed da a in o de o achie e be e le el of nde a ing of ongo- ing e en and p oce e i hin he pa ic la en i onmen and con e . One of he ch p ojec i he Ci P l e [Ko14, Ga15], he p ojec ha a f nded b he EU and a cond c ed b he m l ina ional con o i m. The majo goal of he Ci P l e p ojec i o bene® f om in e connec ed eco em of he ci and imp o e ainabili b mo e ef®- cien (op imal) ope a ion pe fo med i h ci e ice and inf a c e. Ci P l e ied [Ko14, Ga15] o in eg a e a io o ce of he eaming da a gene a ed b he de ice
oge he i h ocial da a eam fo in a abo ci in o an ingle la ge cale eal- ime em.
The o he e ample of he ela ed o k i he p ojec called C be Wa e [CMI1 ]. The main goal of he p ojec i o cond c a em, hich i able p o ide a deci ion p- po in i k ela ed en i onmen foc ing on an in eg a ed managemen of a e hed and poll ion ca ed b ome acciden . D e o he o e helming amo n of da a and geog aphical he e ogenei of he da a o ce ( en o ), he co e em i een o be a di ib ed and ba ed on he hea il ba ed on he Se ice-O ien ed A chi ec e (SOA) p inciple fo in e ope abili .
De pi e he e a e a li of imila i ie be een he p opo ed Wa e Managemen Pla fo m fo Sen o Da a and C be Wa e in main goal , he e a e a li of p incipal diffe ence be een hem on a chi ec e le el. The main goal of he C be Wa e i o help in deci-
ion , he he he main goal of he p opo ed app oach i o implif o king i h da a on all le el and gi e a ong and ob ool o he domain e pe o o k i h da a i ho in e ing m ch ime in o e abli hing a p ope da a enginee ing pipeline. Ano he diffe ence o be men ioned i an in e ac ion be een componen in p opo ed app oach.
A pa ic la in e ac ion i done h o gh p e io l de®ned p o ocol , hich o k i h igni®can ef®cienc in compa i on i h Se ice-O ien ed A chi ec e (SOA), beca e he deci ion he he o e o no o e pa ic la p o ocol fo connec ion , a ell a dep h of in eg a ion in o p o ocol of an componen , can be done ba ed on a io eq i e- men (pe fo mance, fa l - ole ance. e c.). S ch ela ici in p o ocol in eg a ion make p opo ed a chi ec e ¯e ible. O he igni®can diffe ence i ha he Wa e Managemen Pla fo m fo Sen o Da a i e ca e independen . One of he main foc of he p opo ed app oach i o make a an fe p oce of e e ingle da a poin , hich i gene a e b
en o , an pa en fo he end e .
5 Concl ion and F e Wo k
C en o k demon a e demand fo he pla fo m, hich i no onl able o manage a e ela ed da a, b i al o able o face pcoming challenge add e ed b fa e ol - ing ini ia i e ch a Ind 4. and In e ne of Thing . The gene al o e ie of he p incipal a chi ec e fo he Wa e Managemen Pla fo m fo Sen o Da a a ho n and de c ibed. F e o k ill be he de ailed de c ip ion of he a e e ca e and he p o o- pical implemen a ion of he p opo ed pla fo m oge he i h an ind pa ne , hich a e ho ing a g ea in e e in cond c ing a p o o pe ba ed on he p opo ed a chi ec e and fo e eeing a h ge bene® fo hem el e and hei c ome .
Refe ence
[AIM1 ] A o i, L igi Ie a, An onio Mo abi o, Giacomo: The in e ne of hing : A e . Com- p e ne o k , 54(15): 8 ± 8 5, 1 .
[Ap1 ] Appel a h, H.-JÈ gen Gee en, Denni G a nde , Ma co Michel en, Timo Nickla , Daniela: Od e : A Highl C omi able F ame o k fo C ea ing Ef®cien E en S eam Managemen S em . In: P oceeding of he h ACM In e na ional Confe - ence on Di ib ed E en -Ba ed S em . DEBS 1 , ACM, Ne Yo k, NY, USA, pp.
± 8, 1 .
[CMI1 ] Ciolofan, So in N Mocan , Ma iana Ioni a, Anca: Di ib ed C be inf a c e fo Deci ion S ppo in Ri k Rela ed En i onmen . In: Pa allel and Di ib ed Comp ing (ISPDC), 1 IEEE 1 h In e na ional S mpo i m on. IEEE, pp. 1 9±115, 1 . [Da1 ] Da ie , Tim: Open da a, democ ac and p blic ec o efo m. A look a open go e nmen
da a e f om da a. go . k, 1 .
[Ga15] Gan , F iede P chmann, Daniel Ba naghi, Pa am Ca e , F ancoi : A P ac ical E al- a ion of Info ma ion P oce ing and Ab ac ion Techniq e fo he In e ne of Thing .
15.
[Gl9 ] Gleick, Pe e H e al.: Wa e in c i i : a g ide o he o ld f e h a e e o ce . O fo d Uni e i P e , Inc., 199 .
[G 11] G ein, Michael B: Open da a: Empo e ing he empo e ed o effec i e da a e fo e e one? Fi Monda , 1 ( ), 11.
[G 1 ] G bbi, Ja a a dhana B a, Rajk ma Ma ic, Sla en Palani ami, Ma im h : In- e ne of Thing (IoT): A i ion, a chi ec al elemen , and f e di ec ion . F e Gene a ion Comp e S em , 9( ):1 45±1 , 1 .
[Ha1 ] Hancke, Ge ha d P Hancke J , Ge ha d P e al.: The ole of ad anced en ing in ma ci ie . Sen o , 1 (1): 9 ±4 5, 1 .
[Ka1 ] Kage mann, Henning Helbig, Johanne Hellinge , A iane Wahl e , Wolfgang: Rec- ommenda ion fo Implemen ing he S a egic Ini ia i e INDUSTRIE 4. : Sec ing he F e of Ge man Man fac ing Ind Final Repo of he Ind ie 4. Wo king G o p. Fo ch ng nion, 1 .
[Ki14] Ki chin, Rob: The eal- ime ci ? Big da a and ma bani m. GeoJo nal, 9(1):1±14, 14.
8
[Ko14] Kolo ali, Sefki Be m de -Edo, Ma ia P chmann, Daniel Gan , F iede Ba naghi, Pa am: A Kno ledge-ba ed App oach fo Real-Time IoT Da a S eam Anno a ion and P oce ing. In: In e ne of Thing (iThing ), 14 IEEE In e na ional Confe ence on, and G een Comp ing and Comm nica ion (G eenCom), IEEE and C be , Ph ical and Social Comp ing (CPSCom), IEEE. IEEE, pp. 15± , 14.
[La 1] Lane , Do gla : -D Da a Managemen : Con olling Da a Vol me, Veloci and Va ie . META G o p Re ea ch No e, Feb a , , 1.
[LBF14] Lapa, Joaq im Be na dino, Jo ge Fig ei edo, Ana: A Compa a i e Anal i of Open So ce B ine In elligence Pla fo m . In: P oceeding of he In e na ional Confe ence on Info ma ion S em and De ign of Comm nica ion. ISDOC 14, ACM, Ne Yo k, NY, USA, pp. 8 ±9 , 14.
[Ma11] Man ika, Jame Ch i, Michael B o n, B ad B ghin, Jacq e Dobb , Richa d Ro - b gh, Cha le B e , Angela H ng In i e, McKin e Global: Big da a: The ne f on ie fo inno a ion, compe i ion, and p od c i i . 11.
[MRS1 ] Magalhae , G a o Ro ei a, Ca a ina S o e , Sha on: Open Go e nmen Da a In e - media ie : A Te minolog F ame o k. In: P oceeding of he h In e na ional Confe - ence on Theo and P ac ice of Elec onic Go e nance. ICEGOV 1 , ACM, Ne Yo k, NY, USA, pp. ± , 1 .
[PZ1 ] Pla ne , Ha o Zeie , Ale ande : In-Memo da a managemen : Technolog and appli- ca ion . Sp inge Science & B ine Media, 1 .
[To1 ] To n end, An hon M: Sma ci ie : big da a, ci ic hacke , and he q e fo a ne opia. WW No on & Compan , 1 .
[Va1 ] Va ilapalli, Vinod K ma M h , A n C Do gla , Ch i Aga al, Sha ad Kona , Mahade E an , Robe G a e , Thoma Lo e, Ja on Shah, Hi e h Se h, Siddha h e al.: Apache hadoop a n: Ye ano he e o ce nego ia o . In: P oceeding of he 4 h ann al S mpo i m on Clo d Comp ing. ACM, p. 5, 1 .
9
