O v e rv ie w

(1)

S a fe ty -C rit ic a l S y st em s

P ro f. D r. J a n P el es k a U n iv er si t¨a t B re m en — T Z I D r. In g . C o rn el ia Z a h lt en V er ifi ed S y st em s In te rn a ti o n a l G m b H

VERIFIEDSYSTEMSINTERNATIONALGMBH,BREMEN

(2)

O v e rv ie w

1.TheNotionofDependability2.Safety-RelatedStandardsandV-Models3.ModellingSafety-CriticalSystems4.HazardAnalysisandRiskAssessment5.DesignCriteriaforSafety-CriticalSystems6.Validation,VerificationandTestofSafety-CriticalSystems

(3)

O v e rv ie w

(4)

Dependability–GeneralDefinition

Ourobjective:Developmentandverificationofsafety-criticalreactivesystemsthatcontinuouslyinteractwiththeirenvironmentandcontrol(partsof)theenvironmentinordertoperformtheuser-requiredservicesandenforcetherequiredsafetyproperties.

•Dependability:Thetrustworthinessofacomputersystemsuchthatreliancecanbejustifiablyplacedontheserviceitdelivers.

•Service:Systembehaviourasitisperceivedbyitsusers.

•User:Anothersystem(humanorphysical)interactingwiththetargetsystem.

(5)

DependabilityAttributes

•Availability:readinessforusage

•Reliability:continuityofservice

•Safety:avoidanceofcatastrophicconsequencesontheenvironment

•Security:preventionofunauthorisedaccessand/orhandlingofinformation

Securityattributes:

–confidentiality

–integrity

–availability

(6)

Dependability–Faults–Errors–Failures

•Failure:deliveredservicenolongercomplieswithspecification

•Error:partofthesystemstatewhichislikelytoleadtosubsequentfailure

•Fault:causeofanerror

(7)

Dependability–Fault-ToleranceversusFault-Avoidance

Classificationofmethodsachievingdependability:

•Fault-Tolerance:provideaservicecomplyingwiththespecificationinspiteoffaults

•Fault-Avoidance(Fault-Prevention):a-prioripreventionoffaultoccurrenceorintroduction

•Fault-Removal:reducethepresence(number,seriousness)offaults

•Fault-Forecasting:estimatethepresentnumber,thefutureincidence,andtheconsequenceoffaults

OurFavouriteApproach:Fault-ToleranceonHWlevel—Fault-AvoidanceonspecificationandSWlevel!

(8)

Dependability—SafetyversusLivenessProperties

Incomputerscience,specificationsareclassifiedascontaining

•SafetyPropertiesS:Anysequenceofeventsortransitionsetc.violatingScontainsaprefixallofwhoseinfiniteextensionsviolateS.(Salwaysholds)

•LivenessPropertiesL:AnyarbitraryfinitesequenceofeventscanbeextendedtoaninfinitesequencesatisfyingL.(Lfinallyholds)

Forsafety-criticalreal-timesystems,alldependabilityre-quirementsshouldbespecifiedassafetyproperties:Live-nesspropertiescanonlyguaranteethataservicewillFINALLYbedelivered,whichisnotsufficientinthecon-textofhardreal-timesystems.

(9)

Dependability—ClassificationofSystemBehaviour

Systemstatespaceispartitionedintothreeareas.Systemexecutionisregardedassequenceofstatetransitions,possiblyaccompaniedbyvisibleactions(input/output).

oo

ooo

o oo

o

o o trace showing normal behaviour

o

o trace showing acceptable behaviourNormal BehaviourExceptional BehaviourCatastrophic Behaviourtrace showing normal, exceptionaland catastrophic behaviour:

- transition to exceptional behaviour

- transition to catastrophic behaviour (ALARP Region)

(10)

O v e rv ie w

(11)

Safety-RelatedTerminology

•Accident:Anundesiredandunplannedeventthatresultsinaspecifiedlevelofloss.

•SeverityofanAccident:Specificationoftheleveloflosscausedbytheaccident,e.g.,neglectable–minor–critical–catastrophic

•Hazard:Somethingthathasthepotentialtodoharmorcanleadtoanaccident.Hazards“inherit”theirseverityattributesfromthemostharmfulaccidentstheymaycause.

•(System)SafetyRequirements:Aspecificationstatingtheacceptablerelations

hazardseverity↔probabilityofhazardoccurrence

(12)

Safety-RelatedStandards–CommonUnderstanding

SafetyrequirementsshouldbederivedfromaRiskAnalysis,consistingofHazardAnalysisandRiskAssessment.

•HazardAnalysis:listofpossiblehazards,theirimpactontheenvironment,theirpossiblecauses(e.g.,sequencesoffaultsleadingtoahazard).Typically,itconsistsofthefollowingitems:

–HazardList:collectionoftheidentifiedhazards

–Hazard-SeverityMatrix:relateshazardstoseverity

–Hazard-ProbabilityMatrix:relateshazardstotheprobabilityoftheiroccurrence

–HazardModel:descriptionofthepossiblecauses–thatis,sequencesofevents–leadingtoahazard

(13)

•RiskAssessment:quantitativeorqualitativeestimatesfortheprobabilitythatahazardwilloccur

Accordingtotoday’sstateoftheart,ahazardmodelshouldatleastbesemi-formal,forexampleusingfault-trees,eventtreeanalysisofcause-consequenceanalysis.

(14)

•relateseverityofhazard,probabilityofoccurrenceandsystembehaviourinRiskDiagramorHazardRiskIndex

•deriverequiredeffortforvalidation,verificationandtest(VVT)ofsystemcomponentsfromtheriskdiagramandtheimpactofcomponentfailureonhazardoccurrence

•VVTactivitiesforcomponentswithhighestcriticalityshouldbeperformedbyIndependentParties

(15)

Safety-RelatedStandards–RiskDiagram

normal behaviour exceptional behaviour(ALARP region) catastrophic behaviour(unacceptable region)

(acceptable area) probability of hazard occurrence

severity of hazard occasionallylow probabilityimprobableimpossible probable frequently

criticalneglectableminorcatastrophic

system component

VVT effort

lowvery high hazard causedby component failure/VVT effort required

.

(16)

If–inspiteofallsafety-relatedprecautions–ahazardoussituationresultsinanaccident,thendetailedinvestigationsareperformedwiththeobjectivetopreventre-occurrenceofsimilaraccidents:

•RootCauseAnalysisdenotesthetasktoidentifythecrucialcausesofanaccident.Theterm“rootcause”indicatesthatthecrucialcausestolookforarethoseatthebeginningofthecausalchainsfinallyresultingintheaccident.

•Ofspecialinterestisthesubsetofrootcauseswhoseoccurrencecanbecontrolled(thatis,prevented)bytechnicalororganisationalmeasures.

(17)

•Rootcauseanalysisproceedsaccordingtothefollowingsteps:1.Datacollection2.Causalfactor(“causalchain”)charting3.Rootcauseidentification4.Recommendationelaboration5.Recommendationimplementation

(18)

Safety-RelatedStandards

(19)

Importantstandardsforthedevelopmentofsafety-criticalsystems:

•IEC61508:SicherheitelektronischerSysteme

•IEC61511:FunktionaleSicherheit–SicherheitstechnischeSystemef¨urdieProzessindustrie

•IEC61513:Kernkraftwerke–Leittechnikf¨urSystememitsicherheitstechnischerBedeutung–AllgemeineSystemanforderungen

•EN50129:Bahnanwendungen–Telekommunikationstechnik,SignaltechnikundDatenverarbeitungssysteme–SicherheitsrelevanteelektronischeSystemef¨urSignaltechnik

(20)

Importantstandardsforthedevelopmentofsafety-criticalsystems:

•EN62061:SicherheitvonMaschinen–FunktionaleSicherheitsicherheitsbezogenerelektrischer,elektronischerundprogrammierbarerelektronischerSteuerungssysteme

•ISOCD26262:Roadvehicles–Functionalsafety

(21)

Safety-RelatedStandardsforCivilAircraftSystems

Theoveralltestprocessisdrivenbythefollowinghigh-levelstandards

•AirTransportAssociation(ATA)Chapters.Asystematicdecompositionofaconceptualaircraftintoaircraftsystems.Systemdescriptionsinducethefundamentalfunctionsrequiredinanaircraft

Examples.

–ATA-Chapter21.AirConditioning–ATA-Chapter30.IceandRainProtection–ATA-Chapter32.LandingGear

Note.TheATAdescriptionis“slightlyoutdated”fromtoday’spointofviewsinceitalreadysuggestsafunction↔systemassociation

(22)

TechnicalstandardssuchasRadioTechnicalCommissionforAeronautics(RTCA)andAeronauticalRadioIncorporated(ARINC)standardsspecifythe(minimal)requirementsforequipmentimplementingaircraftfunctions

Examples.

•ARINC653:AvionicsApplicationSoftwareStandardInterface

•ARINC664:AircraftDataNetwork(AvionicsFullDuplexSwitchedEthernet(AFDX))

•RTCADO-200A:StandardsforProcessingAeronauticalData

(23)

TechnicalstandardssuchasRadioTechnicalCommissionforAeronautics(RTCA)andAeronauticalRadioIncorporated(ARINC)standardsspecifythe(minimal)requirementsforequipmentimplementingaircraftfunctions

Examples.

•RTCADO-185B:MinimumOperationalPerformanceStandardsforTrafficAlertandCollisionAvoidanceSystemII(TCASII)

•RTCADO-178B:SoftwareConsiderationsinAirborneSystemsandEquipmentCertification

(24)

O v e rv ie w

(25)

ModellingSafety-CriticalSystems

Controller sensor observables actuator observables observables

Control (EUC) Equipment under

(26)

•PhysicalModelspecifieshowtheEquipmentUnderControl(EUC)behaves.Thismodelshouldbeindependentofthepresence/absenceofacontroller.

•(System)HazardModeldescribesthepossiblecausesthatmayleadtotheidentifiedsystemhazards.

•ControllerModel:specifiesrequirementsforacontrolsystemsuchthat

–EUCsystemhazardswillnotoccur(controllersafetyrequirements)

–additionalnonsafety-relatedEUCbehaviourwillbeensured(userrequirements)

(27)

Observations:

•Systemsafetyrequirementsareoftenspecifiedbyseparateauthorities,notbythecustomer.

•Controllersafetyrequirementshavetobespecifiedbytheteamresponsibleforthecontroller.

•Userrequirementsspecifiedbythecustomermaybeinconflictwithsafetyrequirements.

•Ithastobeverifiedthatthecontrollersafetyrequirementswillfulfilthesystemsafetyrequirements.

Thespecificationofcontrollersafetyrequirementsshouldalwaysbeseparatedfromuserrequirements.

(28)

ModellingSafety-CriticalControllers–SpecificationMethods

SpecificationMethodssuitableforphysicalmodelandcontrollermodelcanbeclassifiedaccordingto

MethodDMFMUBMTBMVVTCG

SA/RT/IM++·-··STATECHARTS·++···SDL·++··+CSP··++++CCS··++++LOTOS··+-++Z++·-··VDM++·-·+UML2.0+++·++HybridUML++++++

DM=DataModel,FM=FunctionalModel,UBM=UntimedBehaviouralModel,TBM=

TimedBehaviouralModel,VVT=supportforValidationVerificationandTest,CG=support

forcodegenerationfromspecifications,+=goodsupport,·=weaksupport,-=nosupport

(29)

Example1:PhysicalModelandControllerModelspecificationwithCSP–doorlockingmechanismforlaboratory:Whenlasersystemisactive,doorshallbelockedandlaboratoryshallbeempty.

Physicalmodel(EUC):

EUC=LASER|||(DOOR[|open,close|]PERSON)LASER=switchOn->laserActive->switchOff->laserPassive->LASERDOOR=open->close->DOORPERSON=open->enter->close->stayInLaboratory->open->leave->close->PERSON

(30)

Example1(continued):Hazardoustracesaretheonescontainingeventsequences

<...,open,switchOn,...><...,open,enter,switchOn,...><...,open,enter,close,switchOn,...><...,open,enter,close,stayInLaboratory,switchOn,...><...,open,enter,close,stayInLaboratory,open,switchOn,...><...,open,enter,close,stayInLaboratory,open,leave,switchOn,...<...,switchOn,open,...><...,switchOn,laserActive,open,...><...,switchOn,laserActive,switchOff,open,...>

(31)

Example1(continued):Hazardoustracesareformallyspecifiedusingtracespecifications

HAZARD(tr)≡∃u1,u2,u3:tr=u1⌢hopeni⌢u2⌢hopeni⌢u3⌢hclosei∧#(u1↾{open})mod2=0∧u2↾{open}=hi∧((u1↾{switchOn,laserActive,switchOff,laserPassive}6=hi∧last(u1↾{switchOn,laserActive,switchOff,laserPassive})6=laserPassive)∨(u2↾{switchOn,laserActive,switchOff}6=hi)∨(u3↾{switchOn,laserActive,switchOff}6=hi))

(32)

Example1(continued):Thedevelopmentobjectiveforthecontrolleristoensureforsystem

SYSTEM=EUC[|I|]CONTROLLER

withsomesuitableinterfaceIthesafetyspecification

SYSTEMsat¬HAZARD(tr)

(33)

Example1(continued):Asuitableinterfaceis

I={open,close,switchOn,laserPassive}

andasuitablecontrollercanbespecifiedas

CONTROLLER=open->C1[]switchOn->C2C1=close->open->close->CONTROLLERC2=laserPassive->CONTROLLER

(34)

Example1(continued):ThesafetyverificationistypicallyperformedbyusingawatchdogprocesswhichmonitorsalleventsfromthesystemalphabetAandinducesadeadlockassoonasasafetyviolationoccurs:

A={switchOn,laserActive,switchOff,laserPassive,open,close,enter,stayInLaboratory,leave}WATCHDOG=W(<>)W(tr)=if(\text{HAZARD}(tr))thenSTOPelse([]e:A@e->W(tr^<e>))

IfVERIFY=SYSTEM[|A|]WATCHDOGisfreeofdeadlocks,thisprovesthesafetyofSYSTEM

(35)

Example1–state-baseddescriptionwithspecificationintemporallogic:

State-TransitionSystemSTS=(S,s0 ,V,T):

•V:Setofvariablesymbols

•S:Setofstates,eachstates∈Savaluations:V→Dofvariables(Dtheassociatedvariabledomain)

•T⊆S×S:Thetransitionrelation

Run(execution)ofSTS:Statesequencehs0,s1,s2,...iwith

∀i≥0:(si ,si+1 )∈T

(36)

Example1–modelledasSTS:

•VariablesymbolsV={door,laser,dcnt}

•Auxiliaryvariabledcnt(“door-open/closedcounter”)countshowoftenthedoorhasbeenopenedorclosed

•DomainsD=D(door)∪D(laser)∪D(dcnt),D(door)={open,closed},D(laser)={passive,on,active,off},D(dcnt)=N0

•Valuationcanbewrittenlikes={door7→open,laser7→on,dcnt7→5}

(37)

Example1–modelledasSTS–transitionrelation:

T⊆S×S:withoutcontrol,Tallowsany(possiblyunsafe)transitions(s,s ′)satisfying 1.s0 (door)=closed∧s0 (dcnt)=0∧s0 (laser)=passive2.s(laser)=passive⇒s ′(laser)∈{passive,on}3.s(laser)=on⇒s ′(laser)∈{on,active}4.s(laser)=active⇒s ′(laser)∈{active,off}5.s(laser)=off⇒s ′(laser)∈{off,passive}

(38)

Example1–modelledasSTS–transitionrelation:

(s(door)=d∧s(dcnt)=n)⇒((s ′(door)=d∧s ′(dcnt)=n)∨(s ′(door)6=d∧s ′(dcnt)=n+1))

(39)

Example1–specificationofsaferunsusingtemporallogic:

RecalloperatorsofLinearTemporalLogic(LTL),definedonrunsr=hs0,s1,s2,...iofSTS:

•StateformulaspoverV:logicformulaswithfreevariablesinV,involving∃,∀,¬,∧,∨,⇒,⇐⇒Interpretation:pholdsinstates∈Sifitsvaluations(p)istrue.

Example:door=open⇒laser6=activeisinterpretedinstatesass(door)=open⇒s(laser)6=active

(40)

Example1–specificationofsaferunsusingtemporallogic–Temporaloperators:

•Globallyp:p(orGp)holdsforrunriff∀i≥0:si (p)

•Nextp:p(orXp)holdsinstatesiofrunriffsi+1(p)istrue

•Eventually(finally)p:♦p(orFp)holdsforrunriff∃i≥0:si (p)

•pUntilq:pUqholdsforrunriff

(∃i≥0:si (q))∧(∀j<i:sj (p))

(41)

Example1–physicalmodelspecificationusingtemporallogic:

Therestrictionsaboutthephysicalmodelcanbespecifiedasfollows:

1.s0(door)=closed∧s0(dcnt)=0∧s0(laser)=passive2.(laser=passive⇒(laser∈{passive,on}))3.(laser=on⇒(laser∈{on,active}))4.(laser=active⇒(laser∈{active,off}))5.(laser=off⇒(laser∈{off,passive}))

(42)

Example1–physicalmodelspecificationusingtemporallogic:

∀d∈{open,closed},n∈N0 :(door=d∧dcnt=n⇒((door=d∧dcnt=n)∨(door6=d∧dcnt=n+1)))

Observe:Thelaboratoryisemptyifandonlyifdcntmod4=0

(43)

Example1–hazardspecificationusingtemporallogic:

Hazardscanbecharacterisedbyformula

HAZARD≡♦((door=open∨dcntmod46=0)∧laser6=passive)

Innaturallanguage,thehazardformulaexpresses

•Wheneverthelaserisnotinpassivestate,itishazardousifthedoorisopen.

•Wheneverthelaserisnotinpassivestate,itishazardousifsomebodyisinsidethelaboratory(thoughthedoormaybeclosed).

(44)

Example1–safetyspecificationusingtemporallogic:

Theassociatedsafetyconditionisthenegationofthehazardcharacterisation:

¬HAZARD≡((door=open∨dcntmod46=0)⇒laser=passive)

(45)

Example1–state-baseddescription–Formalmodelforimplementations:

Time-DiscreteInput-OutputHybridSystems

TDIOHSH=(Loc,Init,V,I,O,Trans):

•Loc:Locations

•V:variablesymbols,I,O⊂V,I∩O=∅

•Guard:quantifier-freepredicatesoverV

•Init:Loc→Guard:initialisationconstraints

•Assignthesetofallpairs(~x:= ~t)with

(46)

–~x=(x1,x2,...)vectorofallvariablesinV−I – ~t=(t1 ,t2 ,...)∈T |V−I|

–TtermsoverV

•Trans⊆Loc×Guard×Assign×Loc:Transitions

(47)

Example1–I/O-safeTDIOHS:Programsadheretothefollowingprogrammingmodel:

•Processingisperformedinasequentialtaskoperatinginamainloopwiththefollowingprocessingphases:

–Inputphase:Inputsarereadandcopiedto(global,static,heaporstack)variables–thesevariablesarecalledprocessingvariables–Processingphase:Thecontroldecisionsarecomputed,operatingonprocessingvariablesonly–Outputphase:Theprocessingvariablescontainingpre-computedoutputvaluesarecopiedtotheircorrespondingoutputs

(48)

ModellingSafety-CriticalControllers

Example1–EUCandController:

EQUIPMENT UNDER CONTROL

DOOR SYSTEMLASER CONTROLLER doorOpen:bool door:{open,closed} laserUnlocked:bool laser:{passive,on,active,off}doorRequest:bool

doorOpen == true opened when unlocked via Door can only be press a request button. To open door, users USERlaserSwitch:{on,off}

(49)

Example1–ControllerCode:

1//Processingvariables(initialisedbyfalse/passive/closed)

2dop:bool;dx:{open,closed};lu:bool;l:{passive,on,active,off};

3dcnt:int;doorOld:bool;dr:bool;

4while(true){

5//Inputphase------------------------------------------------

6dx=door;l=laser;dr=doorRequest;

7//Processingphase-------------------------------------------

8if(dx!=doorOld){doorOld=dx;dcnt++}

9dop=dxor(dcnt%4!=0)or(drandl==passive);

10lu=notdop;

11//Outputphase-----------------------------------------------

12doorOpen=dop;laserUnlocked=lu;

13//Safetymonitor---------------------------------------------

14if((dcnt%4!=0ordoorOpenordoor==open)andlaser!=passive)

15EMERGENCY_SHUTDOWN();//Switchoffpowersupplytolaser

16}

(50)

Example1–ControllerCodeSafetyProof:

Firstproveauxiliarypropertythatdcntisupdatedasrequiredforthephysicalmodel,i.e.,accordingtoformula

∀d∈{open,closed},n∈N0:(door=d∧dcnt=n⇒((door=d∧dcnt=n)∨(door6=d∧dcnt=n+1)))(∗)

Thisprooffollowsfromtheprogrampropertywhichholdsatline8

∀d∈{open,closed},n∈N0:(door=d∧dcnt=n∧(door6=d⇔doorOld6=dx))

sotheincrementdcnt++inline8establishes(*).

(51)

Verificationobjective:Showthat¬HAZARDholdsattheendofeachoutputphase.

Proofreliesonphysicalpropertyofelectro-mechanicaldoor/lasersafetysystem:

(laser6=passive⇒laserUnlocked)(1)

Wewillprovethatprogramensures

(doorOpen⇒¬laserUnlocked)(2)(dcnt%46=0⇒¬laserUnlocked)(3)(door=open⇒doorOpen)(4)

Properties(1),(2),(3),(4)obviouslyimply¬HAZARD

(52)

ForI/O-safeTDIOHStheproofofapropertyφisequivalenttoshowingthatφisaninvariantoftheprogram’smainwhile-loop.Wethereforehavetoprovethatthefollowingpropertiesareinvariants:

doorOpen⇒¬laserUnlocked(2 ′)dcnt%46=0⇒¬laserUnlocked(3 ′)door=open⇒doorOpen(4 ′)

(53)

Proofisbasedonoperationalsemanticsofwhilelanguages.

Properties(2’),(3’),(4’)holdwheninitiallyenteringthewhileloopatline4,duetovariableinitialisations.

Nowsupposethat(2’),(3’),(4’)holdinline5(sℓ denotesthevariablestatebeforeexecutionoflineℓ):

s5 |=doorOpen⇒¬laserUnlockeds5 |=dcnt%46=0⇒¬laserUnlockeds5|=door=open⇒doorOpen Ithastobeshownthatthenthesepropertiesalsoholdatline13inprogramstates13 .