Appendix A. Knowledge Elicitation Method to Develop Qualitative BN Model
Root Cause Analysis in Industrial Control Systems – Differentiation of Cyber Attacks and Technical Failures based on Contributory Factors and Test Results (or Observations)
Objectives: To identify contributory factors (or risk factors) and tests that could help to differentiate between (accidental) component failure and an (intentional) attack on the component of Industrial Control System (ICS).
The results of this questionnaire would be used as a basis to develop a Bayesian Network model-based decision support system that could help to distinguish between (accidental) component failure and an (intentional) attack on the component of ICS in the water management sector.
This study is a first-of-its-kind. We will keep you up to date about the results of this study.
Estimated Time: 25 minutes
Examples: The examples provided below would help to clarify the terminologies used in this questionnaire. The lung cancer example is from medical domain which is not directly related to the questionnaire. However, this could help to easily understand the terminologies and translate it into our domain of interest. Furthermore, the computer crash example is from security domain which is closely related to the questionnaire. In general, the contributory factor (or risk factor) increases the likelihood of a disease or problem as shown in Figure 1A and 2A. In addition, the test result (or observations) based on a test would help to diagnose a disease or problem after it occurred.
Figure 1A. Lung Cancer – Example
Figure 2A. Computer crash – Example Case Outline
This is a hypothetical floodgate primarily operated by Supervisory Control and Data Acquisition (SCADA) system. Figure 3A illustrates the physical layout of the floodgate and the view of operations centre.
Figure 3A. Physical Layout of the Hypothetical Floodgate
Contributory Factor: Smoking
Disease: Lung cancer
Test: X-ray
Test Result: Positive chest X-ray
Contributory Factor: USB ports enabled in your computer
Problem: Your computer crashes/restarts/shutdown due to an (intentional) attack
Test: Run a malware scan in your computer
Test Result: The malware scanner detects malware in your computer during the scan performed
Figure 4A. SCADA Architecture of the Hypothetical Floodgate
Note: The case outline is provided to get you started. If you think anything is missing in the case outline, you could make your own assumptions, and explicitly mention it in your response.
Questions
Please answer the following questions to the best of your ability.
Background Information
1. How many years of experience do you have working with Industrial Control Systems (ICS)?
__________________________________________________________________________
2. Which sector(s) do you work in?
☐Chemical
☐Defence
☐Energy
☐Financial
☐Nuclear
☐Transport
☐Water
☐Others, please specify: _______________________________________________________
3. Which community do you associate yourself with based on your experience?
☐Safety (dealing with accidental/non-malicious threats)
☐Security (dealing with intentional/malicious threats)
☐Both safety and security
☐Others, please specify: _______________________________________________________
Problem: The sensor sends incorrect water level measurements.
4. Which contributory factors would increase the likelihood of the problem due to (accidental) sensor failure?
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
5. Which contributory factors would increase the likelihood of the problem due to an (intentional) attack?
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
6. Which tests would you execute to distinguish between (accidental) sensor failure and an (intentional) attack on the sensor for the problem?
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
7. If you have listed more than 1 test for 6., please rank the tests in the order of importance with “first rank” being the most significant test that would provide more clarity on the difference between (accidental) sensor failure and an (intentional) attack on the sensor, and “last rank” being the least significant test that would provide less clarity on the difference between (accidental) sensor failure and an (intentional) attack on the sensor.
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
Miscellaneous
8. In addition to contributory factors and test results, what are other elements that you would take into account when you diagnose an (intentional) attack on a component?
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
9. In addition to contributory factors and test results what are other elements that you would take into account when you diagnose (accidental) component failure?
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
10. What are the important elements that need to be included when you document an (intentional) cyber-attack?
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
11. What are the important elements that need to be included when you document an (accidental) technical failure?
________________________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________________________________
Appendix B. Knowledge Elicitation Method to Develop Quantitative BN Model
Probability Elicitation for the Bayesian Network (BN) Model to Distinguish Between Intentional Attacks and Accidental Technical Failures in Industrial Control Systems (ICS) based Floodgate
Objective: To elicit probabilities corresponding to each variable in our BN model that could help to determine the major cause (intentional attack or accidental technical failure) of the problem (sensor sends incorrect water level measurements) when observed.
The results of this questionnaire would be used to complete a BN model-based decision support system for Rijkswaterstaat to determine the major cause of the problem when observed.
This study is a first-of-its-kind. We will keep you up to date about the results of this study.
Estimated Time: 40 minutes Case Outline
Note: The case outline is provided to get you started and not completely depend on this for answering the questions.
This is a hypothetical floodgate primarily operated by Supervisory Control and Data Acquisition (SCADA) system. Figure 1B schematises a floodgate being primarily operated by SCADA system along with an operation centre.
Figure 2B illustrates the SCADA architecture of the floodgate. The sensor, which is located near the floodgate, is used to measure the water level. There is also a water level scale which is visible to the operator from the operations centre. The sensor measurements are then sent to the PLC. If the water level reaches the higher limit, PLC would send an alarm notification to the operator through the Human-Machine Interface (HMI), and the operator would need to close the floodgate in this case. The HMI would also provide information such as the water level and the current state of the floodgate (open/close). The actuator opens/closes the floodgate.
Figure 1B. Physical Layout of the Floodgate
Figure 2B. SCADA Architecture of the Floodgate
BN Model: Please see below to know about the constructed qualitative BN model to determine the major cause (intentional attack or accidental technical failure) of the problem (sensor sends incorrect water level measurements) when observed. You will find the questions in next pages corresponding to each variable in our BN model.
Questions
Please answer the questions taking into account the type of floodgates that have the criticality rating as “very high” (on a 5-point scale: very low – low – medium – high – very high). Furthermore, please answer the questions by marking the suitable probability among 7 anchors ((almost) impossible (0) - Improbable (15) - Uncertain (25) - Fifty-fifty (50) - Expected (75) - Probable (85) - Certain (almost) (100)) directly or writing fine-grained probability (in the provided space) using the numerical and verbal anchors as a supporting aid.
Easy physical access to sensor Q1.1 How likely is it that the sensor is easily physically accessible to an unauthorized person in a floodgate
operated by ICS?
Easy physical access to communication cable Q2.1 How likely is it that the sensor communication cable is easily physically accessible to an unauthorized person
in a floodgate operated by ICS?
Sensor data integrity verification Q3.1 How likely is it that data integrity verification is performed for the sensor data in a floodgate operated by
ICS?
Sensor is connected to WIFI Q4.1 How likely is it that the sensor is connected to WIFI in a floodgate operated by ICS?
ICS and corporate networks are connected Q5.1 How likely is it that the ICS and corporate networks are connected in a floodgate operated by ICS?
Presence of software in sensor Q6.1 How likely is it that software is present in the sensor in a floodgate operated by ICS?
Sensor firmware update Q7.1 How likely is it that the sensor firmware is updated in a floodgate operated by ICS?
Maintenance of sensor Q8.1 How likely is it that the sensor is physically maintained in a floodgate operated by ICS?
Maintenance of communication cable Q9.1 How likely is it that the sensor communication cable is physically maintained in a floodgate operated by ICS?
Good maintenance process Q10.1 How likely is it that there is a good maintenance process for the sensor in a floodgate operated by ICS?
Use of Electro-Magnetic Interference (EMI) shielding technique Q11.1 How likely is it that EMI shielding technique is used for the sensor in a floodgate operated by ICS?
Location of sensor susceptible to severe weather Q12.1 How likely is it that location of sensor is susceptible to severe weather in a floodgate operated by ICS?
Location of sensor susceptible to physical contact of marine vessel Q13.1 How likely is it that location of sensor is susceptible to physical contact of marine vessel in a floodgate
operated by ICS?
Location of sensor susceptible to biological fouling Q14.1 How likely is it that location of sensor is susceptible to biological fouling in a floodgate operated by ICS?
Please answer the questions (Q15.1 – Q15.9) taking into account the threat level as “substantial” which denotes there is a real chance of an attack (on a 5-point scale: minimal – limited – significant – substantial – critical).
Furthermore, please answer the questions (Q15.2 – Q15.9) taking into account only the corresponding causal factor that is present (Example (Q15.2): “How likely that the major cause for the observed problem is intentional attack given that the sensor/sensor communication cable is easily physically accessible to an unauthorised person?”) and assuming that other causal factors are absent. The double strikethrough text in the questions (Q15.1 – Q15.9) denotes the explicitly mentioned causal factors that are absent (Example: the sensor/sensor communication cable is not easily physically accessible to an unauthorised person). Finally, please answer the question (Q15.1) taking into account the causal factors that are not explicitly mentioned (if any) as the explicitly mentioned causal factors are absent.
Major cause for sensor sends incorrect water level measurements Q15.1 How likely that the major cause for the observed problem (sensor sends incorrect water level
measurements) is intentional attack given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.2 How likely that the major cause for the observed problem is intentional attack given that the sensor/sensor communication cable is easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.3 How likely that the major cause for the observed problem is intentional attack given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is not performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.4 How likely that the major cause for the observed problem is intentional attack given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.5 How likely that the major cause for the observed problem is intentional attack given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.6 How likely that the major cause for the observed problem is intentional attack given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is not updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.7 How likely that the major cause for the observed problem is accidental technical failure given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is not physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.8 How likely that the major cause for the observed problem is accidental technical failure given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is not used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
Q15.9 How likely that the major cause for the observed problem is accidental technical failure given that the sensor/sensor communication cable is not easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is susceptible to external factor (severe weather/marine vessel/biological fouling)?
Please answer the questions (Q16.1 – Q24.2) taking into account the major cause (“Intentional attack” / “Accidental technical failure”) of the observed problem (“Sensor sends incorrect water level measurements”) is already known.
Test/redundant sensor also sends incorrect water level measurements Q16.1 How likely is it that the test/redundant sensor also send incorrect water level measurements in a floodgate
operated by ICS given that the major cause of the problem is intentional attack?
Q16.2 How likely is it that the test/redundant sensor also send incorrect water level measurements in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
Sudden change of water level measurements from sensor Q17.1 How likely is it that there is a sudden change of water level measurements from sensor in a floodgate
operated by ICS given that the major cause of the problem is intentional attack?
Q17.2 How likely is it that there is a sudden change of water level measurements from sensor in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
Suspicious traffic in ICS network Q18.1 How likely is it that there is suspicious traffic in ICS network in a floodgate operated by ICS given that the
major cause of the problem is intentional attack?
Q18.2 How likely is it that there is suspicious traffic in ICS network in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
Abnormalities in other components Q19.1 How likely is it that there are abnormalities in other the components in a floodgate operated by ICS given
that the major cause of the problem is intentional attack?
Q19.2 How likely is it that there are abnormalities in other the components in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
No power in sensor Q20.1 How likely is it that there is no power in the sensor in a floodgate operated by ICS given that the major
cause of the problem is intentional attack?
Q20.2 How likely is it that there is no power in the sensor in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
Communication cable deteriorated Q21.1 How likely is it that the sensor communication cable is deteriorated in a floodgate operated by ICS given
that the major cause of the problem is intentional attack?
Q21.2 How likely is it that the sensor communication cable is deteriorated in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
Sensor sends correct water level measurements after cleaning sensor Q22.1 How likely is it that the sensor sends correct water level measurements after cleaning the sensor in a
floodgate operated by ICS given that the major cause of the problem is intentional attack?
Q22.2 How likely is it that the sensor sends correct water level measurements after cleaning the sensor in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
Sensor sends correct water level measurements after recalibrating sensor Q23.1 How likely is it that the sensor sends correct water level measurements after recalibrating the sensor in a
floodgate operated by ICS given that the major cause of the problem is intentional attack?
Q23.2 How likely is it that the sensor sends correct water level measurements after recalibrating the sensor in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
EMI along cable Q24.1 How likely is it that there is EMI along the sensor communication cable in a floodgate operated by ICS given
that the major cause of the problem is intentional attack?
Q24.2 How likely is it that there is EMI along the sensor communication cable in a floodgate operated by ICS given that the major cause of the problem is accidental technical failure?
Background Information
We will keep the background information anonymised for academic publishing.
Q25. Please write your name and email address (Optional).
__________________________________________________________________________
Q26. How many years of experience do you have working with Industrial Control Systems?
__________________________________________________________________________
Q27. Which sector(s) do you work in?
☐Chemical
☐Defence
☐Energy
☐Financial
☐Nuclear
☐Transport
☐Water
☐Others, please specify: ______________________________________________________
Q28. Which community do you associate yourself with based on your experience?
☐Safety (dealing with unintentional/non-malicious threats)
☐Security (dealing with intentional/malicious threats)
☐Both safety and security
☐Others, please specify: _______________________________________________________
Online Questions: Examples
Q1.1 How likely is it that the sensor is easily physically accessible to an unauthorized person in a floodgate operated by ICS?
Q15.2 How likely that the major cause for the observed problem is intentional attack given that the sensor/sensor communication cable is easily physically accessible to an unauthorised person, data integrity verification is performed for the sensor data, the sensor is not easily accessible via network to an unauthorised person, software is not present in the sensor, the sensor firmware is always updated, the sensor/sensor communication cable is always physically maintained properly, EMI shielding technique is used for the sensor, location of the sensor is not susceptible to external factor (severe weather/marine vessel/biological fouling)?
