the background and its output is recorded, but no interaction with the driver or vehicle is allowed [68]. This kind of testing can only be carried out rather late during development [68], as functions and components must have approval for testing in traffic. Depending on the function in question, additional safety measures must be included to ensure safe testing [38]. One main deficit of testing in real traffic is that specific conditions or scenarios can hardly be triggered and cannot be reproduced easily [29, 38]. A systematic variation of conditions, such as in driving simulators, requires a high effort [38]. The true-positive reaction of safety systems cannot be tested at all, as testing must always be safe for every participant involved [29].
There are several techniques for the analysis of driving behavior in its natural environ-ment, i.e., real traffic [138]. They are summarized under the term Naturalistic Driving Study (NDS). An NDS is “[...] the observation of drivers in naturalistic settings (during their regular, everyday driving) in an unobtrusive way. The essential driver behavior is what is of interest in these studies, usually in relation to crashes” [138]. If a system is in-cluded in the observation, it is called Field Operational Test (FOT). An FOT can include quasi experimental methods and is focused on behavior in combination with a system in the field [138]. Examples for NDS are the 100-Car Naturalistic Driving Study [54] and The Second Strategic Highway Research Program (SHRP2) [15]. Some examples for FOTs are euroFOT [26] or the Integrated Vehicle-Based Safety Systems: Light-Vehicle Field Oper-ational Test (IVBSS) [167].
The consideration applying to testing in real traffic with regard to subjects also apply to NDS and FOT. The advantages of these observational methods in real traffic are that they provide the only way of discovering unexpected behavioral patterns, especially in combination with a safety system [138]. Over an extended observational interval, they provide a very reliable source of information on driver behavior [138] and also generate knowledge on traffic and environment. One crucial point is that these studies allow for an estimation of exposure in various forms, which is not feasible in the methods described above [42]. The downsides are that experimental control is extremely limited, that different methods have been applied in nearly all studies conducted so far, and that these studies require extremely high efforts and costs [138]. Although these methods are the only ones presented in this section which are capable of capturing “real-world effectiveness”, the information derived is in the context of this thesis rather used to derive models.
2.8 Summary and conclusion
The evaluation of safety functions, especially of active and integral safety, is not limited to true-positive system responses. The second section has elaborated on all possible system responses, classified in true positive (or negative) and false positive (or negative). The importance of these terms for system development regarding the operating point as well as system optimization and the implications for evaluation are discussed.
The review of existing methods of evaluation includes a general description of retrospec-tive and prospecretrospec-tive testing and data sources available. For new systems of acretrospec-tive safety a prospective approach seems nearly always to be most promising. However, the data sources available have deficits regarding the amount of data as well as depth of information and quality. Different methods of evaluation have been introduced; especially their advantages and challenges with respect to validity have been discussed.
Given the objective of prospective evaluation regarding the overall effectiveness of a measure of active safety representative for a traffic system, the methods discussed above – ranging from analysis of accident data bases to sophisticated case-by-case simulations – do not seem to be adequate. Mass simulations covering all relevant varieties (e.g., due to human behavior) as well as uncertainties in different situations offer an approach to meet this objective.
As mass simulation seems to be the method of choice for representative evaluation of the effectiveness of systems of active and integral safety, substantial modeling and input data are necessary. The last section has introduced different methods used to gain knowledge needed for implementation in such a simulation. Starting with methods focused on single components or subsystems, examples of different techniques and expected results have been given. The next level is the testing of whole systems, for example, on test tracks or in subject studies. Research on driver behavior is also a vital part, often conducted in driving simulators, on test tracks, or in real traffic. Exposure and long term studies can be conducted as FOT or NDS in real traffic and provide valuable input for modeling different parts of the driver, vehicle and environment system in a simulation or enable validation or even verification of different parts of a process model, especially for critical situations.
3 Approach to integrated safety
evaluation: preventive pedestrian protection
3.1 Process chain for quantitative evaluation of the pre-crash phase
The concept for the process chain for the evaluation of measures taken before a col-lision responds to the challenges in evaluating real-world safety benefits using methods as described in Chapter 2. The objectives and requirements of the process chain can be summarized as follows:
• The method should predict real-world safety benefits of measures applied in the pre-crash phase.
• The method should produce a quantitative and representative evaluation of real-world effectiveness.
• The method should be objective, reliable, valid, reasonable, economic, free of feed-back, safe, and privacy protecting [203].
• Possible, yet undesired aspects of a measure (such as false-positive actions) should also be part of the evaluation in order to predict the overall effect on safety as well as other impacts such as acceptance or efficiency.
The effectiveness of new measures should be quantitatively evaluated during the design and development phases, i.e., before market introduction, so approaches using retrospec-tive analysis (e.g., based on accident data) are not applicable. The method must therefore not only be valid in the sense that it is able to capture the desired effect, but also be valid in its structure, assumptions, and internal procedures in order to produce a realistic and meaningful result. Therefore, real-world effectiveness requires statistical representativity.
Classical methods such as subject studies in driving simulators lack this representativity considering a combination of different possible variations (e.g., subject sample, environ-mental conditions, etc.). The method of choice to fulfill these requirements is a simulation technique.
A stochastic simulation can fulfill the requirements of representativity, economy, safety, and privacy. Reliability and validity of the procedure have to be evaluated. The method itself has evidently no feedback on the subject under investigation. As no subjects are involved in the simulation, questions of ethics and reasonableness do not impose limitations.
Models Simulation Evaluation
D t Models Simulation Evaluation
Data
Figure 3.1: Process chain for the evaluation of the pre-crash phase: overview.
Stochastic simulation can also support evaluation of “undesired” system actions and their side effects. For an overall evaluation of safety effects, undesired system actions (i.e., false positives or false negatives) can reduce the safety benefit by not addressing relevant situations or in the worst case possibly provoking new hazardous situations induced by the system actions.
The general outline of the whole process is illustrated in Fig. 3.1. The basis for evaluation are data sources of various kinds as input for detailed modeling. Basically, a stochastic simulation generates virtual traffic including the vehicle with and without the measure in question as well as other participants, the relevant environmental and boundary conditions.
The results are evaluated regarding positive and negative safety effects of the measure.
Fig. 3.2 gives some details on the different steps of the process chain for the example of preventive pedestrian protection. Concerning data used, knowledge regarding the driver and pedestrian behavior (if not extractable from accident data) are taken from literature.
The vehicle and preventive pedestrian protection related aspects are also based on literature as well as corporate knowledge. The intention is to construct evidence-based models using well-established statistical information to the greatest extent possible. The experiments and methods described in Chapters 2 and 4 are intended to provide information necessary for developing the different models. In case specific parameters are unknown or for some reason cannot be investigated, sensitivity analyses are utilized to quantify the resulting uncertainties.
The modeling step contains data preparation and aggregation, as well as development and assessment of models. The first part of this step is the construction of reference scenarios (see Section 3.2). Another is an operationally defined model of the preventive system and its effects on the other participants in the traffic situation (see Section 3.3).
The implementation of the driver, the pedestrian, and the vehicle into an appropriate traffic model including boundary conditions is briefly described in Section 3.4 together with the simulation itself.
Thesimulation provides the software environment in order to process the input data and correctly manage the interaction of the included models (see Section 3.4). For example, each single scenario could be simulated with and without the measure in question or the whole virtual scenario, including a high number of individual situations, could be simulated one time with and one time without the measure. All relevant characteristics are recorded for the evaluation step. The main advantage of this procedure is the possibility of a realistic consideration of all relevant distributions (e.g., driver reaction times, vehicle responses, sensor reliability, etc.). This characteristic gives the simulation the attribute stochastic.
3.1 Process chain for quantitative evaluation of the pre-crash phase
Data sources:
Data sources:
• Accident data
• Naturalistic Driving Data (NDS)
• Field Operational Test data (FOT)
• Traffic data
• Experiment data (test track, driving simulator, observation, literature, …)
• Vehicle characteristics
• System characteristics
Models (exposure, processes):
• Reference scenarios
• Driver model
• Vehicle model
• Participant model (e.g., pedestrian)
• Traffic model
• System model
• Interaction effects
• Environment model
Simulation:
• Traffic / critical situations / accidents
• With / without measure
• Consideration of realistic distribution of relevant parameters
Evaluation:
• Crash metric
(both for accidents and non-accidents)
Figure 3.2: Process chain for the evaluation of the pre-crash phase: details.
The last step in the process chain is evaluation. The metric used consists of several steps assessing accident situations as well as non-accident situations on both a microscopic and a macroscopic level (see Section 3.5).
An important issue concerning the whole process chain is validation. An effective vali-dation strategy usually begins at the level of a sub-process model. For example particular behavioral aspects of participants include perception of the pedestrian by the driver or the distinct driver reaction stages. Composite processes, such as time for the driver to respond to a complex stimulus, can be simulated by combining these sub-process models and can be independently validated. On the global level, composite processes are again combined to produce outcomes of interest and also secondary data, which are subject to validation as well, e.g., rate of accidents, influence of a change in an environmental condition. As explained in Section 2 (p. 13), verification may be possible for some subprocesses and pro-cesses, but hardly for all. The data used for validation and verification can come from accident statistics, secondary data sources such as other comparable studies, literature or experiments.