Case Study 2: EyePassShapes

study, the setup should have involved a task where the authentication has to be stopped and con-tinued from the previous point. The original study already provided indications that this is rather difficult.

Interestingly, some of the criteria were already implicitly applied during the original evaluation.

On the other hand, just like during the design phase, some important criteria were missing. Even though the data was partially available (e.g. data on the influence of missing consistency), it was originally not analyzed. Even applying the criteria afterwards, in this case study, shows how important they can be and what further insights they can generate.

Conclusion

The results of this case study show that with the help of the criteria, several of the issues that were discovered during the evaluation of VibraPass could have been identified already in an early design phase and therefore effort could have been saved. Additionally, the application of the criteria reveals open issues like the lack of an evaluation of resistance to distractions.

The remaining question is how to handle situations in which some of the criteria are violated.

Should VibraPass have been rejected during design and never been implemented? There are several indications that rejection would have been a mistake. Firstly, even though some of the criteria were partially or completely violated during the design phase, performance and security were rated highly positive. Most importantly, however, instead of rejecting the concept, it could have been improved easily with respect to the criteria. The basic concept of VibraPass allows the conclusion that it is not resistant to distractions. As for any iterative design process, this insight could have been used to add functionality to make it fulfill this criterion, for instance by adding some kind of lie history to the mobile device or more securely by repeating the vibration after a specific timeout. Using the second approach, users would know that if the vibration repeats, the next turn is a lie and if it does not repeat within say three seconds, the current digit is not supposed to be a lie. This means that the criteria can also be counted as valid tools that support iterative software development approaches.

performance, usability as well as security. It has to be noted that the accessibility of the system is limited to people with normal seeing ability and motoric skills.

As for VibraPass, the criteria did not exist in their current form when designing, implementing and evaluating EyePassShapes. Therefore, they were not explicitly considered. However, there were already many lessons learned from earlier designs. In this case study, we will take special care on analyzing whether the good results of the system cope with the application of the criteria.

That is, whether the system implicitly fulfilled the criteria and whether the good results can be attributed to this fact.

Design Phase

The theoretical security evaluation of the original design phase of EyePassShapes was carried out as proposed in criterion 1. A thorough theoretical security analysis was conducted and high security was attested to the system. The main weakness of EyePassShapes is advanced skimming attacks based on multiple cameras that are very unlikely in a public setting.

The influence of criterion 2 on the design process is defined by honestly judging the possible performance in sense of authentication speed. In contrast to VibraPass, in the design phase of EyePassShapes, this aspect has been fully adhered to. In fact, one of the advantages of the concept was the lack of a calibration process. This makes it faster since such a process would have to be counted as part of the authentication attempt if eye tracking itself is not required for the interaction with the public terminal. Using PassShapes as very fast authentication tokens supported this approach. The only open question was whether the same performance and usability advantages apply for gaze interaction as well.

EyePassShapes provides strict outer and inner consistency. Even more than that, it uses an au-thentication token that has been designed to provide advanced memorability by using consistent stroke based shapes that have to be drawn in exactly the same order of strokes every time the user authenticates with the system. Randomization has been completely avoided as well. Therefore, again implicitly, the decisions made during the design phase fulfill the recommendations of cri-terion 3 and 4 since with PassShapes, an authentication token with proven positive memorability qualities has been chosen.

Criterion 5 states that security should not require an active user. Using gaze-based input makes the system resistant to manifold attacks which cannot be negatively influenced by the user by not using gaze input. However, during the evaluation of the system, a security advantage of EyePassShapes executed in several consecutive strokes could be attested. Thus, the level of security can be influenced by the users’ behavior. As opposed to VibraPass, it is not possible to use EyePassShapes in a way that simply gives away the authentication token, meaning that the security of EyePassShapes cannot be “turned off”. From a design phase point of view, this criterion has to be considered fulfilled.

For social compatibility, the situation is much simpler. To authenticate using EyePassShapes, the users have to use their eyes, secretly transmitting the information to the terminal. That is, any friends or other people near the user cannot see the authentication but at the same cannot blame

the user for this fact since there is no alternative to this behavior. This criterion is thus the next that has been implicitly applied during design process. This is also an example of how solving criterion 5 can implicitly solve criterion 6 as well.

Finally, the last criterion that is of importance during the design process is resistance to distrac-tions. That mainly means that it has to be possible to easily continue authentication once it has been interrupted due to distractions. One of the design decisions made for EyePassShapes was that the single strokes of the PassShape can be executed in arbitrary consecutive chunks. Even though the possibility is low that a user will be interrupted within the five seconds that it takes to authenticate in a single stroke, it is theoretically possible to stop authentication and continue from where it stopped. Based on the fact that the shape is always the same and no randomization is used, this is feasible to achieve by a user. That is, also the last criterion has been integrated into the design of EyePassShapes.

As mentioned before, performance and security of EyePassShapes are very promising. Interest-ingly, all the criteria of the design process were implicitly considered in its concept. We argue that this involuntary fact is responsible for the positive attributes of the system. This also shows that much of the positive and negative properties of an authentication mechanism are already decided during the design stage.

Implementation Phase

For measuring authentication speed, only the authentication phase and confirmation were recorded. Using gaze-based technology, it would have been rather easy to record preparation.

As opposed to most systems, no “tricks” would have been required but simple observing the gaze before the actual interaction. Consistency is already fully considered by the design and the implementation could only minimally influence it. A technical evaluation of different implemen-tation alternatives has been conducted for this purpose. Based on this, a background picture was selected, providing additional consistency.

Evaluation Phase

In all work on authentication mechanisms presented in this thesis, an appropriate and in-depth security evaluation was always one of the most important factors. Therefore, as demanded by criterion 1, thorough theoretical as well as practical security analyses and evaluations were con-ducted. The attacker was an expert on EyePassShapes who employed a worst-case scenario based on video recordings with the information on when the authentication began and when it ended.

This way, the difficult relationship between security and authentication speed could be revealed.

Thus, like for the design phase, criterion 1 was fulfilled for the evaluation phase as well.

Since recording the preparation phase was neither considered during design nor implementation, it was not recorded during the evaluation as well. Besides this, all times were precisely recorded and compared. To record preparation, the time from the moment the background picture was shown to the first button press could have been used. We assume that the time is similar to the preparation phase of standard PIN-entry as presented in chapter 4. It often consists of a mental

task in which authentication is performed or the PIN is recalled using a memory strategy. Without having this data, we cannot say this for sure and thus, criterion 2 was only partially implemented in the evaluation.

The EyePassShapes prototype provided full outer and inner consistency (criterion 3). This means that no negative effects due to its absence could be measured. Consistency played an important role of the long-term memorability evaluation of the system and we argue that it is one of the reasons for the good memorability attributes of EyePassShapes.

Criterion 4 states that an authentication mechanism should use a memorable authentication to-ken, for instance, a token that exploits the user’s muscle memory. This was already considered during the design phase and was the main reason why PassShapes were chosen for the system.

Additionally, it requires long-term evaluations of the system, in the best case with multiple au-thentication tokens. Within the evaluation of EyePassShapes, this was only partially fulfilled by applying a long-term study without using multiple tokens. In this study, the system was attested very good memorability properties.

As mentioned in the analysis of the design phase, the criterion that security should not require an active user was fulfilled. The evaluation of the system showed that security cannot be turned off by insecure behavior during the interaction. Nevertheless, an in-depth analysis of the data of the security evaluation revealed that the degree of security can be influenced by one basic decision: performing the authentication in one or in several consecutive strokes. While the latter approach was more secure, one-stroke interaction was faster. In both cases, the system was highly secure. This is a major difference to VibraPass for which the evaluation showed that security can basically be completely turned off by insecure behavior. Again, this criterion can be considered fulfilled for EyePassShapes.

The last criterion that has an influence on the evaluation of EyePassShapes is resistance to dis-tractions. As for VibraPass, this has not been considered during the evaluation. There was no extra condition in which this resistance was tested. Theoretically, a high resistance can be as-sumed but it has not been proven and thus this criterion has not been considered in the evaluation phase. In [44], the authors describe a possible solution for this problem. They used a system of surrounding screens to simulate a realistic ATM setting. Inserting explicit distractions in such a setup could be a good candidate to fulfill criterion 7.

Summarized, only criterion 7 was not included in the evaluation process of EyePassShapes at all.

In addition to that, criterion 2 was only partially evaluated since times for preparation were not recorded.

Conclusion

The most interesting outcome of this case study was on the very good performance of the system even though the criteria did not exist when we conducted the work on EyePassShapes. We could show that during the whole development process, most criteria were implicitly applied. We cannot attribute this to the specific development process we used since it did not noticeably differ from the one that was applied to VibraPass. EyePassShapes, as a concept, proved to be very

Figure 6.9: The criteria applied to the different development stages of VibraPass and Eye-PassShapes. The comparison shows that VibraPass only fulfills part of the criteria while EyePassShapes fulfills most of them. This result correlates with the overall ratings of the systems based on their original evaluation.

efficient and a promising candidate for an authentication mechanism for public spaces. That is, if the criteria would have existed at that point, the design phase would have already revealed its positive properties and would have given first indications that we were “on the right way”.

It was not a lucky accident that the criteria were applied but rather an effect of the good con-cept. Furthermore, several lessons learned from previous work positively influenced the design of EyePassShapes. We argue that this approach can work the other way round as well. If Eye-PassShapes is good since the criteria are passively fulfilled, an authentication mechanism can be improved and appropriately judged by actively assigning the criteria as well. This can be partially seen in case study 1, VibraPass (chapter 6.2.1), in which improvements to the concept based on the criteria were proposed.