Lessons Learned

Figure 6.9: The criteria applied to the different development stages of VibraPass and Eye-PassShapes. The comparison shows that VibraPass only fulfills part of the criteria while EyePassShapes fulfills most of them. This result correlates with the overall ratings of the systems based on their original evaluation.

efficient and a promising candidate for an authentication mechanism for public spaces. That is, if the criteria would have existed at that point, the design phase would have already revealed its positive properties and would have given first indications that we were “on the right way”.

It was not a lucky accident that the criteria were applied but rather an effect of the good con-cept. Furthermore, several lessons learned from previous work positively influenced the design of EyePassShapes. We argue that this approach can work the other way round as well. If Eye-PassShapes is good since the criteria are passively fulfilled, an authentication mechanism can be improved and appropriately judged by actively assigning the criteria as well. This can be partially seen in case study 1, VibraPass (chapter 6.2.1), in which improvements to the concept based on the criteria were proposed.

could be identified as a very good candidate. These findings from the original evaluations and especially the overall quality of the systems highly correlate with findings from the case studies about the degree in which the system fulfills the seven criteria in the different development stages.

An overview is depicted in figure 6.9. The view on this schematic overview reveals this fact.

Our experience throughout this whole work as well as the results from the case study analysis give strong support to what we claimed at the beginning of this thesis. The criteria provide use-ful help in the design process of authentication mechanisms. They help to judge concepts in a very early design stage and more importantly help to improve them to avoid design flaws that can lead to decreased security and usability. The criteria furthermore influence the evaluation of authentication mechanisms for public spaces and, once applied, make them more easily compa-rable (see again figure 6.9). That is, the criteria allow for a more critical view on the concept and performance of a system.

When evaluating an authentication mechanism with respect to a specific scenario, all criteria have validity and have to be applied. However, the criteria have to be differently assessed in different settings. As mentioned before, in a scenario, which requires a connected mobile device, connection times can be counted as part of the primary interaction, while in other scenarios, it is part of the authentication process. Another example is the evaluation of distractions. While in an ATM setting on a street, diverse distractions have to be considered, this might be different in a more closed environment.

From the case studies, we learned that violating one criterion or several criteria should not neces-sarily lead to overall rejection of a concept. In contrast, it should be used to improve the concept.

However, being unable to fulfill a criterion, even in several iterations, can be a good indicator that there is something wrong with the overall approach of the authentication mechanism. Taking into account the negative influences that not fulfilling specific criteria can have, this can be an important factor.

Therefore, we claim that one of the main lessons learned in this chapter are not that the criteria work as proposed. It is that they can be considered valid tools that support iterative software development approaches. That is, in each iterative step, the criteria are checked and based on the results, the concept, system, software has to be extended or improved. In the next iteration, this has to be checked upon again until the desired level of compliance is reached.

Chapter 7

Conclusion and Future Work

It’s not wise to violate rules until you know how to observe them.

– T. S. Eliot –

Secure and usable authentication is a very accessible and grateful sub-field of usable privacy and security. The community working on solutions is rather small and related literature is mainly limited to the last two decades. This means that proposing a solution or having and implementing an idea can be done in a very small fraction of time with a good chance to have the work published as “yet another solution”. This is one of the reasons why, as we claimed in this thesis, there is no common ground on which design and evaluation of authentication mechanisms can be based upon. Additionally, most systems are proposed by researchers as a one-time contribution to the field after which they go back to dealing with other usability or security problems.

At the beginning of this work, we identified different problems in the field. There is a big variety of proposed solutions for the authentication problem. However, they do have different disadvan-tages. Another problem is the lack of a common approach on how authentication mechanisms should be rated and how they should be evaluated. Evaluation approaches are so diverse that it is nearly impossible to compare different solutions in the literature to each other, making it hard to judge which one is better for a specific purpose.

Therefore, in this thesis, we created criteria based on analyses of diverse systems and a large amount of implementations and studies. We could show that fulfilling the criteria comes with benefits while violating them has to be paid with diminished usability and security.