Deploying authentication in the wild: Towards greater ecological validity in security usability studies

Pico is a token-based login method that claims to be simultaneously more usable and more secure than passwords. It does not ask users to remember any secrets, nor to type one-time passwords. We evaluate Pico's claim with two deployments and user studies, one on a web-based service and another within an organization. Our main aim is to collect actionable intelligence on how to improve the usability and deployability of Pico. In our first study we team up with an established website, Gyazo, to offer this alternative login mechanism to users intent on performing a real task of image sharing. From the lessons of this first study, we retarget Pico's focus from replacing web passwords to replacing desktop login passwords; and thus in our second study we engage with a government organization, Innovate UK, to offer employees the ability to lock and unlock their computer automatically based on proximity. We focus particularly on the ecological validity of the trials and we thereby gain valuable insights into the viability of Pico, not only through the actual responses from the participants but also through the many practical challenges we had to face and overcome. Reflecting on the bigger picture, from our experience we believe the security usability community would greatly benefit from pushing towards greater ecological validity in published work, despite the considerable difficulties and costs involved.