Why Johnny Can

1
Why Johnny Cant EncryptA Usability Evaluation
of GPG 5.0

• Presented by Yin Shi

2
Overview

• Introduction
• Understanding the Problem
• Cognitive Walkthrough
• User Test
• Conclusion

3
Introduction

• Effective security requires a different usability
  standard
• Security mechanisms are effective only when used
  correctly
• Matt Bishop claimed that configuration errors are
  the cause of more than 90 of all computer
  security failures
• Making security usable will require the
  development of domain-specific user interface
  design principles and techniques
• Choose PGP 5.0 for our case study
• Designed by general consumer software standards
• Significantly improved graphical user interface
  makes complex mathematical cryptograph accessible
  for novice computer users.

4
Understanding the Problem

• Defining Usability for Security
• Definition Security software is usable if the
  people who are expected to use it
• Are reliably made aware of the security tasks
  they need to perform
• Are able to figure out how to successfully
  perform those tasks
• Dont make dangerous errors
• Are sufficiently comfortable with the interface
  to continue using it

5
Understanding the Problem

• Problematic Properties of Security
• Five inherent properties of security
• The unmotivated user property
• The abstraction property
• The lack of feedback property
• The barn door property
• The weakest link Property
• A Usability Standard for PGP
• Need for privacy and authentication
• What needs to be done
• How to do it and avoid dangerous errors

6
Evaluation Methods

• Two Methods
• An informal cognitive walkthrough
• A user test performed in a laboratory

7
Cognitive Walkthrough

• Visual Metaphors (keys)
• PGPs user interface relies on graphical
  depictions of keys and locks
• Improvements
• An extension of the metaphor to distinguish
  public keys for encryption and private keys for
  decryption
• Different icons for public and private keys

8
Cognitive Walkthrough

• Visual Metaphors (signatures)
• The icon of the blue quill pen is used to
  indicate signing is problematic
• Quill pen icon will not help user understand they
  need to use their private keys to generate
  signatures
• Improvements
• Keep quill pen to represent signing, but modify
  it to show a private key as the nib of the pen
• Use some entirely different icon for signatures

9
Cognitive Walkthrough

• Different Key Types
• Originally, PGP used the RSA algorithm for
  encryption and signing
• PGP 5.0 uses the Diffie-Hellman/DSS algorithm
• PGP 5.0 can handle RSA keys, but other version
  PGP cant handle DSS keys
• Lack of forward compatibility
• Recipients with RSA keys cant decrypt it
• Recipients with RSA keys cant verify signatures
• PGP 5.0 alerts its users to this compatibility
  issues in two ways

10
Cognitive Walkthrough

• Different Key Types
• Uses different icons to depict the different key
  types
• When user attempt to encrypt documents using
  mixed key types, a warning message is showed
• Improvement
• Double-clicking on a key pops up a Key properties
  window

11
Cognitive Walkthrough

• Metaphor of choosing people
• Human icons obscure the key type information
• Better to display multiple keys that person owns

12
Cognitive Walkthrough

• Key Server
• Are publicly accessible databases
• PGP offers three key server operations under the
  Keys pull-down menu

13
Cognitive Walkthrough

• Problems with the presentation of the Key Server
• Users may not realize that it exists
• No representation of it in the top level of
  PGPkeys display
• PGPkeys keeps no records of key server access
• PGPs key revocation operation does not send the
  resulting revocation certificate to the key server

14
Cognitive Walkthrough

• Key Management Policy
• Two ratings for each public key
• Validity how sure the user is that the key is
  safe to encrypt with
• Trust how much faith the user has in the key
• May not realize PGP can automatically sets the
  validity rating of a key based on whether it has
  been signed by a certain number of sufficiently
  trusted keys.

15
Cognitive Walkthrough

• Irreversible Actions
• Accidentally deleting the private key
• Accidentally publicizing a key
• Accidentally revoking a key
• Forgetting the passphrase
• Failing to back up the key rings
• Consistency
• encoding
• Too Much Information
• PGPkeys application presents the user with too
  much information to make sense of
• Owners name, validity, trust level, creation
  date, and size
• Nothing to help the user figure out which parts
  of the display are the most important to pay
  attention to

16
User Test

• Test Design
• Initial task is to send the secret message to the
  team members in a signed and encrypted email
• Main steps
• Generate a key pair, get the public keys
• Make their own public key available to team
  members
• Type the secret message into an emails
• Sign the email using private key, encrypt the
  email using the team members public keys

17
User Test

• One of the member had an RSA key
• Participant would encounter mixed key types
  warning message
• Each of the five campaign members was represented
  by a dummy email account and a key pair
• These were accessible to the test monitor through
  a network laptop
• The test monitor could send email to the
  participant from the appropriate dummy account

18
User Test (Results)

• Avoiding dangerous errors
• Three of them accidentally emailed the secret
  without encryption
• One forgot her passphrase
• Figuring out how to encrypt with any key
• One couldnt figure out how to encrypt at all
• A reconfiguration of PGP may required
• Another one kept sending unencrypted test
  messages, and finally succeeded after being
  prompted to use the PGP plug in buttons

19
User Test (Results)

• Figuring out the correct key to encrypt with
• 11 participants figured out how to encrypt, but
  failed to understand the public key model
• Another one so completely misunderstood the model
  that he generated key pairs for each team member
  rather than for himself
• Decrypting an email message
• Five participants received encrypted email
• One cant figure how to decrypt it
• Two took a very hard time to figure it out
• Other two were able to decrypt without any
  problem

20
User Test (Results)

• Publishing the public key
• Ten could make their public key available to the
  team members
• Two never addressed key distribution
• Those ten, five sent their keys to key server
• Three emailed to the team members
• Other two did both
• Getting other peoples public keys
• Eight successfully got the team members public
  keys
• The others either never seemed aware they need
  other peoples public key, or they did know how
  to get it

21
User Test (Results)

• Handing the mixed key types problem
• Only four managed to send encrypted email
  correctly
• One didnt have mixed key types problem
• The other three received a reply email for
  complaining that they couldnt decrypt email
• Signing an email message
• Verifying a signature on an email message
• Creating a backup revocation certificate
• Only three participants managed to successfully
  send encrypted email and decrypt a reply
• In response to direct prompting for backup
• One didnt send the key pair to the key server
• One sent email to the campaign manager
• One simply ignored the prompt

22
User Test (Results)

• Deciding whether to trust keys from the key
  server
• Of the eight participants, only three expressed
  some concern over if they should trust the keys
• None of the three made use of the validity and
  trust labeling provided by PGPKeys

23
Conclusion/Questions

• PGP 5.0s user interface does not come even
  reasonably close to achieving our usability
  standard
• It does not make public key encryption of
  electronic mail manageable for average computer
  users
• Public work on usability evaluation in a security
  context would be extremely valuable
• We expect to find better design strategies