Protecting People from Phishing: The Design and Evaluation of an Embedded Training Email System

1
Protecting People from Phishing The Design and
Evaluation of an Embedded Training Email System

• P. Kumaraguru, Y. Rhee, A. Acquisti,
• L. Cranor, J. Hong, E. Nunge

2
Phishing email
3
Phishing email
Subject eBay Urgent Notification From Billing
Department
4
Phishing email
We regret to inform you that you eBay account
could be suspended if you dont update your
account information.
5
Phishing email
https//signin.ebay.com/ws/eBayISAPI.dll?SignInsi
dverifyco_partnerid2sidteid0
6
Phishing website
7
What is phishing?

• Phishing is a broadly launched social
  engineering attack in which an electronic
  identity is misrepresented in an attempt to trick
  individuals into revealing personal credentials
  that can be used fraudulently against them.

Financial Services Technology Consortium.
Understanding and countering the phishing threat
A financial service industry perspective. 2005.
8
Phishing is growing

• 73 million US adults received more than 50
  phishing emails a year in 2005
• Gartner found approx. 30 users changed online
  banking behavior because of attacks like phishing
  in 2006
• Gartner predicted 2.8 billion loss in 2006

9
Why phishing is a hard problem?

• Semantic attacks take advantage of the way humans
  interact with computers
• Phishing is one type of semantic attack
• Phishers make use of the trust that users have on
  legitimate organizations

10
Counter measures for phishing

• Silently eliminating the threat
• Regulatory policy solutions
• Email filtering (SpamAssasin)
• Warning users about the threat
• Toolbars (SpoofGuard, TrustBar)
• Training users not to fall for attacks

11
Why user education is hard?

• Security is a secondary task (Whitten et al.)
• Users are not motivated to read privacy policies
  (Anton et al.)
• Reading existing online training materials
  creates concern among users (Anandpara et
  al.)

12
Our hypotheses

• Security notices are an ineffective medium for
  training users
• Users make better decision when trained by
  embedded methodology compared to security notices

13
Design constraints

• People dont proactively read the training
  materials on the web
• Organizations send security notices to train
  users and people dont read security notices
• People can learn from web-based training
  materials, if only we could get people to read
  them! (Kumaraguru, 2006)

P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor,
and J. Hong. Teaching Johnny Not to Fall for
Phish. Tech. rep., Cranegie Mellon University,
2007. http//www.cylab.cmu.edu/files/cmucylab07003
.pdf.
14
Embedded training

• We know people fall for phishing emails
• So make training available through the phishing
  emails
• Training materials are presented when the users
  actually fall for phishing emails

15
Embedded training example
Subject Revision to Your Amazon.com Information
16
Embedded training example
Subject Revision to Your Amazon.com Information
Please login and enter your information
http//www.amazon.com/exec/obidos/sign-in.html
17
Comic strip intervention
18
Design rationale

• What to show in the intervention?
• When to show the intervention?
• Analyzed instructions from most popular websites
• Paper and HTML prototypes, 7 users each
• Lessons learned
• Two designs
• Present the training materials when users click
  on the link

19
Comic strip intervention
20
Intervention 1 - Comic strip
21
Intervention 1 - Comic strip
22
Intervention 1 - Comic strip
23
Intervention 2 - Graphics and text
24
Study design

• Think aloud study
• Role play as Bobby Smith, 19 emails including 2
  interventions, and 4 phishing emails
• Three conditions security notices, text /
  graphics intervention, comic strip intervention
• 10 non-expert participants in each condition, 30
  total

25
Intervention 1 - Security notices
26
Intervention 2 - Graphics and text
27
Intervention 3 - Comic strip
28
Legitimate
Phish
Training
Spam
29
User study - results

• We treated clicking on link to be falling for
  phishing
• 93 of the users who clicked went ahead and gave
  personal information

30
User study - results
31
User study - results

• Significant difference between security notices
  and the comic strip group (p-value lt
  0.05)
• Significant difference between the comic and the
  text / graphics group (p-value
  lt 0.05)

32
Conclusion

• H1 Security notices are an ineffective medium
  for training users

Supported

• H2 Users make better decision when trained by
  embedded methodology compared to security notices

Supported
33
Latest comic strip design
34
Ongoing work

• Measuring knowledge retention and knowledge
  transfer
• Knowledge retention is the ability to apply the
  knowledge gained from one situation to another
  same or similar situation after a time period
• Knowledge transfer is the ability to transfer the
  knowledge gained from one situation to another
  situation after a time period
• Is falling for phishing necessary for training?

35
Coming up

• WWW 2007
• CANTINA A Content-Based Approach to Detecting
  Phishing Web Sites
• Learning to Detect Phishing Emails
• Our other research in anti-phishing
  http//cups.cs.cmu.edu/trust.php
• Symposium On Usable Privacy and Security (SOUPS),
  July 18 - 20, 2007 at Carnegie Mellon
  University

36
Acknowledgements

• Members of Supporting Trust Decision research
  group
• Members of CUPS lab

37
CMU Usable Privacy and Security
Laboratoryhttp//cups.cs.cmu.edu/