Understanding the Network-Level Behavior of Spammers

1
Understanding the Network-Level Behavior of
Spammers
By Anirudh Ramachandran and Nick Feamster
Defense Team

• Mike Delahunty
• Bryan Lutz
• Kimberly Peng
• Kevin Kazmierski
• John Thykattil

2
Agenda

• Introduction
• Background and Related Work
• Data Collection
• Network-level Characteristics of Spammers
• Spam from Botnets
• Spam from Transient BGP Announcements
• Lessons from Better Spam Mitigation
• Conclusion

3
Introduction

• Spam
• Multiple emails sent to many recipients
• Unsolicited commercial messages
• Study based on network level behavior of spammers
• IP address ranges
• Spamming modes (route hijacking, bots, etc.)
• Temporal persistence of spamming hosts
• Characteristics of spamming botnets
• Much attention has been paid to studying the
  content of spam

4
Introduction Cont.

• Study posits that Network Level properties need
  to be investigated in order to determine creative
  ways to mitigate spam
• Paper analyzes network properties of spam that is
  observed at a large spam sinkhole
• BGP route advertisements
• Traces of command and control messages of a Bobax
  botnet
• Legitimate emails
• Surprising Conclusions
• Most spam comes from a small IP address space
  (but so does legitimate email)
• Most spam comes from Microsoft Windows hosts
  bots
• Small set of spammers use short-lived route
  announcements to remain untraceable

5
Background

• Methods and Mitigation
• Spamming Methods
• Direct Spamming via spam friendly ISPs or
  dial-up IPs
• Open Relays and Proxies mail serves that allow
  unauthenticated to relay email
• Botnets hijacked machines acting under the
  control of centralized botmaster
• BGP Spectrum Agility short-lived route
  announcements to the IP addresses from which they
  send spam hampers traceability
• Mitigation Techniques
• Filtering Content based and IP Blacklists

6
Related Work

• Related Work Previous Studies
• Packet traces to determine bandwidth bottlenecks
  from spam sources
• Project Honeypot
• Sink for email traffic and hands out trap email
  addresses to determine harvesting behavior and
  identity of spammers
• Time monitoring from harvesting to receipt of
  first spam message
• Countries where harvesting infrastructure is
  located
• Persistence of spam harvesters

7
Related Work Cont.

• Mitigation
• SpamAssassin Project reverse engineering via
  mail content analysis
• DNS blacklist 80 of IPs sending spam were in
  the blacklist
• Unusual Route Announcements
• Bogus Well-Known addresses
• Suggestions of short lived route announcements

8
Data Collection

• Reserve a sinkhole
• Registered domain with no legitimate email
  addresses
• Establish a DNS Mail Exchange record for it.
• All emails received by the server are spam
• Run metrics on incoming emails
• IP address of the relay also run a traceroute
• TPC fingerprint to get the source OS
• Results of DNS blacklist from 8 different
  blacklist servers

9
Data Collection Cont.

• Spam received per day at sinkhole (Aug. 2004
  Dec. 2005)

10
Data Collection Cont.

• Hijack the DNS server for the domain running a
  botnet
• Have botnet commands go to a known machine
  instead.
• Monitor the BGP update from the networks where
  the spams are received
• Collect logs from large email provider (40
  million mailboxes)
• Allows analysis of network characteristics for
  spam and non-spam

11
Data Analysis

• Study focuses on network level characteristics
• Distribution of spam across IP address space is
  similar to legitimate emails (although not exact)
• Spam over IP address range is not uniform
• 12 of all received spam comes from two
  Autonomous Systems (AS)
• 37 come from top 20 ASes.
• Offers insight into spam prevention
• Classifying spam by country China, Korea, US
  dominate
• Defense suggestion
• Correlate originating country with IP range to
  estimate probability of spam.

12
Cumulative Distribution Function (CDF) of Spam
and Legitimate Email
Greater probability of legitimate emails
Big increase in probability of received spam
13
Spam Persistence
85 of unique spammers send 10 emails or less
If this is true for all, whats the value in
filtering by a specific IP address?
14
Effectiveness of Blacklists

• About 80 of spam listed in at least one major
  blacklist

15
Effectiveness of Blacklists Cont.

• Most spam bots are detected by at least one
  DNSRBL
• Only 50 of spammers using transient BGP
  announcements detected by one DNSRBL

16
Spam from Botnets

• Circumstantial evidence suggests that most spam
  originates from bots
• Spamming hosts and Bobax drones have very similar
  distributions across IP address space
• Suggests that much spam received may be due to
  botnets such as Bobax

17
More on Bots

• Most individual bots send low volume of spam
  individually

18
Operating Systems Used by Spammers

• Used OS fingerprinting tool p0f in Mail Avenger
• Able to identify OS of 75 of hosts that sent
  spam
• Of this 75 identifiable segment, 95 run Windows
• Consistent with percentage of hosts on Internet
  that run Windows
• Only about 4 run other OS, but are responsible
  for 8 of received spam.
• This goes against common perception that most
  spam originates from Windows botnet drones

19
Spam from Transient BGP Announcements

• Some spammers briefly hijack large portions of IP
  address space (that do not belong to them), send
  spam, and withdraw routes immediately after
  spamming
• Not much known, not well defended against
• Very difficult to trace
• Allows spammer to evade DNSRBLs
• Used 10 or less of the time, as complementary
  spamming tactic

20
Lessons on Spam Mitigation

• Why should we use network-level information?
• Information is less malleable
• More constant than spam email contents, which
  content-based filters monitor
• Information is observable in the middle of the
  network
• Closer to the source of the spam than other
  techniques
• Will result in more effective spam filters
• When combined with other techniques
• Has potential to stop spam that other techniques
  miss

21
More Lessons

• Improves knowledge of host identity
• Bases detection techniques on aggregate behavior
• Protects against route hijacking
• BGP spectrum agility
• Other techniques do not
• Uses network-level properties to detect and
  filter

22
Conclusion

• Studying the network-level behavior of spammers
• Designing better spam filters with network-level
  filters
• Network-level behavior filters vs. content-based
  filters
• Should not replace content-based filters, but
  complement them

23
Questions?