The SpamFilter Project by Kritsmer Ilya

1
The SpamFilterProjectbyKritsmer Ilya Geller
AlexunderHai Vortman supervising

• Computer Networks Laboratory
• 2004 Spring Semester

2
Motivation

• Spam is an actual and ever growing problem for
  whole Internet community.
• More than 66 of mail traffic is currently the
  spam messages in according to last reports.
• Spam mail wastes an enormous amount of human
  hours and network resources.
• World economics losses are estimated at
• billions of dollars.

3
Solution

• Most modern spamfilters operates on client side.
• We propose to filter the mail traffic before it
  comes to a client.
• Hence, a mail server should control and filter
  the traffic.
• More information available, more power and
  abilities.

4
The goals

• Stop incoming and outgoing messages that are
  classified as spam.
• No additional actions, like informing the
  receiver or returning the message to the sender.
• Do not interfere a regular mail traffic among
  mail servers.
• Friendly user interface.

5
The instruments

• We've taken the Linux based machine to take the
  advantage of the open source.
• Since we deals with servers traffic, SMTP
  protocol will be used.
• C language is a native choose for implementation.
• GUI development environment to make the admins
  life easier .

6
With that , on with the show
7
Design

• Several options were checked .
• The first one To change sendmail server code,
  insert the SpamFilter as an integral part of it.

• The drawbacks
• To learn the complicated code.
• Modifying existing product not modular
• Solution for specific software not generic

Internet
LAN
Mail Server
The Spam Filter
8
The second way To implement SpamFilter as a
separate machine and catch the network traffic
and filter it in frontof the mail server, like
firewalls do.
Internet
This way has been chosen.
9
The advantages of a firewall-like spam filter

• Does not require understanding of sendmails
  complicated code.
• Independence of sendmails language and
  implementation.
• Hence, this filter is generic and can operate in
  front of any type of mail server.

10
Implementation

• Three basic steps
• The first to catch the mail traffic
• The second to analyze the content and take the
  decision.
• The third decision execution.
• All this with with flexible and friendly GUI.

11
The first step.

• We connect a mail server to bridge computer so as
  all the traffic will pass over the bridge like
  proxy server.

• We use the well-known firewall application
  ip-tables to catch the network traffic, incoming
  and outgoing

Internet
LAN
Bridge
Mail Server
The SpamFilter configures the firewall so it will
only catch mail traffic.
12
How will it work?

• IP tables puts all tcp port 25 packets on the
  queue.
• The SpamFilter pull the packets one by one and
  pass it to the analyzing module.
• After making decision, SpamFilter bring the
  packet back to the firewall with a verdict.
• IP tables executes the verdict.

13
The advantages

• Bridge - Transparent
• No complicated multithreading data structures
• No IP address one can not attack the spamfilter
• Saving resources all spam checks are made on
  the bridge, hence no additional resources are
  needed on the server.

14
The second step

• The SpamFilter analyzes arrived packets packet by
  packet.

• The SpamFilter analyzes From,To fields for
  forbidden addresses

SpamFilter

• The SpamFilter analyzes message data for
  forbidden content

• We dont save all the message, but make decision
  on the DATA packets of the message

• Some additional features

15
The advantages

• A single DATA packet the SpamFilter gets contains
  IP,TCP and SMTP Headers.
• It makes possible to block the spam message in
  efficient way.
• Avoids turning SpamFilter to the additional mail
  server.
• Makes the filtering easier while still remaining
  accurate.

16
The third step and The Problem

• It seems to be enough to drop the packet when
  identified as spam.
• But then we face The Problem.
• The Problem is that the sending server resends
  packets which are supposed to be lost.
• After dropping the packet, the SpamFilter gets it
  again and again many times.
• It increases network traffic dramatically.
• Interferes with the mail servers work.

17
The Problem Solution

• Not just drop the packet but also make the sender
  end the connection immediately.
• SMTP protocol error seems to be a good idea.
• The error should make the sender to stop sending
  the message and notify the receiver to end the
  connection.
• The error 550 Access denied was chosen.
• Is that the happy end? Not quite

18
Fighting the TCP

• To succeed, we must to make the sender believe
  that we are the receiving server.
• In other words we have to perform IP and TCP
  spoofing.
• The problem TCP sequence numbers.
• It is the well known issue many attacks were
  executed, many protections were invented.
• So what is the problem?

19
TCP sequence numbers

• TCP connection is established by three-way
  handshake.
• Both sides sets packets seq numbers to follow
  after the arriving packets
• These numbers help TCP to ensure reliable
  connection
• Guessing the sequence numbers allows to hijack
  the session.

20
The solution

• There is no need to guess the sequence number
  exactly, it is enough to guess a number which is
  inside a window. This window starts with the
  sequence number of the previous packet (not
  included) and ends with a number we cannot know.
• From the DATA packet, we extract the ack and
  sequence numbers.
• Now we need to reverse those numbers. The seq.
  number of the sender is the ack that the
  receiver should send. The ack number of the
  sender was the previous sequence number of the
  receiver.
• All we need to do now is to increase this number
  (ack of the sender) by 1, and we fit into the
  window.

21
The third step - final

• After identifying spam, SpamFilter drops the
  packet

Upon receiving the packets, the sender
understands that an error has happened. The
sender discards the mail and sends the receiver a
connection-ending packet.
SpamFilter
SpamFilter sends 3 packets to the sender- an Ack
packet, a packet with the error message and a
reset packet
I P T A B L E S
The Sender
The Receiver
Error 550,RST
The Bridge
22
And the result
23
The SpamFilter features

• Ability to block the entire mail traffic.
• Mail addresses black list works with RegEx.
• Permitted Mails with trusted addresses which
  will be passed without any checks.
• Detailed log table with date, time, address
  fields, action taken and the reason to the action
  for each mail.
• Ability to save the logs to the file.

24
The SpamFilter features (cont)

• Advanced filtering
• All the filtering is based on regular
  expressions.
• Hence, when user defines viagra as forbidden
  word, it also catches V I a G r a ,
  _v_i_a!g!r_a and etc.
• To decrease negative hits, forbidden expressions
  are available. Number of intermediate words is
  also the parameter.
• Defining Buy now! with 3 intermediate words as
  forbidden expression, one makes SpamFilter catch
  Buy this wonderful product now! and Buy it
  now!, but I dont want to buy the computer you
  told to me for now will be identified as a legal
  expression.

25
The SpamFilter features (cont)

• The Necklace feature
• Spammers used to send mails by special programs
  which produce a huge number of mails in a very
  short time.
• Hence such event can indicate that the special
  mail address is a spammer address.
• One can set a maximal mails number per time slot
  for a user. When the number exceeds, sender
  address is inserted to the addresses black list.

26
What we have studied?

• SMTP, TCP/IP protocol in a deep level.
• Packets constructing, IP spoofing.
• Firewalls working principles.
• Regular expressions programming.
• GUI programming.
• Real time working application creation.