IPSec Implementation

1
IPSec Implementation

• Kevin Fleming
• Department of Pathology
• University of Washington

2
Outline

• What is IPSec?
• Why do we want to use it?
• How does IPSec work?
• Case study UW Pathologys implementation

3
What is IPSec?

• IP Security is a set of protocols and standards
  to support the securing of data at the IP layer.
  IPSec is a framework, not an implementation.
• Supports authentication and encryption of
  traffic.
• Certifies the originator of the packet.
• Protects the data from interception and tampering
  while in transit.

4
Why do we want to use IPSec?

• Secure our network
• Transparent operation
• IPSec allows us to secure any IP based protocol
  transparent to the application.
• Support for legacy software which is inherently
  insecure (telnet,ftp,SMB).
• An alternative mechanism to implementing
  application level security such as using SSL.
• Widest industry support e.g. Cisco, Microsoft,
  Network Associates, CheckPoint Software, Bay
  Networks, etc.
• IETF standard Will be mandatory in IPv6

5
Why not use IPSec?

• Processor overhead to encrypt verify each
  packet can be great.
• Added complexity in network design.

6
IPSec Modes

• Transport Secures the payload part of the IP
  packet, leaves the IP header unsecured. Commonly
  use for securing traffic on a LAN.
• Tunnel Secures the entire IP packet and
  encapsulates it within a new IP packet. Commonly
  used for creating a VPN.

7
IPSec protocols AH protocol

• AH - Authentication Header
• Defined in RFC 1826
• Integrity Yes, including IP header
• Authentication Yes
• Non-repudiation Depends on cryptography
  algorithm.
• Encryption No
• Replay Protection Yes

Transport Packet layout
IP Header
AH Header
Payload (TCP, UDP, etc)
Tunnel Packet layout
IP Header
AH Header
Payload (TCP. UDP,etc)
IP Header
8
IPSec protocols ESP protocol

• ESP Encapsulating Security Payload
• Defined in RFC 1827
• Integrity Yes
• Authentication Depends on cryptography
  algorithm.
• Non-repudiation No
• Encryption Yes
• Replay Protection Yes

Transport Packet layout
IP Header
ESP Header
Payload (TCP, UDP, etc)
Tunnel Packet layout
IP Header
ESP Header
Payload (TCP. UDP,etc)
IP Header
Unencrypted
Encrypted
9
What protocol to use?

• Differences between AH and ESP
• ESP provides encryption, AH does not.
• AH provides integrity of the IP header, ESP does
  not.
• AH can provide non-repudiation. ESP does not.
• However, we dont have to choose since both
  protocols can be used in together.
• Why have two protocols?
• Some countries have strict laws on encryption.
  If you cant use encryption in those countries,
  AH still provides good security mechanisms. Two
  protocols ensures wide acceptance of IPSec on the
  Internet.

10
Security Associations

• One of the most important concepts in IPSec is
  called a Security Association (SA). Defined in
  RFC 1825.
• SAs are the combination of a given Security
  Parameter Index (SPI) and Destination Address.
• SAs are one way. A minimum of two SAs are
  required for a single IPSec connection.
• SAs contain parameters including
• Authentication algorithm and algorithm mode
• Encryption algorithm and algorithm mode
• Key(s) used with the authentication/encryption
  algorithm(s)
• Lifetime of the key
• Lifetime of the SA
• Source Address(es) of the SA
• Sensitivity level (ie Secret or Unclassified)

11
How IPSec works Phase 1

• Internet Key Exchange (IKE) is used to setup
  IPSec.
• IKE Phase 1
• Establishes a secure, authenticated channel
  between the two computers
• Authenticates and protects the identities of the
  peers
• Negotiates what SA policy to use
• Performs an authenticated shared secret keys
  exchange
• Sets up a secure tunnel for phase 2
• Two modes Main mode or Aggressive mode
• Main Mode IKE
• Negotiate algorithms hashes.
• Generate shared secret keys using a
  Diffie-Hillman exchange.
• Verification of Identities.
• Aggressive Mode IKE
• Squeezes all negotiation, key exchange, etc. into
  less packets.
• Advantage Less network traffic faster than
  main mode.
• Disadvantage Information exchanged before a
  secure channel is created. Vulnerable to
  sniffing.

12
How IPSec works Phase 2

• An AH or ESP packet is then sent using the agreed
  upon main SA during the IKE phase 1.
• IKE Phase 2
• Negotiates IPSec SA parameters
• Establishes IPSec security associations for
  specific connections (like FTP, telnet, etc)
• Renegotiates IPSec SAs periodically
• Optionally performs an additional Diffie-Hellman
  exchange

13
How IPSec works Communication

• Once Phase 2 has established an SA for a
  particular connection, all traffic on that
  connection is communicated using the SA.
• IKE Phase 1 exchange uses UDP Port 500.
• AH uses IP protocol 51.
• ESP uses IP protocol 50.

14
Case Study UW Pathology

• A network with 200 workstations, 100 thin clients
  and 30 servers used by 500 users.
• Spans Health Sciences Building, UWMC, Harborview,
  and Roosevelt clinics.
• Network is a Windows 2000 Active Directory domain
  running on top of an IP based Ethernet
  infrastructure.
• Network spans over 30 different IP subnets.
• A variety of networking infrastructure including
  10Mbps shared, 100Mbps switched, and 1Gbps
  switched.

15
Case Study Goals

• Goals
• Initially secure VNC, MS SQL traffic, and MS
  NetBIOS (SMB) traffic.
• Provide this security transparently. Could not
  do application level security (e.g. SSL) with
  these services.
• Provide another layer of protection for
  workstations.
• Integrate seamlessly into existing environment.
• Technologies used
• UW Pathology runs a Windows 2000 Active Directory
  infrastructure. Therefore Microsoft Windows
  IPSec implementation was chosen.

16
Case Study Design Concepts

• Make workstation rules as simple as possible for
  ease of implementation and to ensure transparent
  migration to IPSec on the network.
• Make server rules control what traffic should be
  secured. This allows a one server at a time
  migration to IPSec.
• Dont try to protect everything. Use IPSec as a
  tool to protect certain services.
• Authenticate systems against the AD domain.
• Centralized deployment by Group Policy.

17
Case Study General Design

• When using IPSec security
• Use ESP with 3DES and SHA-1.
• Use Kerberos authentication against AD.
• Accept unsecured communication, but always reply
  with IPSec.
• Generate new keys ever 100MB or 60 mins.
• Permit unencrypted traffic to WINS Domain
  Controller servers.

18
Case Study Workstation Policy

• Enable a Default IPSec response rule that allows
  negotiation of policies with the servers.
• Secure VNC NetBIOS traffic.

19
Case Study Server Policy

• Enable Default response.
• Define different policies based upon services
  that require protection.

20
Case Study IPSec Helpful hints

• Do not enable IPSec for the entire system
  (Microsofts built-in Secure Server policy).
  This is a bad idea because the system will not be
  able to access any systems not running IPSec such
  as DNS and Web.
• Every system on the AD domain must be able to
  logon via the network to other systems otherwise
  Kerberos authentication fails.
• Remember to always add the Default Response to
  computers, otherwise you will have to explicitly
  setup every type of IPSec policy it should
  respond to.
• Remember to open up UDP Port 88 500, TCP Port
  88, and IP protocols 50 51 through any
  firewalls.
• Do not use IPSec on Domain Controllers. Only
  setup Domain Controllers to have the Default
  Response.

21
Case Study IPSec Windows Tools

• Netdiag on the W2K resource kit shows a lot of
  information about IPSec policies on the system.
  Run netdiag /testipsec /debug
• IPSecmon.exe on W2K shows active SAs.
• On WinXP, use the MMC IP Security Monitor
• Enable Debug logging
• HKEY_LOCAL_MACHINE\System\CurrentControlSet\Servic
  es\IPSEC\DiagnosticModeDWORD1
• HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic
  es\PolicyAgent\Oakley\EnableLoggingDWORD1

22
Case Study Conclusions

• IPSec successfully protected Pathology systems
  from RPC DCOM attacks.
• No noticeable slowdowns although system processor
  usage increased. NIC cards that offload
  encryption tasks greatly help.
• Microsofts IPSec implementation is buggy.
• SA synchronization issue (?). Systems do not
  negotiate for appx. 30 minutes.

23
References

• http//www.forsitesolutions.com/Techstuff/freeswan
  /ipsec_overview.html
• http//www.informit.com/content/index.asp?product_
  id7BC7E9CB3C2D29952D4EC62D9CCF2DE0D8823516AA
  7D
• Microsoft Help files White papers
• RFCs 1825,1826,1827