Securing Data with Internet Protocol Security (IPSec)

1
Securing Data with Internet Protocol Security
(IPSec)

• Designing IPSec Policies
• Planning IPSec Deployment

2
Designing IPSec Policies

• Making IPSec design decisions
• Describing IPSec communications
• Planning IPSec protocols
• Planning IPSec modes
• Designing IPSec filters
• Designing IPSec filter actions
• Designing IPSec encryption and integrity
  algorithms
• Designing IPSec authentication

3
IPSec Design Decisions

• Decide which IPSec protocols to use.
• Decide whether to implement IPSec transport mode
  or IPSec tunnel mode.
• Design IPSec filters that identify which packets
  to protect with IPSec.
• Determine which actions will take place if the
  packets meet the IPSec filter criteria.
• Determine which encryption levels will be used if
  packets meet the IPSec filter criteria.
• Design how computers using IPSec protection will
  authenticate each other.

4
Describing IPSec Communications

• IPSec implements encryption and authenticity at a
  lower level in the TCP/IP stack than do Secure
  Sockets Layer (SSL) and Transport Layer Security
  (TLS).
• An application does not have to be IPSec-aware.

5
The IPSec Process (Using a Telnet Protocol
Example)
6
Planning IPSec Protocols

• IPSec provides two protocols for protecting
  transmitted data.
• Authentication Headers (AH)
• Encapsulating Security Payloads (ESP)
• AH and ESP are separate protocols.
• Use AH and ESP individually or combined to
  provide both integrity and inspection protection.
  

7
Assessing AH

• Provides authentication, integrity, and
  anti-replay protection to transmitted data
• Does not protect transmitted data from being read
  
• Eliminates the possibility of the data being
  modified during transmission
• Supported only by Microsoft Windows 2000 clients
  in a Microsoft networking environment

8
IPSec AH Header Fields
9
Deploying AH

• Authenticates computers involved in data
  transmissions
• Provides integrity to the transmitted packets so
  an attacker cannot modify or replay the
  transmitted data
• Used to restrict communications to specific
  computers in a workgroup or project
• Ensures that mutual authentication takes place
  between the computers so that only authenticated
  computers can participate in communications
• Allows mutual authentication capabilities to
  protocols that do not support mutual
  authentication

10
Assessing ESP

• Provides encryption, authentication, integrity,
  and anti-replay services
• Encrypts the Transmission Control Protocol (TCP)
  or User Datagram Protocol (UDP) header and the
  application data included within an IP packet
• Does not include the original IP header unless
  IPSec tunnel mode is used

11
IPSec ESP Fields
12
Deploying ESP

• ESP is necessary when the application does not
  recognize application-layer security.
• The application does not have to support IPSec.
• The IPSec encryption and decryption process takes
  place at the IP/IPSec layer.
• The application is unaware that IPSec protection
  takes place.
• Only operating systems and network devices that
  support IPSec can apply ESP encryption.
• ESP provides digital signing of the transmitted
  data.

13
Application Is Encryption Unaware
14
AH and ESP Differences

• AH protects the entire packet.
• ESP protects only the TCP/UDP header and the data
  payload from inspection.
• To ensure complete packet protection, configure
  the security association (SA) to implement both
  IPSec AH and ESP protocols.

15
Allowing IPSec Traffic to Pass Through a Firewall

• To pass protected traffic, configure a firewall
  to allow connections to UDP port 500 and protocol
  ID 50 for ESP or protocol ID 51 for AH.
• IPSec using ESP may lead to a firewall losing the
  ability to inspect data as it is transmitted
  through the firewall.
• The firewall must not be performing Network
  Address Translation (NAT).
• IPSec packets cannot pass through a NAT.
• The fields protected by IPSec cannot be modified
  by NAT without invalidating the packets.

16
Making the Decision Using AH, ESP, or a
Combination of AH and ESP

• Use AH in the IPSec security design
• To protect the entire packet against modification
• To provide mutual authentication of both client
  and server
• To limit communication to authorized computers
  for a project
• Use ESP in the IPSec security design
• To protect the application payload from being
  observed during transmission
• To protect the TCP/UDP header and application
  data from modification during transmission
• Use both AH and ESP when encryption of
  transmitted data and protection of the entire
  packet against modification is required.
• Negotiate an SA that requires both AH and ESP to
  ensure total protection of transmitted data

17
Applying the Decision Applying AH and ESP for
Fabrikam

• For the data collection software
• Apply both AH and ESP protection to each packet
• Configure ESP to allow the data payload to be
  encrypted as it is transmitted from the client to
  the server
• For the network link to A. Datum Corporation
• Only use ESP to encrypt all data transmitted over
  the Internet between the two networks

18
IPSec Transport Mode
19
IPSec Tunnel Mode
20
AH Tunnel Mode Packet
21
ESP Tunnel Mode Packet
22
Making the Decision Using IPSec Transport Mode
or Tunnel Mode

• Use IPSec transport mode when
• Communications are taking place where inspection
  of transmitted data must be prevented
• NAT is not being performed on the packets as they
  are transmitted from the source computer to the
  destination computer
• Data must be encrypted over the entire path from
  the source computer to the destination computer
• The connection is between only two computers
• Use IPSec tunnel mode when
• Data must be protected when being transmitted
  over a public portion of the network
• Encryption can only take place between perimeter
  servers to avoid passing through a firewall or a
  perimeter server implementing NAT

23
Applying the Decision Using IPSec Transport Mode
at Fabrikam

• Fabrikam requires the use of IPSec transport mode
  for the data collection software.
• All data is being transmitted between the Windows
  2000based laptops and the server at the
  Washington office.
• The data must be encrypted as it passes across
  the network to ensure that no one can read it.
• The data must be signed to prove its
  authenticity.

24
Applying the Decision Using IPSec Tunnel Mode at
Fabrikam
25
Designing IPSec Filters

• Characteristics Used to Identify a Protocol
• Source address information
• Destination address information
• Protocol type
• Source port
• Destination port

26
Protecting Response Packets by Using IPSec

• Configure all defined IPSec filters as mirrored
  filters.
• A mirrored filter reverses the source and
  destination information so that response packets
  are protected by IPSec when they are sent back.
• Do not use mirrored rules when filters for IPSec
  tunnel mode are defined.
• Design separate filters to reflect the tunnel
  endpoint that is used at each end of the tunnel.

27
When IPSec Filters Are Not Required

• Whenever the Layer Two Tunneling Protocol (L2TP)
  is used to establish a virtual private network
  (VPN), IPSec filters do not have to be defined.
• Windows 2000 automatically enables IPSec ESP
  protection for the L2TP tunnel.

28
Determining IPSec Exclusions

• IP broadcast addresses
• Multicast addresses
• Resource ReSerVation Protocol (RSVP)
  (protocol ID 46)
• Kerberos
• Internet Key Exchange (IKE)

29
Making the Decision Defining IPSec Filters

• Only one IPSec policy can be assigned per
  computer.
• Define policies for computers, not for users.
• Define the protocol requirements so that explicit
  filters can be defined, and determine attributes
  for each required filter.
• IPSec encrypted traffic cannot be identified if
  it passes through a firewall.
• If multiple filters are defined, the most
  specific filters are evaluated first and the
  least specific filters are evaluated last.

30
Making the Decision Defining IPSec Filters
(Cont.)

• Always mirror defined packet filters when using
  IPSec transport mode.
• Define an IPSec filter for each direction when
  defining IPSec tunnel mode connections.

31
Applying the Decision Fabrikam WAN Configuration
32
Possible IPSec Filter Actions

• Permit
• Block
• Negotiate Security

33
Windows 2000 IPSec Filter Settings and New
Session Key Frequency

• Accept Unsecured Communication, But Always
  Respond Using IPSec
• Allow Unsecured Communication With
  Non-IPSec-Aware Computers
• Session Key Perfect Forward Secrecy

34
Making the Decision Defining IPSec Filter
Actions

• Block
• Permit
• Negotiate
• Enable Fallback To No Security
• Accept Unsecured Communication, But Always
  Respond Using IPSec
• Session Key Perfect Forward Secrecy

35
Applying the Decision Defining IPSec Filter
Actions for Fabrikam

• For the data collection software, set the filter
  action to Negotiate Security.
• To allow or disallow other protocols, define
  another filter that is set to be any protocol.
• The tunnel servers between Fabrikam's Washington
  office and the A. Datum Corporation office
  require two different IPSec filter actions.

36
Designing IPSec Encryption and Integrity
Algorithms

• Configure IPSec filter properties to specifically
  define which algorithms IPSec uses when
  negotiating security.
• Define separate algorithms for AH and
  ESP-protected data streams.

37
Custom Settings for IPSec Protection

• Can be used to define how IPSec protects
  transmitted data
• If AH protection is required
• Define Message Digest v5 (MD5) or Secure Hash
  Algorithm v1 (SHA1) as the integrity algorithm
• If ESP encryption is required
• Set the digital signing algorithm to be MD5 or
  SHA1
• Set the encryption algorithm to be Data
  Encryption Standard (DES) or Triple DES (3DES)

38
Multiple Algorithms for the Negotiate Security
Action

• Can be used to define desired IPSec protection
  while allowing less secure variations that are
  used only if negotiation fails for the
  higher-level encryption

39
New Key Generation

• Can define key generation based on the amount of
  data that is transmitted (in kilobytes) and the
  lifetime of the key (in seconds).
• Configuring these options can protect the key
  from compromise.

40
Making the Decision Planning Encryption and
Integrity Algorithms for an SA

• If configuring for multiple algorithm support,
  sort the algorithms from strongest to weakest.
• Include security methods only for the required
  algorithms.
• Use of strong encryption protocols requires the
  installation of the Windows 2000 High Encryption
  Pack.
• Modify the default key generation settings in
  higher-security networks.

41
Applying the Decision Planning Encryption and
Integrity Algorithms for Fabrikam

• Fabrikam will use ESP to protect their
  transmitted data, with authenticity required for
  the data payload but not for the entire packet.
• Assuming the Windows 2000 High Encryption Pack is
  not installed, provisions must be made to allow
  the clients to connect without it.

42
Designing IPSec Authentication

• Methods for authentication
• Kerberos
• Certificates
• Preshared keys

43
Making the Decision Planning IPSec
Authentication Protocols

• Use Kerberos authentication
• When all computers using IPSec are members of the
  same Active Directory directory service forest
• To minimize the amount of configuration involved
  in authenticating hosts, but still maintain
  security for authentication
• Use public key authentication
• When strong authentication is required between
  hosts not in the same forest
• When a common root Certification Authority (CA)
  exists for the two hosts using IPSec
• When each host has a valid machine certificate
  installed that can be used to authenticate the
  host
• To use L2TP/IPSec for a VPN solution

44
Making the Decision Planning IPSec
Authentication Protocols (Cont.)

• Use preshared keys
• When Kerberos or public key authentication cannot
  be used
• When testing a new IPSec filter, to ensure that
  authentication problems are not causing the SA's
  failure
• When establishing an IPSec SA between two hosts
  and the association will only be between the two
  hosts
• When the preshared key is set to be complex and
  access to the IPSec configuration interface is
  secured to prevent inspection of the preshared
  key established between the two hosts

45
Applying the Decision Planning IPSec
Authentication Protocols for Fabrikam

• For the data collection software, the easiest
  authentication method is Kerberos.
• For the tunnel servers between the two
  organizations, the most secure authentication
  method is public key.
• Ensures that the certificates for each tunnel
  server are recognized and trusted by the other
  organization

46
Planning IPSec Deployment

• Assessing the preconfigured IPSec policies
• Deploying IPSec policies in a workgroup
  environment
• Deploying IPSec policies in a domain environment
• Automatically deploying computer certificates
• Troubleshooting IPSec problems

47
Predefined IPSec Policies

• Secure Server (Require Security)
• Server (Request Security)
• Client (Respond Only)

48
Custom IPSec Policies

• Used when specific protocols must be excluded
  from default policies
• Created when modifications are required to the
  default policies

49
Restoring Default Policies

• Right-click the IPSec Policies On Local Machine
  or IPSec Policies On Active Directory console,
  and then click Restore Default Policies.
• This action will restore the default setting for
  all three default IPSec policies.

50
Making the Decision Deploying the Default IPSec
Policies

• Use the Secure Server (Require Security) policy
  when any of the following business requirements
  exist
• The highest level of security is required
• All traffic sent to the server must be protected
  by using IPSec
• Fallback to unprotected data transmissions is not
  desired
• Only Windows 2000based computers are required to
  connect to the server
• All servers that require the IPSec configuration
  are placed in the same organizational unit (OU)
  or OU structure

51
Making the Decision Deploying the Default IPSec
Policies (Cont.)

• Use the Server (Request Security) policy when any
  of the following business requirements exist
• All traffic sent to the server should be
  protected by using IPSec
• Fallback to unprotected data transmissions is
  supported for legacy clients
• The server must support a mix of Windows 2000 and
  nonWindows 2000 clients
• All servers that require the IPSec configuration
  are placed in the same OU or OU structure

52
Making the Decision Deploying the Default IPSec
Policies (Cont.)

• Use the Client (Respond Only) policy when any of
  the following business requirements exist
• The Windows 2000based computer should be enabled
  to use IPSec protection when requested by a
  server
• The client computer should not initiate IPSec
  protection
• All computers within an OU or OU structure are to
  be enabled for IPSec protection

53
Applying the Decision Default IPSec Policies for
Fabrikam

• Fabrikam requires custom IPSec policies to meet
  its security objectives.
• The data collection software could possibly use a
  default IPSec policy.
• If more than one laptop is used, assign the
  Client (Respond Only) IPSec policy.
• Modify the IPSec policy applied to the server
  hosting the data collection software to accept
  unsecured communication, but always respond using
  IPSec.

54
Deploying IPSec Policies in a Workgroup
Environment

• A workgroup environment cannot depend on Active
  Directory for the consistent application of IPSec
  policies.
• IPSec policies in a workgroup environment can
  only be configured by connecting to the local
  computer security settings.
• To achieve consistent IPSec configuration
• Export properly configured IPSec settings to an
  .ipsec export file
• Import the settings to all matching computers
• IPSec settings cannot be configured through
  security templates.

55
Making the Decision Deploying IPSec in a
Workgroup Environment

• Define the required IPSec policies at a test
  machine.
• Create a lab environment that emulates the
  production network.
• Export the IPSec policies to an .ipsec export
  file.
• Store the exported IPSec policies in a secure
  location.

56
Applying the Decision Deploying IPSec in a
Workgroup Environment at Fabrikam

• The two tunnel servers may not be members of the
  domain at Fabrikam or A. Datum Corporation.
• IPSec must be defined in the local computer
  policy for each tunnel server.
• Deploy the IPSec policy by manually configuring
  the IPSec policy at each tunnel server.

57
Deploying IPSec Policies in a Domain Environment

• Define IPSec policies for the site, domain, or
  OU.
• The use of Group Policy ensures that a computer's
  administrator cannot override the desired IPSec
  settings at the local computer.
• The settings inherited from Group Policy always
  supersede local policy settings.

58
Making the Decision Deploying IPSec in an Active
Directory Environment

• Place computer accounts with the same IPSec
  requirements into the same OU or OU structure.
• Know the processing order for Group Policies and
  local computer policies.
• Assign the default Client (Respond Only) policy
  to the Default Domain Policy.
• Assign the default Client (Respond Only) policy
  to a specific OU.
• A computer can have only a single IPSec policy
  assigned at any time.

59
Applying the Decision Deploying IPSec in a
Domain Environment at Fabrikam

• If Fabrikam deploys additional laptops
• The best strategy is to place all the Windows
  2000based laptops in a common OU.
• Define a Group Policy object that applies the
  custom IPSec policy.
• Two options for the Washington office
• Place the data collection server in a separate
  OU.
• Have the Group Policy object applied with a
  filter so that only the data collection server
  applies the Group Policy object.

60
Automatically Deploying Computer Certificates

• IPSec gives two computers entering into an SA the
  ability to authenticate with certificates.
• Only domain controllers (DCs) acquire
  certificates by default in a Windows 2000
  network.
• To use certificates for authentication
• Manually configure each computer with the
  necessary certificate
• Or enable automatic certificate enrollment

61
Automatic Certificate Enrollment

• Automatic certificate enrollment is configured
  within Group Policy objects.
• Apply the Group Policy object at the site,
  domain, or OU.
• A CA trusted by both computers in the SA must
  issue the certificates.

62
Certificate Templates Available for Enabling
IPSec

• IPSec
• This is a single-use certificate template.
• It allows only the computer associated with the
  certificate to use IPSec.
• Computer
• This is a multipurpose certificate template that
  can also be used for IPSec authentication.
• Assign the computer certificate template to
  non-domain controllers (DCs).
• DC
• This is a multipurpose certificate template that
  allows IPSec authentication.
• Assign the DC certificate template only to DCs.

63
Making the Decision Designing Certificate-Based
Authentication for IPSec

• Determine which certificate template to issue.
• Ensure that a CA is configured to issue the
  certificate template.
• Ensure that all required computers have the Read
  and Enroll permissions for the certificate
  template.
• Configure a Group Policy object to perform the
  automatic certificate request.
• Distribute certificates to all client computers
  requiring L2TP tunnel connectivity.

64
Applying the Decision Designing
Certificate-Based Authentication for IPSec at
Fabrikam

• If certificate-based authentication is used for
  the data collection software IPSec solution,
  configure automatic certificate requests.
• Apply Group Policy at the OU containing the
  laptops and at the OU containing the data
  collection server.
• For the laptops, define the autoenrollment
  certificate request to issue either IPSec or
  computer certificates.
• An existing CA must be configured to issue the
  IPSec certificates.

65
Troubleshooting Tools

• Ping
• IPSec Monitor (Ipsecmon.exe)
• Netdiag
• System Management Server (SMS) Network Monitor
• Oakley logs

66
Making the Decision Troubleshooting IPSec
Connection Problems
67
Applying the Decision Troubleshooting IPSec
Connection Problems at Fabrikam

• Configure the authentication mechanism to use a
  preshared key and see if the connection succeeds.
  
• If the authentication continues to fail, run the
  IPSec Monitor to see if an SA is established, and
  determine if any errors are occurring during the
  session.
• If no session is established, review the IPSec
  policy assigned to each computer.
• Run the System Management Server (SMS) Network
  Monitor to ensure that Internet Security
  Association and Key Management Protocol (ISAKMP)
  packets are being received at each of the tunnel
  servers.
• Enable the Oakley logs to record detailed
  information about the ISAKMP process.

68
Chapter Summary

• IPSec design decisions
• Describing IPSec communications
• Planning IPSec protocols
• Planning IPSec modes
• Designing IPSec filters
• Designing IPSec filter actions
• Designing IPSec encryption and integrity
  algorithms
• Designing IPSec authentication

69
Chapter Summary (Cont.)

• Assessing preconfigured IPSec policies
• Deploying IPSec policies in a workgroup
  environment
• Deploying IPSec policies in a domain environment
• Automatically deploying computer certificates
• Troubleshooting IPSec problems