Server and Domain Isolation Using IPsec and Group Policy

Appendix A: Overview of IPsec Policy Concepts

Published: March 17, 2005 | Updated: July 24, 2006

This appendix provides a detailed overview of the terms, processes, and concepts of IPsec. It is designed to provide the prerequisite level of understanding for IPsec as described in the Server and Domain Isolation Using IPsec and Group Policy guide.

The content of this appendix was originally published as part of the "Using Microsoft Windows IPsec to Help Secure an Internal Corporate Network Server" white paper which was jointly written by Microsoft and Foundstone Strategic Security.

Additional information in the white paper describes the first model for using IPsec to secure network access to internal Microsoft® Windows® servers that process or store sensitive data. Although this extra information is not required to understand the Server and Domain Isolation Using IPsec and Group Policy guide, it is useful background information.

On This Page
IntroductionIntroduction
IPsec Policy FiltersIPsec Policy Filters
IKE Negotiation ProcessIKE Negotiation Process
Security MethodsSecurity Methods
IPsec Encapsulation Modes and Protocol Wire FormatsIPsec Encapsulation Modes and Protocol Wire Formats
IKE AuthenticationIKE Authentication
IKE Authentication Methods and Security Method Preference OrderIKE Authentication Methods and Security Method Preference Order
Security Negotiation OptionsSecurity Negotiation Options

Introduction

When you create an IPsec policy, you configure IPsec rules (which determine IPsec behavior) and settings (which apply regardless of the rules that are configured). After you configure an IPsec policy, you must assign it to a computer for the policy to be enforced. Although multiple IPsec policies can exist on a computer, only one IPsec policy can be assigned to a computer at a time.

An IPsec rule determines which types of traffic IPsec must examine; whether traffic is permitted, blocked, or security is negotiated; how to authenticate an IPsec peer; and other settings. When you configure an IPsec rule, you configure a filter list that includes one or more filters, a filter action, authentication methods, a connection type, and an IPsec encapsulation mode (transport mode or tunnel mode). An IPsec rule is typically configured for a specific purpose (for example, “Block all inbound traffic from the Internet to TCP port 135”).

Filters define the traffic that you want to inspect, similar to a firewall rule, with source and destination IP addresses, protocols, and port numbers, if applicable. A filter action defines the security requirements for the network traffic. You can configure a filter action to permit, block, or negotiate security (negotiate IPsec). If you configure a filter action to negotiate security, you must also configure key exchange security methods (and their preference order), whether to accept initial incoming unsecured traffic, whether to allow unsecured communication with computers that do not support IPsec, and whether to use perfect forward secrecy (PFS).

Key exchange settings and key exchange security methods determine the IPsec protocol wire formats (authentication header (AH) or encapsulated security payload (ESP)), encryption and hashing algorithms, key lifetimes, and other settings required to configure both the Internet Key Association (IKE) main mode and IPsec security associations (SA). An SA is the agreement of security settings associated with keying material.The SA created during the first IKE negotiation phase is known as the IKE main mode SA (also known as the ISAKMP main mode SA). The IKE main mode SA protects the IKE negotiation itself. The SAs created during the second IKE negotiation phase are known as the IPsec SAs (also known as IKE quick mode SAs because each IKE quick mode negotiation negotiates the IPsec SA for each direction). The IPsec SAs protect application traffic.

This section provides information about the following important IPsec policy concepts:

IKE negotiation process

IPsec policy filters

Security methods

IPsec protocol wire formats

IKE authentication

IKE authentication method and security method preference order

Security negotiation options  

For more information about IPsec policy concepts, see the Help and Support Center for Microsoft Windows Server™ 2003.

IPsec Policy Filters

Filters are the most important part of an IPsec policy. If you do not specify the proper filters in either client or server policies, or if the IP addresses change before the policy's filters are updated, security might not be provided. IPsec filters are inserted into the IP layer of the TCP/IP networking protocol stack on the computer so that they can examine (filter) all inbound or outbound IP packets. Except for a brief delay, which is required to negotiate a security relationship between two computers, IPsec is transparent to end-user applications and operating system services. Filters are associated with a corresponding filter action by the security rule in an IPsec policy. Windows IPsec supports both IPsec tunnel mode and IPsec transport mode as an option in the rule. IPsec tunnel mode rule configuration is very different from IPsec transport mode rule configuration.

The filtering rules associated with an IPsec policy are similar to firewall rules. By using the graphical user interface (GUI) provided by the IP Security Policy Management Microsoft Management Console (MMC) snap-in, you can configure IPsec to permit or block specific types of traffic based on source and destination address combinations and specific protocols and ports.

Note   Windows IPsec is not a full-featured, host-based firewall, and it does not support dynamic or stateful filtering features, such as tracking the established bit during the TCP handshake to control the direction in which communication can flow.

Understanding IPsec Filtering

Filter lists are simply listings of known subnets and infrastructure IP addresses. What is important to understand is how all of the filters contained in all of the rules will combine to provide the required inbound and outbound access controls. This section provides the most important details to understand about how IPsec filters affect packet processing.

The Windows Server 2003 IPsec Monitor MMC snap-in provides the most detailed view of the ordering of IPsec filters. When the IPsec service processes a set of IPsec policy rules, the filters are copied into two types in order to provide control of the two phases of the IKE negotiation:

1.

IKE main mode filters. These filters use only the source and destination address of the filters defined in IPsec policy to control IKE main mode. The IKE main mode-specific filters each have an IKE main mode negotiation policy associated that defines:

IKE main mode security methods defined for the IPsec policy under key exchange settings, such as Diffie Hellman master key strength and the encryption and integrity algorithms used to protect the IKE negotiation itself.

IKE main mode lifetimes and limits on the number of session keys generated from the same master key.

Authentication method(s).  

2.

IKE quick mode filters. These filters contain the full filter information about addresses, protocols, and ports. IKE quick mode negotiates this filter definition to determine what traffic can be secured inside an IPsec security association pair. Each specific filter has a corresponding weight and a set of security methods that define:

Options for AH or ESP encapsulation in transport or tunnel mode.

A list of encryption and integrity algorithms.

IPsec security association lifetimes in kilobytes and seconds.

Perfect forward secrecy security settings.  

The IKE quick mode-specific filters are the list of filters that are given to the IPsec driver to enforce. The IPsec driver matches all inbound and outbound IP traffic against these filters, in the order specified by the highest weight. The following section on the IKE negotiation process describes how IKE negotiates and manages IPsec security associations using these policy controls.

Filters defined in IPsec policy are considered "generic" filters because they may have to be interpreted by the IPsec service when the policy is applied. The IPsec service interprets all generic filters into specific filters at the time that the IPsec policy (or change) is being applied on the computer. Specific filters have a built-in algorithm for calculating the weight, or order, which is also referred to a how specific the filter is when selecting traffic. A higher weight value corresponds to a more specific filter. All of the specific filters are sorted according to their weight. The weight of the filter is evaluated first on IP address, then on protocols, and finally on ports that may be defined within the filter. This approach ensures that the order of rules in a policy, and the ordering of filters in each different filter list, have no effect on the filtering behavior enforced by the IPsec driver during packet processing. The packets are matched against the most specific filters first to minimize the time required to process each packet against the total set of filters. The filter action that corresponds to the most specific filter matching a packet is the only action taken for that packet. The most generic filter that can be defined would be one that matches any IP address, all protocols, and ports. For example, consider the following four filter definitions:

Any <-> Any, any protocol

Any <-> 192.168.1.0/24, any protocol

Any <-> 192.168.1.10/24, any protocol

Any <-> 192.168.1.10/24, TCP source port Any, destination port 25  

Any to Any filter is the most general filter possible to define. It is only supported by Windows Server 2003 and Windows XP Service Pack 2 (SP2). Typically, this filter is used with a block action to achieve a default behavior of "Deny All." If this filter is being used to block all traffic, the rest of the more specific filters could be considered exceptions to the first filter. For Windows 2000, the supported, most general filter is Any <-> subnet, any protocol, or Any <-> My IP address if subnets are not used.

All four filters would match inbound traffic from any IP address to 192.168.1.10 using TCP port 25 and the corresponding outbound responses from port 25. The fourth filter is the most specific because it specifies a destination IP address, protocol, and port number. When all four are being enforced by the IPsec driver, an inbound packet destined to TCP port 25 will only match the fourth and most specific filter. If a remote system sends TCP traffic to any port other than 25 to 192.168.1.10, the third filter is matched. Finally, if traffic is sent to any IP address in the 192.168.1.0 subnet with the exception of 192.168.1.10, the second filter will be the most specific for that traffic.

Potential Filter Design Issues

When defining filters, certain combinations of source and destination address options should not be used. As noted above, filters that specify Any IP Address to Any IP Address should be avoided for hosts running Windows 2000.

Generally, the more filters in a policy, the more performance impact to packet processing. This performance impact appears as degraded throughput, increased non-paged pool kernel memory utilization, and increased CPU utilization. The precise impact on performance is very difficult to estimate because it depends on the overall traffic volume, the amount of IPsec-secured traffic being processed, and the CPU loads on the computer. Therefore, performance testing of IPsec policy designs should be factored into the planning effort. The impact of a few hundred filters is not likely to be noticed except on very high throughput computers.

Windows 2000 does not have optimizations for handling large numbers of filters. The IPsec driver must scan the whole filter list sequentially to find a match.

In Windows XP and Windows Server 2003, many optimizations were made to speed up filter processing so that larger numbers of filters can be used in the IPsec policy. Filters of the form From <IP address> To <IP address> regardless of protocol or ports were optimized by using the Generic Packet Classifier (GPC) driver for extremely fast lookup. GPC can handle almost any number of these filters without throughput performance degradation. Therefore, large exemption lists using My IP address to <specific exemption IP address> are easily supported provided there is enough non-paged kernel memory available to hold the entire filter list. Filters that do not have a specific IP address for both source and destination cannot be optimized by GPC, which means that Any IP <-> specific IP (or subnet) filters will require sequential searching. But the implementation is improved over Windows 2000.

The use of My IP Address may be appropriate in many cases but may also cause problems for hosts with many IP addresses, such as a Web server hosting many virtual Web sites. It may also cause a delay in the availability of IPsec driver packet filtering if there are a large number of filters using My IP Address. The IPsec service processes them during service startup and when an address change event happens. The delay may cause a window of vulnerability or delays in connecting securely with IPsec. Again, performance testing should confirm acceptable impacts of a particular policy design.

My IP Address might be most appropriately used when permitting or denying traffic to a specific port or protocol. For example, in the IPsec policy design for Woodgrove Bank, My IP Address filters are used to create a more specific filter that permits Internet Control Message Protocol (ICMP) traffic to be sent and received in clear text among all computers.

If a mobile client in the organization is assigned a My IP Address <-> Any IP Address rule and is then placed on an external network, the mobile client may fail to communicate in that environment. If the client is allowed to Fall back to clear, then the client will experience three-second and greater delays connecting to every destination. If the destination replies with an IKE response, the IKE negotiation will fail because IKE will not be able to authenticate using domain trust (Kerberos). Clearly, if RFC 1918 private addresses are used as internal network subnets, mobile clients will be affected when they connect to hotels, home networks, and potentially other internal networks. If mobile clients experience connectivity problems, they may require Local Administrator rights to stop the IPsec service when connected to other networks. Consequently, a domain logon script may need to be used to check if the IPsec service is running when they connect to the internal network.

Windows 2000 was not originally designed to provide filtering for packets using multicast and broadcast addresses because this traffic could not be secured using IKE negotiation. Therefore, multicast and broadcast packet types were part of the original default exemptions that bypassed IPsec filters. See Microsoft Knowledge Base article 811832, "IPSec Default Exemptions Can Be Used to Bypass IPsec Protection in Some Scenarios" for an in-depth explanation of the security implications of default exemptions and the changes implemented by Service Pack 3 to remove some of them by default. The TCP/IP integration with IPsec in Windows XP and Windows Server 2003 was enhanced to filter all types of IP packets. However, because IKE cannot negotiate security for multicast and broadcast, only limited support for filtering is provided. See Knowledge Base article 810207, "IPSec default exemptions are removed in Windows Server 2003" for details on the removal of default exemptions and the degree of filtering support for multicast and broadcast traffic. Windows XP SP2 supports the same filtering capabilities as Windows Server 2003.

IKE Negotiation Process

The IKE protocol is designed to help securely establish a trust relationship between each computer, negotiate security options, and dynamically generate shared, secret cryptographic keying material. In order to ensure successful and secure communication, IKE performs a two-phase operation: Phase 1 (main mode) negotiation and Phase 2 (quick mode) negotiation. Confidentiality and authentication may be ensured during each phase by the use of encryption and authentication algorithms that are agreed upon by the two computers during security negotiations.

Main Mode Negotiation

During main mode negotiation, the two computers establish a secure, authenticated channel. First, the following IPsec policy parameters are negotiated: the encryption algorithm (DES or 3DES), the integrity algorithm (MD5 or SHA1), the Diffie-Hellman group to be used for the base keying material (Group 1, Group 2, or, in Windows Server 2003, Group 2048), and the authentication method (Kerberos version 5 protocol, public key certificate, or preshared key). After IPsec policy parameters are negotiated, the Diffie-Hellman exchange of public values is completed. The Diffie-Hellman algorithm is used to generate shared, symmetric, secret keys between computers. After the Diffie-Hellman exchange is complete, the IKE service on each computer generates the master key that is used to help protect authentication. The master key is used, with the negotiation algorithms and methods, to authenticate identities. The initiator of the communication then presents an offer for a potential SA to the responder. The responder sends either a reply accepting the offer or a reply with alternatives. The result of a successful IKE main mode negotiation is a main mode SA.

Quick Mode Negotiation

During quick mode negotiation, a pair of IPsec SAs is established to help protect application traffic, which can include packets sent over TCP, User Datagram Protocol (UDP), and other protocols. First, the following policy parameters are negotiated: the IPsec protocol wire format (AH or ESP), the hash algorithm for integrity and authentication (MD5 or SHA1), and the algorithm for encryption (DES or 3DES), if encryption is requested. During this time, a common agreement is reached regarding the type of IP packets to be carried in the IPsec SA pair that is established. After IPsec policy parameters are negotiated, session key material (cryptographic keys and key lifetimes, in seconds and kilobits, for each algorithm) is refreshed or exchanged.

Each IPsec SA is identified by a security parameter index (SPI), which is inserted into the IPsec header of each packet sent. One SPI identifies the inbound IPsec SA; the other SPI identifies the outbound IPsec SA.

IKE Main Mode SAs and IPsec SAs

Each time IPsec is used to help secure traffic, one IKE main mode SA and two IPsec SAs are established. In the example scenario, for IPsec-secured communications to occur between CORPCLI and CORPSRV, the following SAs are established:

CORPCLI [IP1] <-------- IKE main mode SA [IP1, IP2] -----> [IP2] CORPSRV

CORPCLI [IP1] ---------- IPsec SA [SPI=x] --------------------> [IP2] CORPSRV

CORPCLI [IP1] <-------- IPsec SA [SPI=y] ---------------------- [IP2] CORPSRV

Where:

IP1 is the IP address of CORPCLI.

IP2 is the IP address of CORPSRV.

x is the SPI that identifies the inbound IPsec SA for CORPSRV from CORPCLI.

y is the SPI that identifies the outbound IPsec SA for CORPSRV to CORPCLI.  

As this summary indicates, the IKE main mode SA between CORPCLI and CORPSRV is bidirectional. Either computer can initiate a quick mode negotiation by using the protection provided by the IKE main mode SA. IPsec SAs are not dependent on the state of upper-layer protocols. For example, TCP connections can be established and ended while IPsec SAs continue, and IPsec SAs can expire before a TCP connection ends. IKE attempts to renegotiate, by using the quick mode negotiation to establish two new IPsec SA pairs before the lifetime of the existing IPsec SA pair expires, to help prevent a connection from being disrupted. Although this process is commonly referred to as rekeying the IPsec SA, two new IPsec SAs are actually established. The life of the IKE main mode SA is measured only by time and the number of IPsec SAs that have been attempted (not by the number of bytes of data that is transferred in the IKE protocol). The IKE main mode SA expires independently of the IPsec SA pair. If a new IPsec SA pair is needed, an IKE main mode SA is automatically renegotiated as required (when a main mode SA has expired). By Internet Engineering Task Force (IETF) design, IKE must be able to rekey the main mode SA and negotiate IKE quick mode in either direction. Therefore, the authentication method that is configured in the IPsec policy on both computers for the IKE main mode SA should allow authentication to succeed in the direction from which the IKE main mode negotiation is initiated. Likewise, the IPsec policy settings in the filter action for quick mode should allow successful bidirectional quick mode negotiation.

Security Methods

Security methods are used during the IKE main mode negotiation to define the encryption and hashing algorithms and the Diffie-Hellman group that is used to create the main mode SA and to help secure the IKE negotiation channel. Security methods are also used during the quick mode negotiation to define the encapsulation mode (transport or tunnel), IPsec protocol wire format (AH or ESP), encryption and hashing algorithms, and key lifetimes that are used to create the quick mode inbound and outbound SAs.

IPsec Encapsulation Modes and Protocol Wire Formats

IPsec helps protect data in an IP packet by providing cryptographic protection of an IP payload. The protection that is provided depends on the mode in which IPsec is used and the protocol wire format. You can use IPsec in transport mode or tunnel mode.

IPsec Encapsulation Modes

IPsec tunnel mode is most commonly used to help protect site-to-site (also known as gateway-to-gateway or router-to-router) traffic between networks, such as site-to-site networking through the Internet. When IPsec tunnel mode is used, the sending gateway encapsulates the entire, original IP packet by creating a new IP packet that is then protected by one of the IPsec protocol wire formats (AH or ESP). For information about IPsec in tunnel mode, see the “Deploying IPsec” chapter in the Windows Server 2003 Deployment Kit.

IPsec transport mode is used to help protect host-to-host communications, and it is the default mode for Windows IPsec. When IPsec transport mode is used, IPsec encrypts only the IP payload; the IP header is not encrypted. Windows IPsec is used in transport mode primarily to help protect end-to-end communication (such as communications between clients and servers).

IPsec Protocol Wire Formats

IPsec supports two protocol wire formats: AH or ESP. IPsec transport mode encapsulates the original IP payload with an IPsec header (AH or ESP).

AH

AH provides data origin authentication, data integrity, and anti-replay protection for the entire packet (both the IP header and the data payload carried in the packet), except for the fields in the IP header that are allowed to change in transit. AH does not provide data confidentiality, which means that it does not encrypt the data. The data is readable but protected from modification and spoofing.

As shown in the following figure, integrity and authentication are provided by the placement of the AH header between the IP header and the TCP data.

Figure A.1  Authentication header in a packet

Figure A.1  Authentication header in a packet
See full-sized image

To use AH, in the properties for the appropriate rule, in the Custom Security Method Settings dialog box, select the Data and address integrity without encryption (AH) check box, and then specify the integrity algorithm to use.

ESP

ESP provides data origin authentication, data integrity, anti-replay protection, and the option of confidentiality for the IP payload only. ESP in transport mode does not protect the entire packet with a cryptographic checksum. The IP header is not protected. As shown in the following figure, the ESP header is placed before the TCP data, and an ESP trailer and ESP authentication trailer are placed after the TCP data.

Figure A.2  ESP data in a packet

Figure A.2  ESP data in a packet
See full-sized image

To use ESP, in the properties for the appropriate rule, in the Custom Security Method Settings dialog box, select the Data integrity and encryption (ESP) check box, and then specify the integrity and encryption algorithms to use.

IKE Authentication

IKE uses mutual authentication between computers to establish trusted communications and requires the use of one of the following authentication methods: Kerberos version 5 protocol, a computer X.509 version 3 public key infrastructure (PKI) certificate, or a preshared key. The two communication endpoints must have at least one common authentication method, or communication fails.

IKE Authentication Process

During IKE negotiation, the IKE initiator proposes a list of authentication methods to the IKE responder. The responder uses the source IP address of the initiator to identify which filter controls the IKE negotiation. The authentication method list that corresponds to the filter in the responder’s IPsec policy is used to select one authentication method from the initiator’s list. The responder then replies to inform the initiator of the agreed-upon authentication method. If the selected authentication method fails, IKE does not provide a method for trying a different authentication method. If authentication is successful and the main mode negotiation is successfully completed, the main mode SA lasts for eight hours. If data is still being transmitted at the end of eight hours, then the main mode SA is renegotiated automatically.

IKE Authentication Methods

It is important to choose the authentication method that is appropriate for your IPsec policy. An IPsec policy rule associates each IP address in a filter with an authentication method list so that IKE can determine which authentication method list to use with each IP address.

Kerberos Version 5 Protocol Authentication

The Kerberos version 5 protocol is the default authentication standard in Windows 2000 and Windows Server 2003 Active Directory domains. Any computer in the domain or in a trusted domain can use this method of authentication.

When Kerberos authentication is used, during main mode negotiation each IPsec peer sends its computer identity in unencrypted format to the other peer. The computer identity is unencrypted until encryption of the entire identity payload takes place during the authentication phase of the main mode negotiation. An attacker can send an IKE packet that causes the responding IPsec peer to expose its computer identity and domain membership. For this reason, to help secure computers that are connected to the Internet, certificate authentication is recommended.

By default, in Windows 2000 through Service Pack 3 and in Windows XP, Kerberos protocol traffic is exempt from IPsec filtering. To remove the exemption for Kerberos protocol traffic, you must modify the registry and then add an appropriate IPsec filter to help secure this traffic. For information about the default exemptions in Windows 2000 and Windows Server 2003, consult Special IPsec considerations.

Public Key Certificate Authentication

In Windows 2000 Server, you can use Certificate Services to automatically manage computer certificates for IPsec throughout the certificate lifecycle. Certificate Services is integrated with Active Directory and Group Policy, and it simplifies certificate deployment by enabling certificate auto-enrollment and renewal and by providing several default certificate templates that are compatible with IPsec. To use certificates for IKE authentication, you define an ordered list of acceptable root certificate authorities (CAs) to use, not which specific certificate to use. Both computers must have a common root CA in their IPsec policy configuration, and clients must have an associated computer certificate.

During the certificate selection process, IKE performs a series of checks to help ensure that specific requirements are met for the computer certificate. For example, the computer certificate must have a public key length that is greater than 512 bits and use a Digital Signature key usage.

Note   Certificates obtained from Certificate Services with the advanced option set for Enable strong private key protection do not work for IKE authentication because you cannot enter the required personal identification number (PIN) to access the private key for a computer certificate during IKE negotiation.

Preshared Keys

If you are not using Kerberos authentication and do not have access to a CA, a preshared key can be used. For example, a stand-alone computer on a network might need to use a preshared key because neither Kerberos authentication (through the computer’s domain account) nor certificates from a CA can enable successful IKE authentication in some scenarios.

Important   Preshared keys are easily implemented but can be compromised if they are not used correctly. Microsoft does not recommend the use of preshared key authentication in Active Directory because the key value is not securely stored, and it is therefore difficult to keep secret. The preshared key value is stored in plaintext in an IPsec policy. Any member of the local Administrators group can view a local IPsec policy, and a local IPsec policy can be read by any system service with Local System user rights. By default, any authenticated user in the domain can view a preshared key if it is stored in an Active Directory-based IPsec policy. Additionally, if attackers can capture IKE negotiation packets, published methods can enable the attackers to discover preshared key values.
For more information, see "Authentication Vulnerabilities in IKE and Xauth with Weak Pre-Shared Secrets" authored by John Pliam of the Institute for Mathematics and its Applications.

Preshared key authentication is provided for interoperability purposes and compliance with RFC standards. If you must use preshared key authentication, use a 25-character or longer random key value and a different preshared key for each IP address pair. These practices result in different security rules for each destination and help ensure that a compromised preshared key compromises only those computers that share the key.

IPsec CRL Checking

If you use certificate-based authentication, you can also enable IPsec certificate revocation list (CRL) checking. By default in Windows 2000, IPsec CRLs are not automatically checked during IKE certificate authentication.

To enable IPsec CRL checking

Caution   Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.

1.

Under HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\PolicyAgent\, add a new Oakley key, with a DWORD entry named StrongCrlCheck.

2.

Assign this entry any value from 0 through 2, where:

1.

A value of 0 disables CRL checking (default for Windows 2000).

2.

A value of 1 causes CRL checking to be attempted and certificate validation to fail only if the certificate is revoked (default for Windows XP and Windows Server 2003). Other failures that are encountered during CRL checking (such as the revocation URL being unreachable) do not cause certificate validation to fail.

3.

A value of 2 enables strong CRL checking, which means that CRL checking is required and that certificate validation fails if any error is encountered during CRL processing. Set this registry value for enhanced security.  

3.

Do one of the following:

1.

Restart the computer.

2.

Stop, and then restart the IPsec service by running the net stop policyagent and net start policyagent commands at a command prompt.

Note   IPsec CRL checking does not guarantee that certificate validation fails immediately when a certificate is revoked. There is a delay between the time that the revoked certificate is placed on an updated and published CRL and the time when the computer that performs the IPsec CRL checking retrieves this CRL. The computer does not retrieve a new CRL until the current CRL has expired or until the next time the CRL is published. CRLs are cached in memory and in \Documents and Settings\UserName\Local Settings\Temporary Internet Files by CryptoAPI. Because CRLs persist across computer restarts, if a CRL cache problem occurs, restarting the computer does not resolve the problem.

IKE Authentication Methods and Security Method Preference Order

You can configure an IPsec rule to specify only one authentication method or one security method. Alternatively, you can specify a preferred list of authentication and security methods. Preference order applies to authentication methods and security methods so that you can specify each method from most preferred to least preferred. For example, you can specify that both the Kerberos version 5 protocol and public key certificate authentication are offered as authentication methods but give the Kerberos protocol the higher preference, as shown in the following figure.

Figure A.3  Authentication method preference order

Figure A.3  Authentication method preference order

If a client attempts to connect to CORPSRV but only accepts public key certificates for authentication, then CORPSRV uses this authentication method and continues establishing communication. IKE must succeed using the selected authentication method, or the communication is blocked. IKE does not attempt to use a different authentication method if the negotiation fails. The same principle applies to security methods, where, for example, ESP might be preferred over AH.

Security Negotiation Options

You can configure whether an IPsec policy allows Fall back to clear (fall back to unsecured communication), Inbound passthrough, and Session key PFS on the Security Methods tab, in the properties for a filter action. You can configure Master key PFS in the Key Exchange Settings dialog box, in the general properties for a rule.

Fall Back to Clear

When Fall back to clear is allowed, traffic is secured by IPsec when possible (if the computer at the other end of the connection supports IPsec with a complementary filter action and filter in its policy), but traffic can be sent unsecured if the peer does not have an IPsec policy to respond to the request for security negotiation. If the peer does not respond to the request for security negotiation within three seconds, an SA for plaintext traffic (a soft SA) is created. Soft SAs allow normal TCP/IP communication with no IPsec encapsulation to occur. Keep in mind that although IPsec might not secure such traffic, another application might help secure the traffic (for example, traffic might be secured by Lightweight Directory Access Protocol (LDAP) encryption or remote procedure call (RPC) authentication mechanisms). If the peer does respond within three seconds and the security negotiation fails, the communication that matches the corresponding filter is blocked.

Fall back to clear is a setting that allows interoperability with the following:

Computers running operating systems earlier than Windows 2000

Computers running Windows 2000 or later systems that do not have IPsec policy configured

Computers running non-Microsoft operating systems that do not support IPsec  

To enable or disable Fall back to clear, on the Security Methods tab, in the properties for a filter action, select or clear the Allow unsecured communication with non-IPsec-aware computers check box.

For client policy, you can either enable or disable this option. If you enable this option and the server does not respond to the client’s request to negotiate security, you can allow the client to Fall back to clear. If you clear this check box, and if the server does not respond to the client’s request to negotiate security, communication is blocked. In some cases, it is useful to allow Fall back to clear. However, IKE allows Fall back to clear only if there is no reply. For security reasons, Windows IPsec does not allow unsecured communication if the IKE negotiation fails or if an error is experienced during an IKE negotiation (after the reply), such as failure to authenticate or to reach agreement on security parameters.

For initial deployments, it is recommended that you select this check box so that the client can Fall back to clear and initial connectivity can be established when IPsec is disabled on the server.

Inbound Passthrough

When Inbound passthrough is allowed, normal inbound TCP/IP traffic (traffic that is not secured by IPsec, for example a TCP SYN packet) is accepted if it matches the inbound filter associated with the filter action. The upper-layer protocol response packet (for example, a TCP SYN ACK packet) matches the corresponding outbound filter and triggers a security negotiation. Two IPsec SAs are then negotiated, and the traffic is IPsec-secured in both directions. The Inbound passthrough option allows a server to use the default response rule to initiate the security negotiation to clients. When you enable the default response rule in the client IPsec policy, clients do not need to maintain a filter that contains the IP address of the server. If you do not enable the default response rule in the client IPsec policy, then you do not need to enable the Inbound passthrough option in the server IPsec policy. Additionally, you should never enable this option on computers connected to the Internet.

To enable or disable Inbound passthrough, on the Security Methods tab, in the properties for a filter action, select or clear the Allow unsecured communication, but always respond using IPsec check box.

Session Key and Master Key PFS

PFS is a mechanism that determines whether the existing keying material for a master key can be used to derive a new session key. PFS helps ensure that the compromise of a single key permits access only to data that is protected by PFS, not necessarily to the entire communication. To achieve this, PFS helps ensure that a key used to protect a transmission cannot be used to generate additional keys. Session key PFS can be used without a reauthentication and is less resource-intensive than master key PFS. When session key PFS is enabled, a new Diffie-Hellman key exchange is performed to generate new master key keying information.

If you enable session key PFS in a server policy, you must also enable it in the client policy. You can enable session key PFS by selecting the Use session key perfect forward secrecy (PFS) check box, in the Key Exchange Settings dialog box, in the general properties for a rule. Master key PFS requires a reauthentication and is resource-intensive. It requires a new main mode negotiation for every quick mode negotiation that occurs. You can configure master key PFS by selecting the Master key perfect forward secrecy (PFS) check box. If you enable master key PFS in a server policy, you do not need to enable it in the client policy. Because there is significant overhead in enabling this option, it is recommended that you enable session key PFS or master key PFS only in hostile environments where IPsec traffic might be exposed to sophisticated attackers who will try to compromise the strong cryptographic protection provided by IPsec.


**
**