Virtual Private Networking with IPSec

The entire process of encapsulation, routing, and de-encapsulation is called tunneling. Tunneling hides, or encapsulates, the original packet inside a new packet. This new packet may have new addressing and routing information, which enables the new packet to travel through networks. When tunneling is combined with privacy, the original packet data (as well as the original source and destination) is not revealed to those listening to traffic in the network. The network could be any internetwork: a private intranet, or the Internet. Once the encapsulated packets reach their destination, the encapsulation header is removed and the original packet header is used to route the packet to its final destination.

The tunnel itself is the logical data path through which the encapsulated packets travel. To the original source and destination peer, the tunnel is usually transparent and appears as just another point-to-point connection in the network path. The peers are unaware of any routers, switches, proxy servers, or other security gateways between the tunnel’s beginning point and the tunnel’s end point. When tunneling is combined with privacy, it can be used to provide Virtual Private Networks (VPN).

In Windows 2000, two types of tunneling are provided that use IPSec:

  1. Layer 2 Tunneling Protocol (L2TP/IPSec), in which L2TP provides encapsulation and tunnel management for any type of network traffic and IPSec in transport mode provides the security for the L2TP tunnel packets.
  2. IPSec in tunnel mode, in which IPSec itself does the encapsulation for IP traffic only.

Before using either type of tunneling, a complete understanding of the functionality should be obtained. For more information, see "Virtual Private Networking and IPSec" in the Windows 2000 Resource Kit.

The encapsulated packets travel through the network inside the tunnel. (In this example, the network is the Internet.) The gateway may be an edge gateway which stands between the outside Internet world and the private network--a router, firewall, proxy server, or other security gateway. Also, two gateways may be used inside the private network to protect traffic across less trusted parts of the network.

L2TP and IPSec

IPSec and L2TP are combined to provide both tunneling and security for IP, IPX and other protocol packets across any IP network. IPSec can also perform tunneling without L2TP, but it is only recommended for interoperability, when one of the gateways does not support L2TP or PPTP.

L2TP encapsulates original packets inside a PPP frame, performing compression when possible and then inside a UDP-type packet assigned to port 1701. Since the UDP packet format is an IP packet, L2TP automatically uses IPSec to secure the tunnel, based on the security settings in the user configuration of the L2TP tunnel. The IPSec Internet Key Exchange (IKE) protocol negotiates security for the L2TP tunnel using certificate-based authentication by default. This authentication uses computer certificates, not user certificates, to verify that both source and destination computers trust each other. If IPSec transport security is successfully established, then L2TP negotiates the tunnel, including compression and user authentication options and performs access control based on the user identity. Thus, L2TP/IPSec is the easiest, most flexible, most interoperable and more secure tunneling option for both client remote access VPN and gateway-to-gateway VPN tunnels.

Configuration for L2TP/IPSec VPN remote access clients is performed using Network and Dial-up Connections. Configuration for the VPN remote access server and for gateway-to-gateway tunnels is performed using the Routing and Remote Access console.

The original packet header, shown here as the IP or IPX header, carries the original and ultimate source and destination addresses (addresses used on the private network), and the outer IP header, shown as New IP Header contains the source and destination addresses of the tunnel end points (addresses used in the public network). The L2TP header carries tunnel control information. The PPP header identifies the protocol of the original packet, for example IP or IPX. For more information on L2TP/IPSec, see Network and Dial-up Connections in Windows 2000 Server Help.

IPSec Tunneling

The primary reason for using IPSec tunnel mode is for interoperability with other routers, gateways, or end-systems which do not support L2TP/IPSec or PPTP VPN tunneling technology. IPSec tunnel mode is supported only in gateway-to-gateway tunneling scenarios and for certain server-to-server or server-to-gateway configurations as an advanced feature. The Windows 2000 Resource Kit chapter on IPSec describes these scenarios and configurations in more detail and should be understood before using IPSec tunnel mode. IPSec tunnel mode is not supported for client remote access VPN scenarios. L2TP/IPSec or PPTP should be used for client remote access VPN.

The two formats of IPSec packets can be used also in tunnel mode:

IPSec tunnels provide security for "IP only" traffic. The tunnel is configured to protect traffic either between two IP addresses or between two IP subnets. If the tunnel is used between two hosts instead of between two gateways, the outer IP address is the same as the inner IP address. In Windows 2000, IPSec does not support protocol-specific, port-specific, or application-specific tunnels. Configuration is performed using the IPSec Policy console by specifying a security rule containing a filter to describe the traffic that goes into the tunnel, a filter action for securing the tunnel and an authentication method to be used by the tunnel end points. Three types of authentication are supported: certificates, pre-shared key, and Kerberos.

For conceptual information on IPSec policy tunnel settings, see IPSec tunneling For information on configuring an IPSec tunnel, see To specify an IPSec tunnel