Transport and Tunnel Modes in IPsec

The IPsec standards define two distinct modes of IPsec operation, transport mode and tunnel mode. The modes do not affect the encoding of packets. The packets are protected by AH, ESP, or both in each mode. The modes differ in policy application when the inner packet is an IP packet.

• In transport mode, the outer header determines the IPsec policy that protects the inner IP packet.

• In tunnel mode, the inner IP packet determines the IPsec policy that protects its contents.

In transport mode, the outer header, the next header, and any ports that the next header supports, can be used to determine IPsec policy. In effect, IPsec can enforce different transport mode policies between two IP addresses to the granularity of a single port. For example, if the next header is TCP, which supports ports, IPsec policy can be set for a TCP port of the outer IP address. Similarly, if the next header is an IP header, the outer header and the inner IP header can be used to determine IPsec policy.

Tunnel mode works only on IP-in-IP datagrams. Tunneling in tunnel mode can be useful when computer workers at home are connecting to a central computer location. In tunnel mode, IPsec policy is enforced on the contents of the inner IP datagram. Different IPsec policies can be enforced for different inner IP addresses. That is, the inner IP header, its next header, and the ports that the next header supports, can enforce a policy. Unlike transport mode, in tunnel mode the outer IP header does not dictate the policy of its inner IP datagram.

Therefore, in tunnel mode, IPsec policy can be specified for subnets of a LAN behind a router and can be specified for ports on those subnets. IPsec policy can also be specified for particular IP addresses, that is, hosts, on those subnets. The ports of those hosts can also have a specific IPsec policy. However, if a dynamic routing protocol is run over a tunnel, neither subnet selection nor address selection should be used, because the view of the network topology on the peer network could change. Changes would invalidate the static IPsec policy. For examples of tunneling procedures that include configuring static routes, see Protecting a VPN With IPsec.

In Solaris, tunnel mode can be enforced only on an IP tunneling network interface. The ipsecconf command provides a tunnel keyword to select an IP tunneling network interface. When the tunnel keyword is present in a rule, all selectors that are specified in that rule apply to the inner packet.

In transport mode, ESP, AH, or both ESP and AH, can protect the datagram.

Figure 19-3 shows an IP header with an unprotected TCP packet.

Figure 19-3 Unprotected IP Packet Carrying TCP Information

In transport mode, ESP protects the data as shown in Figure 19-4.

Figure 19-4 Protected IP Packet Carrying TCP Information

In transport mode, AH protects the data as shown in Figure 19-5.

Figure 19-5 Packet Protected by an Authentication Header

AH actually covers the data before the data appears in the datagram. Consequently, the protection that is provided by AH, even in transport mode, covers some of the IP header.

In tunnel mode, the entire datagram is inside the protection of an IPsec header. The datagram in Figure 19-3 is protected in tunnel mode by an outer IPsec header, and in this case ESP, as is shown in Figure 19-6.

Figure 19-6 IPsec Packet Protected in Tunnel Mode

The ipsecconf command includes keywords to set tunnels in tunnel mode or in transport mode.

• For details on per-socket policy, see the ipsec(7P) man page.

• For an example of per-socket policy, see How to Secure a Web Server With IPsec.

• For more information about tunnels, see the ipsecconf(1M) man page.

• For an example of tunnel configuration, see How to Protect a VPN With an IPsec Tunnel in Tunnel Mode Over IPv4.