IPsec VPN Fundamentals

In the previous lesson, you saw that a public WAN sends your traffic across the Internet, and that this traffic needs to be encrypted.
Take a company with a headquarters in New York and a branch office in San Francisco: R1 and R2 exchange traffic through the Internet every day.

Figure 1 – Enterprise sites network topology

But there is a problem.

The Internet provides zero encryption. Every packet traveling between the two sites is in cleartext.
Anyone capturing traffic between the two ISP routers can read your data.

IPsec solves this.
It wraps every packet in an encrypted envelope before sending it across the untrusted network.
Only the two IPsec peers hold the keys to decrypt the content.

Figure 2 – IPsec encrypted tunnel concept

IPsec is not a single protocol. It is a framework that combines several components. Let's go through each one:

• IKE (Internet Key Exchange): Negotiates encryption parameters and exchanges keys between peers.

• ESP (Encapsulating Security Payload): Encrypts and authenticates the data payload.

• AH (Authentication Header): Provides authentication and integrity without encryption. Rarely used in modern deployments because ESP can do both.

• SA (Security Association): A one-way agreement between two peers that defines the encryption algorithm, keys, and lifetime for a specific traffic flow.

In real-world deployments, the combination you will see most often is IKE + ESP.

Now that you know what IPsec is, let's see how it actually establishes a secure tunnel.

IPsec uses a two-phase process. Before any of your traffic is encrypted, the two peers must agree on security parameters and exchange cryptographic keys.

This negotiation happens through IKE (Internet Key Exchange).
Here is the high-level packet processing flow when IPsec is active:

Figure 3 – IPsec packet processing flow

The process works in three stages:

1. Interesting traffic detection: The router checks outgoing packets against an ACL. If the packet matches, it triggers the IPsec process.

2. IKE Phase 1: The two peers establish a secure management channel (ISAKMP SA) to protect the negotiation itself.

3. IKE Phase 2: Using the secure channel from Phase 1, the peers negotiate the actual IPsec SA that will encrypt user traffic.

IKE Phase 1

IKE Phase 1 creates a bidirectional ISAKMP Security Association between the two peers.
Think of it as the secure meeting room where both sides sit down and agree on how they will protect the actual data.

Figure 4 – IKE Phase 1 tunnel establishment