IPsec Overhead Calculator
Calculate online IPsec ESP tunnel overhead, usable inner MTU, TCP MSS clamp, NAT-T impact, and padding headroom for safer VPN MTU planning.{{ result.summaryTitle }}
| Metric | Value | Detail | Copy |
|---|---|---|---|
| {{ row.label }} | {{ row.value }} | {{ row.detail }} |
| Component | Bytes | Applies as | Note | Copy |
|---|---|---|---|---|
| {{ row.component }} | {{ row.bytesLabel }} | {{ row.applies }} | {{ row.note }} |
| Check | Recommendation | Detail | Copy |
|---|---|---|---|
| {{ row.check }} | {{ row.recommendation }} | {{ row.detail }} |
IPsec Encapsulating Security Payload (ESP) tunnel overhead is the extra byte cost added around a protected packet before it crosses the outside path. The outside path still has a maximum transmission unit (MTU), so every byte spent on outer IP headers, ESP fields, NAT traversal, padding, and link-specific encapsulation leaves fewer bytes for the original packet.
This matters when a VPN works for small traffic but drops, stalls, or fragments larger flows. A 1500-byte underlay does not automatically carry a 1500-byte protected packet after ESP is applied. The safe inner MTU is smaller, and the TCP maximum segment size (MSS) often needs to be clamped so TCP data fits inside the tunnel without relying on fragmentation.
IPsec overhead estimates are planning numbers, not live packet captures. Vendor defaults, negotiated transforms, extended sequence handling, and extra carrier or access-link encapsulation can change the real byte count. The safest use is to model the tunnel proposal, leave room for known path additions, and then verify with path MTU testing or production telemetry before treating the number as final.
MSS clamping only helps TCP. UDP, QUIC, ICMP, and other payloads still need Path MTU Discovery, application-level packet sizing, or a deliberately conservative inner MTU. If ICMP Packet Too Big or Fragmentation Needed messages are blocked, an apparently correct MTU can still produce black-hole symptoms on larger packets.
Technical Details:
ESP tunnel mode carries the original IP packet as protected payload and adds a new outer IP header for the path between VPN peers. The ESP header includes the security parameters index and sequence number, followed by transform-dependent fields such as an explicit IV or nonce, payload data, padding, Pad Length, Next Header, and an integrity check value or authentication tag when the selected transform uses one.
Padding is not a fixed constant. ESP can add padding so the encrypted data satisfies the transform's block or alignment requirements, and the Pad Length field itself records how many padding bytes were used. Traffic Flow Confidentiality (TFC) padding is separate from ordinary ESP alignment padding and consumes real space when configured.
Formula Core:
The budget calculation searches for the largest inner IP packet whose complete outer packet still fits inside the underlay MTU after any safety reserve is subtracted.
OuterIP is 20 bytes for IPv4 and 40 bytes for IPv6. Native ESP has no UDP header, while NAT traversal adds the 8-byte UDP/4500 header to ESP data packets. The ESP header is modeled as 8 bytes for SPI and sequence number, and the explicit ESP trailer is modeled as 2 bytes for Pad Length and Next Header.
ESP Profile Assumptions:
| Profile | IV or nonce | ICV or tag | Block alignment | Practical note |
|---|---|---|---|---|
AES-GCM-16 |
8 B |
16 B |
1 B |
AEAD profile with an explicit nonce and full authentication tag. |
AES-GCM-12 |
8 B |
12 B |
1 B |
Same explicit nonce model with a shorter tag. |
AES-CBC/SHA1 |
16 B |
12 B |
16 B |
Classic CBC layout where padding often changes with packet size. |
AES-CBC/SHA256 |
16 B |
16 B |
16 B |
CBC layout with a larger integrity value. |
ChaCha20-Poly1305 |
8 B |
16 B |
1 B |
AEAD layout modeled with an explicit nonce and tag. |
NULL/SHA1 |
0 B |
12 B |
1 B |
Integrity-only ESP profile with no IV. |
Custom ESP |
User value | User value | User value | Uses the Advanced byte fields for non-listed transforms. |
Path and Payload Boundaries:
| Boundary | Rule used here | Why it matters |
|---|---|---|
| Underlay MTU | Input range is 576 to 65535 bytes. |
Values below the IPv4 minimum reassembly size are flagged as not meaningful for this estimate. |
| IPv6 protected packet | A recommended inner MTU below 1280 bytes is flagged when the inner payload is IPv6. |
IPv6 has a minimum link MTU of 1280 octets, so smaller protected IPv6 packets need review. |
| TCP MSS clamp | Inner MTU minus inner IP header, fixed 20 B TCP header, and optional TCP option reserve. |
The displayed clamp is a TCP payload limit, not a full packet size. |
| ESP padding | Padding is recalculated for each candidate inner MTU and can range from 0 to 255 bytes. |
The largest fitting inner MTU is not always found by subtracting one fixed overhead number. |
| Safety reserve | Reserve bytes reduce the recommendation but are labeled separately from wire overhead. | This keeps conservative planning space distinct from bytes actually transmitted in the ESP packet. |
The result is deterministic for a given input set. The same selected transform, path-extra choice, NAT-T flag, safety reserve, TFC padding, and TCP option reserve drive the MTU ledger, overhead stack, clamp guidance, profile comparison ladder, and JSON output.
Everyday Use & Decision Guide:
Start with the outside path MTU and the transform that most closely matches the tunnel proposal. For a common Ethernet underlay, 1500 bytes with AES-GCM with 16-byte tag, IPv4 outer header, IPv4 inner payload, and NAT-T enabled gives a practical first pass for many site-to-site or remote-access tunnels.
Use Path extra only for bytes that still sit outside the ESP packet and still consume the same underlay MTU. PPPoE, VLAN, GRE, and IP-in-IP presets are useful when the physical or provider path adds those bytes after you have chosen the VPN transform. If the underlay MTU you entered already accounts for that access overhead, leave Path extra at None to avoid double-counting.
- Turn on
Include NAT-T UDP/4500when ESP data packets are encapsulated for NAT traversal. Leave it off for native ESP protocol 50 paths. - Set
Inner IP versionto the protected traffic family before using the MSS clamp. IPv6 subtracts a40 Binner header instead of20 B. - Use
Safety reservewhen vendor padding, carrier tags, or nested features are not fully known. The reserve intentionally makes the recommendation smaller. - Add
TCP optionsonly when you want the clamp to leave explicit room beyond the fixed TCP header. The common field hint is12 Bfor timestamp or SACK option space. - Use
Custom ESP byte profileonly when the selected transform is not represented by the built-in profiles and you know the IV, tag, and alignment values.
The first trust check is the summary badge. fits path means the modeled packet fits the safety-adjusted budget, while review MTU means a minimum-size or usable-payload warning needs attention. Open MTU Ledger for the recommended inner MTU, TCP MSS clamp, on-wire overhead, total MTU reduction, outer packet size, ESP padding, and expansion percentage.
A good next step is to compare Overhead Stack against the platform configuration and then check Clamp Guidance. If the tunnel carries UDP, QUIC, or mixed traffic, keep PMTUD working rather than assuming the TCP MSS clamp solves every packet-size problem.
Step-by-Step Guide:
- Enter the outside path limit in
Underlay MTU. Use the real transport path value, not the desired inner VPN interface MTU. - Choose the closest
ESP profile. If none matches the negotiated transform, chooseCustom ESP byte profileand enter the custom IV or nonce, ICV or tag, and block alignment inAdvanced. - Set
Outer IP versionfor the tunnel between peers, then setInner IP versionfor the protected traffic whose MSS you want to clamp. - Choose any real
Path extraand decide whetherInclude NAT-T UDP/4500applies to ESP data packets on this path. - Open
Advancedonly for safety reserve, TCP option reserve, TFC padding, additional path bytes, or custom ESP byte values. - Read
MTU Ledgerfirst. The most portable planning values areRecommended inner MTU,TCP MSS clamp,On-wire overhead, andOuter packet at recommendation. - Use
Overhead Stackto audit which components consumed bytes, then useProfile MTU Ladderto compare built-in ESP profiles under the same path assumptions. - Copy or export the result only after checking that warnings are resolved and the assumptions match the actual tunnel policy.
Interpreting Results:
The recommended inner MTU is the largest protected IP packet size that fits the modeled outer path. The on-wire overhead is the ESP and path byte cost at that recommendation. The total MTU reduction can be larger than the on-wire overhead because it also reflects safety reserve and leftover padding headroom.
| Output cue | Meaning | Action |
|---|---|---|
Recommended inner MTU |
Largest protected IP packet that fits inside the modeled underlay budget. | Use it as the candidate tunnel interface MTU or as the ceiling for protected payload sizing. |
TCP MSS clamp |
TCP payload limit after subtracting inner IP, TCP, and optional TCP option bytes. | Apply it where PMTUD is unreliable or where avoiding TCP fragmentation matters more than maximum segment size. |
Outer packet at recommendation |
Modeled full packet size after ESP and outer path bytes are added. | Confirm it is at or below the safety-adjusted underlay budget. |
ESP padding at recommendation |
Alignment padding needed for the selected transform at the chosen inner MTU. | Expect it to change when the inner MTU, profile, TFC padding, or alignment changes. |
review MTU |
At least one warning is active, such as an IPv6 inner MTU below 1280 or no usable TCP payload budget. |
Raise the underlay MTU, reduce overhead, disable unnecessary extras, or choose a smaller protected payload. |
The profile ladder is a comparison under the same path assumptions, not a recommendation to change cryptography. If one transform appears to leave more inner MTU, that only reflects the byte model. Security policy, interoperability, hardware offload, compliance, and platform support still decide which ESP transform is appropriate.
The JSON view mirrors the same assumptions and outputs for documentation or ticket notes. Treat exported results as a snapshot of the selected assumptions, because changing NAT-T, outer IP version, path extras, TFC padding, or custom profile bytes can change the result immediately.
Worked Examples:
IPv4 ESP tunnel with NAT-T
Use an underlay MTU of 1500, AES-GCM with 16-byte tag, IPv4 outer header, IPv4 inner payload, no path extra, NAT-T enabled, and no safety reserve. The fixed parts are 20 B outer IPv4, 8 B UDP/4500, 8 B ESP header, 8 B explicit nonce, 2 B trailer, and 16 B tag, plus alignment padding. The recommendation lands at a smaller inner MTU than the underlay, and the displayed TCP MSS clamp subtracts another 40 B for the inner IPv4 and TCP headers.
IPv6 payload warning
If the inner payload is IPv6 and the modeled overhead leaves less than 1280 bytes of inner MTU, the result shows a review warning. That warning does not mean ESP cannot encrypt the packet. It means the selected tunnel and path budget are below the normal IPv6 minimum and should be changed before carrying IPv6 traffic at that size.
Broadband access with PPPoE and VLAN
Choose PPPoE plus VLAN (+12 B) when those bytes still consume the same outside MTU. The added path cost reduces the inner MTU before ESP profile differences are even considered. If the provider or interface MTU has already been reduced to account for PPPoE and VLAN, keep the preset at None and enter that lower MTU directly.
FAQ:
Does NAT-T always add more than 8 bytes?
For ESP data packets, this calculator models NAT traversal as the standard 8-byte UDP header. IKE port sharing, keepalives, and the non-ESP marker are separate control-plane or packet-format concerns and are not counted as ESP data overhead here.
Why does padding change when the MTU changes by one byte?
ESP padding depends on the selected transform's block or alignment requirements. A one-byte change in inner packet size can move the packet onto a different alignment boundary, so the padding value can jump rather than stay constant.
Should I use the TCP MSS clamp as the tunnel MTU?
No. The inner MTU is the protected IP packet size. The MSS clamp is only the TCP payload size after inner IP and TCP header bytes are subtracted.
Does MSS clamping replace Path MTU Discovery?
No. MSS clamping can keep TCP segments smaller, but it does not control UDP, QUIC, ICMP, or other non-TCP payloads. PMTUD or packetization-layer discovery still matters for mixed traffic.
Where does the calculation run?
The calculation, charting, table exports, and JSON output run in the browser. The current option set can be reflected in the URL, so avoid placing sensitive network design details there if the URL will be shared broadly.
Glossary:
- ESP
- Encapsulating Security Payload, the IPsec protocol that protects packet payloads and adds ESP header, trailer, padding, and integrity or authentication fields.
- MTU
- Maximum transmission unit, the largest IP packet size a path can carry without fragmentation at that layer.
- MSS
- Maximum segment size, the largest TCP data payload advertised for a TCP segment. It excludes the IP and TCP headers.
- NAT-T
- NAT traversal for IPsec ESP data packets, commonly modeled here as ESP carried inside UDP/4500.
- TFC padding
- Traffic Flow Confidentiality padding, optional extra ESP payload padding used to obscure traffic size patterns when negotiated and configured.