Orhan Ergun 21 Comments

In this article you will learn about the DMVPN design along with various IGP protocols such as EIGRP,OSPF and BGP.

 

DMVPN uses two major technologies for its operation :

 

  1. NHRP Next Hop Resolution Protocol
  2. mGRE Multipoint GRE

In this post I will explain all the basics of Cisco DMVPN.

Detailed routing protocol design over DMVPN will be covered in a different post which will be published in a few days.

Let’s use the topology below to understand the specific pieces of DMVPN. The picture below has been received from cisco.com.

cisco dmvpn

Figure-1 Cisco DMVPN Architecture

Three different address types are used in DMVPN.

1. Public – NBMA – Underlay address :  All these terms can be used interchangeably and refers to a static known address on the hub and a dynamic or static address on the spokes.

This address is used to bring up the tunnels. The underlay addresses of HUB and spoke must be reachable.

If a spoke to spoke tunnel is created, spoke to spoke underlay (BGP between Service Providers) addresses must be reachable as well.

If you are using DMVPN over Internet, the remote location reaches to its local service provider and service providers use BGP for public-NMBA-underlay DMVPN address reachability.

Dynamic NAT (Network Address Translation) is allowed on the spokes.

2. Private – Tunnel – Overlay address : You may see one of these addresses in different resources, but all refer to the same thing. They are used interchangeably.

In a single DMVPN cloud, one big tunnel subnet is used.

Spokes register their private to public address mapping to Hub which is the Next Hop Server with the NHRP register message.

For those who are familiar with LISP (Locator and Identity Separation Protocol), this operation is very similar to a Mapping database. LISP is out of the scope of this post.

All the routing protocols ( RIP, EIGRP, OSPF, PIM, BGP) can be used over the overlay tunnels, except for IS-IS !

IS-IS doesn’t run over IP but uses layer2 (IS-IS has an ether type), so IS-IS can not be used over DMVPN.

3. Remote or Local address :  This address is used in the local area and advertised through the routing protocol over the overlay DMVPN tunnel.

In the topology above, spokes or remote sites register to the hub via NHRP protocol.

The NHRP register message is used for spokes to register their underlay to overlay address mapping.

DMVPN hub knows all the public IP to tunnel IP mapping before the routing protocol starts to advertise local addresses. 

DMVPN supports IPv4 and IPv6 unicast and multicast transport through the overlay tunnels.

DMVPN also supports IPv4 and IPv6 transport through the underlay, which might be Internet or MPLS VPN.

DMVPN can be used without IPsec encryption. IPsec is not mandatory for the DMVPN operation,but most if not all of the networks use IPsec with DMVPN.

For large-scale DMVPN deployments, GETVPN can be used to encrypt the tunnels.

Since  HUB would create crypto key for each spoke separately,with IPSEC, with its group key capabilities, GET VPN provides excellent scaling if the encryption is the needed.

DMVPN has three Phases.

Phases 1, 2 and 3. Phasse 1 and 2 are made obsolete, by Phase 3, but I will explain how DMVPN works in Phase 1 and 2 as well.

DMVPN Phase 1 :

Spokes use Point to Point GRE but Hub uses a multipoint GRE tunnel.

A mGRE tunnel simplifies configuration greatly on the Hub.

Spokes have to specify the tunnel destination as Hub since they run P2P GRE tunnel not mGRE in DMVPN Phase 1.

Summarization is allowed from the Hub down to spokes. The Hub can only send default route as well ,since the remote spokes next hop is not preserved by the Hub.

Hub changes the IGP next hop to itself hence spoke to spoke traffic passes through the Hub.

Spoke to spoke tunnel cannot be created in DMVPN Phase 1.

In the topology above, for the 192.168.2.0/24 network behind Spoke 2, next hop is the tunnel IP of the HUB in the routing table of Spoke 1, not the tunnel IP of the Spoke 2.

Thus, the data plane traffic always passes through Spoke 1 – Hub – Spoke 2 with this topology in DMVPN Phase 1.

Why would you want to design DMVPN with Phase 1?

You may want to bring all the traffic to your centralised place where all your security devices are located.

You can filter unwanted traffic between the spokes first.

Imagine you are using DMVPN over the Internet.

After you create a DMVPN network, spoke to spoke communication, or a centralised shared services devices will be reachable through the Hub and traffic can be filtered before reaching the destinations.

Spokes can use their local router for Internet traffic. Internet traffic doesn’t have to go the HUB.

Although Phase 1 is obsolete and, as we will see, Phase 3 gives better scalability and feature sets, it still might be used. It provides excellent scalability for the routing compared to Phase 2.

If you are using DMVPN over IPSEC, since the HUB router needs to encrypt and decrypt packets twice, Phase 1 with IPSEC reduces the overall scalability.

DMVPN operation is very similar to the Ethernet ARP process. The difference is Ethernet is Broadcast Multiaccess, DMVPN is a Non Broadcast Multiaccess (NBMA).

DMVPN Phase 2:

Spoke to spoke dynamic on demand tunnels are first introduced in Phase 2.

In contrast to Phase 1, mGRE (Multipoint GRE, not Multicast) is used in Phase 2.

Thus, spokes don’t require tunnel destination configuration under the tunnel interface and tunnel mode is configured as ” Multipoint Gre”.

Spoke to spoke traffic doesn’t have to go through the HUB. Spokes can trigger on demand tunnel between them.

The biggest disadvantage of Phase 2 is, each spoke has to have all the Remote – LAN subnets of each other since the Hub, preserves the next hop of spokes.

In the topology above, for the 192.168.2.0/24 network behind the Spoke 2, Spoke 1 has the tunnel interface (Private-Overlay) address of Spoke 2 as a next hop. In Phase 1, hub is the next hop.

Thus, spokes have to have a reachability to the tunnel addresses of each other.

This disallows the summarization or default routing from Hub down to the spokes.

This is a serious design limitation for the large scale DMVPN networks.

For the distance vector protocols, Split horizon needs to be disabled and “no next-hop self” should be enabled on the HUB.

These are serious design limitations for the link state protocols that I will explain later in this post.

DMVPN Phase 3:

Spoke to spoke dynamic tunnels are allowed.

Spokes don’t have to have the next hop of each other’s private address for the local subnets.

192.168.2.0/24 network behind the Spoke 2, has a next hop as HUB in the routing table of Spoke1.

An NHRP redirect message is sent to the spokes to trigger spoke to spoke tunnels. Hub tells, Spoke 2 that Spoke 1 wants to communicate, and provides the Public-Underlay-NBMA address of Spoke 1 in the above topology.

Spokes have an NHRP Shortcut. Which means, even in the routing table of Spoke 1, the tunnel IP address of HUB is the next hop for the 192.168.2.0/24 behind the Spoke 2, and the FIB table of Spoke1 has the private-tunnel address of Spoke 2 as a next hop.

Since the next hop in the routing table of the spokes is HUB’s tunnel address, spokes don’t have to have the specific next hop information of each other.

This allows summarization and default routing in Phase 3.

Hub can send a summary or just a default route down to the spokes.

Hence, Phase 3 can scale extremely.

 

 
0.00 avg. rating (0% score) - 0 votes
  • Anonymous

    nice!!!

  • Nick Moody

    Hi Orhan, Excellent post regarding DMVPN. One subject I’d like to see a post about are the MTU challenges associated with IPSEC overheads on DMVPN and possible resolutions to them. Path MTU Discovery being one option but from my experience there are usually devices that deny the ICMP messages (Firewalls for example) for PMTUD to work. The option we sometimes use is ‘tcp mss adjust’ on the tunnel interfaces to dynamically adjust the MTU in the initial TCP SYN packet on behalf the client . This method comes with its own considerations with regards to being process switched vs hardware switched etc. I’d value your opinion on the subject.

    • @Nick Excellent points. As you said, since IPsec will bring additional header, we would have less space for the payload.

      PMTUD would be a problem because of the reason as you stated and you may end up fighting with security folks.

      Thus, recommendation is to set IP Mtu is to 1400 and mss as 1360. With MSS adjustment, we force the hosts to arrange segment size, excellent feature.
      Also as a reminder, mtu issue is not only specific to IPsec. Whenever we encapsulate the ethernet frame inside another layer, we should consider MTU.
      OTV,MPLS VPNs,LISP just couple other examples.

      • Nick Moody

        Thanks Orhan, At least with DMVPN we have a layer 3 interface to apply the tcp mss adjust, with some of the overlay technologies for layer 2 the MTU on the underlying transport network needs to be increased to accommodate the overheads. And that depends if the service provider will support that request . I’ll look forward to any future articles you’ll write about the subject, If I may request one 🙂

        • @Nick Thats absolutely important. If you receive layer 0 circuit like DWDM , or layer 1 circuit such as SONET/SDH it is okay but otherwise you need to negotiate with the service provider as well.
          Definitely the new posts will come on the subject as well. By the way do you have DMVPN on your network or did you design DMVPN networks if you are a consultant ? Can we hear about your design ?

  • Pingback: MPLS VPN and DMVPN Design Challenge - Network Design and Architecture()

  • Pingback: Dynamic Multipoint VPN (DMVPN) | bijigandum()

  • Pingback: OSPF Design Challenge - Network Design and Architecture()

  • Ugur Ersoy

    Thank you Orhan. I feel like running DMVPN instead of getting MPLSVPN service is better for small size companies. By this way, you have all access to resolve problems you encounter but in MPLS VPN you are mostly dependent to ISP even in selecting routing protocol between CE and PE. QoS can be enabled on both and you may have traffic engineering option partly in DMVPN via phases. Why would i choose MPLS VPN then? I would like to hear anything from you for comparison of them.

    • Hi Ugur.Is Question DMVPN vs SP Provided L3VPN or DMVPN vs SP Provided L2VPN ?

      • Ugur

        I’d like to hear about DMVPN vs. SP Provided L3VPN.

        • Actually excellent question, and this would be a good article content. Allow me to write an article for this so not only my answer but we can have a chance to listen other people opinion as well. Okay ?

          • Ugur

            I am waiting 🙂 Thanks in advance

  • Ugur

    I am waiting 🙂 Thanks in advance

  • Pingback: CCDE Training - Network Design and Architecture()

  • Maher El Zein

    Excellent!

  • Pingback: DMVPN point-to-point GRE and mGRE - Network Design and Architecture()

  • Pingback: DMVPN vs. GETVPN - Cisco Network Design and Architecture | CCDE Bootcamp | orhanergun.net()

  • Pingback: DMVPN Point-to-Point GRE and mGRE - Cisco Network Design and Architecture | CCDE Bootcamp | orhanergun.net()

  • Pingback: EIGRP RFC 7868 | Cisco Network Design and Architecture | CCDE Bootcamp | orhanergun.net()