Orhan Ergun 16 Comments

[follow_me]OSPF as a PECE routing protocol can be used in the MPLS Layer 3 VPN design between customer and the service provider.

If the customer receives an MPLS Layer 3 VPN service , routing protocol is enabled between the customers and the Service Providers.

Don’t forget that static routing is a routing protocol !

This VPN connection is called a peer-to-peer.

Another VPN connection type is overlay VPN and I will explain them in a different article.

If you decide to receive an MPLS layer 3 VPN service but you don’t know which routing protocol you should use with the Service Provider, my advice; just continue with whatever protocol you use in your datacenter, campus, remote location and so on.

There are many caveats which I will explain in this post.

I will explain how OSPF as a PE-CE routing protocol works in the context of MPLS Layer 3 VPN in this article, but as always I will explain what are the business cases, technical requirements, alternative solutions and limitations as well.


Before I start to explain how OSPF works as a PE-CE routing protocol in MPLS Layer 3 VPN, please be aware that OSPF as a PE-CE may not be provided by your service provider.

Service provider can say that they only provides a static or BGP as an MPLS VPN PE-CE routing protocol.

Then you will not have an option but continue with either static or BGP.

Also CE can be managed by the Service Provider if it is managed MPLS Layer 3 VPN service.

In this case you don’t have to deal with routing over the WAN at all and all the complexity handled by service provider.

Lets start with the basic topology which I took from the cisco.com then continue with the advanced topics of MPLS Layer 3 VPN by using OSPF as PE-CE protocol.

ospf pe-ce

In the topology above two companies merge. Site-1 and Site-2 using different OSPF area. On the Service Provider PE; OSPF process IDs are different.

Customer prefixes are received from the VRF, redistributed into MP-BGP.

On the another site PE; customer prefixes are seen as BGP route within that customers VRF. From that PE, BGP is redistributed into OSPF back.

Although there is BGP to OSPF redistribution, if the process ID are the same on the PEs for that customer, prefixes are advertised to customer as an OSPF Type 3 LSA (Inter Area).

In this topology since the OSPF process IDs are not the same, routes would be received as Type 5 (External LSA).

You cannot avoid this if you don’t want to change VRF and process ID for merge scenarios but OSPF domain ID attribute is used to overcome.

If OSPF domain ID is set as the same on the both PE for different VRF name, still routes are advertised as Type 3 LSA (Inter Area).

Since Site-1 and Site-2 has different OSPF area, even if you have backdoor link, MPLS VPN cloud can be used by metric adjustment.

In the above topology; even Site-1 and Site-2 would be in the same area, still route would be advertised as Type 3 LSA at best.

In the below topology customer wants to use MPLS VPN as primary link and they will setup DMVPN and use it as secondary.

Let’s examine the topology and see what are the issues and how can we solve it.

ospf shamlink

In the above topology Customer sites are in same OSPF area, Area 1.

OSPF Process IDs are the same on the PE1 and PE2.

Routes should have been advertised as Type 3, but the problem customer also has a backdoor link in the same area, Area1. Thus CE2 receives the prefixes which are coming behind CE1,as Type 1 from the backdoor link.

When CE2 receives CE1 prefixes as Type 1, CE2 floods them to the PE2.

Since PE2 receives those prefixes from CE2 as OSPF Type 1, it doesn’t advertise Type 3 LSA towards CES.

Solution can be achieved by creating an OSPF virtual link over the MPLS VPN network.

This is special type of virtual-link and called as OSPF sham-link.

In this topology you create an OSPF sham-link and put in the same area as the customer sites, which is Area 1 in this case.

When you create an OSPF sham-link and put  it into an Area 1, PE2 will send CE1 prefixes to CE2 as Type 1 or Type 2 LSA. Then with the OSPF metric adjustment, MPLS VPN link can be used as primary.

How should you design your areas ?

Should you put your sites in Area 0 or non-backbone area ?

Whatever you do unless Service Provider creates a sham-link over the MPLS core, you will receive Type 3 LSA at best.So Area 0 or non-backbone area doesn’t matter.

If you don’t put the network behind CE in a different area you will not have an ABR.

ospf pe-ce multi area

In the above topology, if you use between PE-CE Area0 and switch to router link in Area 10, CE1 become an ABR, so summarisation can be done on CE1 likewise for the networks behind Switch2, CE2 becomes an ABR, so it can summarize.

If you place all subnets, including transit links and loopback in Area0, since you will not have an ABR, summarisation cannot be done on the customer site.

If you don’t summarise, you can have easily thousands of prefixes in the routing table of CE routers.

Most of the time those routers in the remote branch office has small amount of CPU and memory. You don’t want to send thousands of prefixes.

Also Service Provider most probably will tell you that they support up to some amount of prefixes, more than that you pay extra.

Still on the PE, summarisation either on the Ingress PE or at the egress PE can be done.

If you do it on ingress PE under the BGP, you can send a summary to all your locations.If you do it on the Egress PE, you can send a summary to whichever PE you want selectively. Of course it is not scalable, but can be done.

ospf pe-ce different areas

What is the problem with the above topology?

PE-CE links in the Area 1 but the switch to router link is in area 10. CE1 cannot be an ABR unless it has an interface in Area0. So this design normally doesn’t work. But if you create only a loopback interface and put it in Area0, it is enough for CE1 to be an ABR, so it works.

Although sham-link is a concept can be enabled on the service provider backbone, if you design carefully your backdoor link, you don’t have to deal with the service provider. They won’t either do it, or don’t know how to do it.

ospf pe-ce no sham-link

I would put PE-CE links in Area0, Local subnets and backdoor links in different areas.CE2 would get CE1 local subnets from MPLS cloud as type 3, as it would receive from backdoor link, to prefer MPLS VPN  you would arrange a metric.

What about you ?

If you are a service provider engineer, do you configure sham-link for your customer ?

If you are an enterprise engineer who uses MPLS Layer 3 VPN, what is your PE-CE routing protocol ?

Let’s discuss these as always in the comment box below.

0.00 avg. rating (0% score) - 0 votes
  • That is So helpfull.

    Eyvallah üstad 🙂

    • Thanks Numan for reading and commenting !

  • Hi Orhan, thanks for this article, I´m currently working for a service provider and I’m also preparing for CCDx track (I´m starting my journey so CCDA is my first milestone), I mention this because I´m starting to look at things with “Design” mindset (I’m currently CCNP R&S and have several years of experience), so basically I have a question, is there any best practice regarding as what PE-CE protocol to use? and the main reason for this questions, is this, currently I’ve seen OSPF deployed as the protocol of choice whenever a customer add redundancy to their primary links, that is regardless of what IGP the customer is using (and I’m refering particulary to the environment that I see on my day to day job), must likely customers in our región (Central America) do not run any IGP, and mostly rely on the provider for routing. I’ve been reading a bit lately, and it seems to me actually eBGP could be the prefered choice, but is there any specific RFC or best practice as to justify which is the PE-CE routing protocol of choice under the specific case that customer is not runing any IGP inside their network?

    • Hi Hector, As far as I know there is no RFC on this specific subject. However, when you choose to deploy MPLS Layer 3 VPN, you are choosing to handover your network control to service provider.

      Think about if you have 100 sites,based on your business (It might be retail), you may need to have partial or full mesh connectivity model.If you would receive P2P service you would manage it.

      If you receive an Internet service and decide to implement GRE or even mGRE tunnel (DMVPN maybe), you would manage all the routing. If business only requires Hub and spoke things get easier.
      How MPLS L3 VPN helps, you don’t have to manage your routing,you just peer with the peer and they do all the routing magic for you.

      It scale very well for the full mesh requirement for the customer since the routing operations are done by the provider.

      If I would require traffic engineering, such as, I want to send my data traffic to Path1, but voice traffic to Path2, then since I would need to manage my routing, I would choose layer 2 connectivity.

      I wouldn’t give the control to the provider. But you need to have an expert to manage routing and traffic engineering in your company.

      There are many other considerations , such as if multi tenancy is your requirement and you need to send 10s of thousands of route to the provider, they wouldn’t accept or you would pay more, so if i have an l2 connection, I can send as much route as I want between my sites.

      If you need a fast convergence, service provider most likely will not run BFD for fast detection with you if the service is l3 vpn.
      if it is l2 vpn, your peer is your another device.

      Tune the timers, use bfd, use FRR,implement traffic engineering, do effective summarisation (which you can’t do easily with l3 vpn as I showed in this article) whatever as you like.

      But if, I don’t have any of these requirements, small shop, but still want redundant connectivity for some sites , maybe single for non-critical site (which is typical l3vpn candidate) I would go with static route (If I have to manage the CEs, otherwise let the service provider think.)

  • Hi, I think that is great advice from customer end standpoint, what would you suggest from service provider perspective, let´s say you have a customer A which is currently not running any IGP inside their network, and is using your MPLS L3VPN service, and he has redundant links in their central site and in few other key remote branch offices, and it is up to you as service provider to decide which protocol to use to peer with customer CE routers, would you choose eBGP or OSPF and what is the reational behind picking one over the other

    • I wouldn’t choose an IGP since I wouldn’t want to deal with redistribution on every site.

      IGP to BGP and BGP to IGP redistribution I mean, also PE devices at the service provider site has an IGP process limit.

      If you run BGP, you don’t need to worry about redistribution.

      If the customer datacenter in the central office,default route from the offices to the MPLS VPN cloud is enough.

      If datacenter is located in different place than central office, send default route towards everywhere.

      If the networks grow, static route redistribution become an issue,so if the frequency of change is not high, static route should be chosen, otherwise I would choose to deploy BGP.

      But it also depends what the customer site equipment supports, if you use static route you can use even very basic,cheap switches which don’t support IGP or BGP.

  • Orhan, thanks for taking the time to reply to my queries, it has been very heplful, really appreciate!

    • Welcome,glad that it was helpful.Would like to see your continuous participation.

  • burak kebapci

    Thanks for sharing this article,
    DMVPN and MPLS are one of the most important subjects in advanced networking.

    • Thanks Burak, glad you liked it. Yes they are important and major protocols

  • Sukhjit Hayre

    Hi Orhan

    Good article, one of the main design choices to not use a sham-link (if they knew how to configure one :)) is because the CE routers don’t want to be apart of SPF events over the SP core network if a link were to go down/up, hence its more of a patch then a long-term fix in this regard.

    Keep up the good work.

    • Hi Sukhjit,

      Thanks for the comment. Yes you are right that it is not a good practice since it adds complexity to SP and they won’t do it by the way you can fix it on your site, without sham-link on the SP site as I showed in the article.

      SP OSPF Superbackbone isolates the SPF events, so if failures happen in one location, it will not be propagated to the other site so you put domain boundary at the SP edge.

  • Sukhjit Hayre

    Hi Orhan

    Thankyou, yes what I meant was where a sham-link is configured between the concerned PE routers then the CE routers are apart of the O intra-area SPF domain and where a linkup/linkdown event on the sham-link (albeit it may be redundant path between loopbacks) then it would cause an SPF run…..hence this solution is considered a patch for OSPF TE then a long term solution for this reason alone.

    • Hi Sukhjit,Okay then we are on the same page.
      Did you write your comment to design scenarios ? You should definitely participate to scenarios as well. Cheers

  • MHussein

    Thanks for this unique topic. But, I want to ask if the customer needs to run 2 different IGPs between PE and CE at 2 different sites for example OSPF at a site and BGP at another site, Is this scenario applicable ?

    • Hi @MHussein , Yes of course. In any site customer can use any protocol as PE-CE in theory. But as I shared in my many articles, BGP and Static routing is the preferred choice by the Service Providers and many of them only provides those two protocols only.

      Technically it is possible to use different protocols in different site , but managing them , understanding the details and critical points such as loop avoidance, sub optimal routing would be hard compare to using only one protocol. Thats why my design advice always use at least the same protocol as PE-CE.