Orhan Ergun 19 Comments

Inter AS Option C is the most complex, insecure, uncommon, but extremely scalable inter provider MPLS VPN solution.

Why do companies need Inter-AS MPLS VPN service ? 

Why Inter AS MPLS VPN

Figure 1: Why Inter-AS VPNs ?

In the simplified topology shown in fig. 1, customer A has two locations: location A1 and location A2.

Location A1 is the dual home connected to service provider A.

Location A2 is the dual home connected to service provider B.

It is not compulsory that locations should be dual-homed.

However, there is a restriction: service provider A does not have a POP to connect to the location of customer A2.

For Customer A to has reachability between A1 and A2 locations via MPLS VPN service, service providers must create Inter AS MPLS VPN connection between the two locations.

In this article, I will explain how service providers can use Inter AS Option C to assist customers to have an end-to-end MPLS VPN service.

In the Inter AS Option B post, I explained that ASBR routers between the service providers do not keep a VRF table for the VPN customers.

As depicted in the fig.2 (shown below), as for Inter AS Option B, MP-BGP VPNv4 session is set up between service providers’ ASBR PEs.

optionb

 


 Figure 2: Inter-AS Option B

As for Inter AS Option B, ASBR routers – the provider-edge devices between the service providers – maintain only the VPN prefixes of the customers in the BGP table.

In fact, I have shown that VPNv4 BGP session has been set up between the ASBRs.

The high-level operational differences between Inter AS Option C and Inter AS Option B are in two folds: one is that ASBRs do not have VRF table; the other is that unlike Inter AS Option B, Inter AS Option C do not keep VPN prefixes for the customer on the ASBR.

ASBR is the edge routers which are used to connect the Service Provider each other.

As depicted in fig. 3 (shown below), as for Inter AS Option C, the VPNv4 BGP session is set up between route reflectors of the service providers.

 

option c

Figure-3 Inter AS Option C 

In fig.3, there are two service providers: service provider A and service provider B.

Service Provider A: 

Service provider A has two customers: customer A and B

For scaling purposes, all the provider-edge routers run VPNv4 BGP session with VPNv4 route reflector.

Service provider B:

Service provider B has two customers: customer A and B

A and B are companies which require Inter AS MPLS VPN service.

Service provider B runs IS-IS internally ( It could run other IGP protocols as well for sure)

PE-CE routing protocols are enabled on the VRF; thus service provider A has two separate VRF table.

For scaling purposes, all the provider-edge routers run VPNv4 BGP session with VPNv4 route reflector.

Inter AS Option C runs by enabling VPNv4 BGP session between the Route Reflectors of the Service Providers.

Provider-edge routers of service providers do not set up VPNv4 neighborship neither with AS nor between ASes.

In order to use the route reflectors to setup VPNv4 neighborship, they should move towards each other. i.e. BGP runs over TCP.

For reachability, route reflector loopback interfaces are advertised on the global routing table of service providers.

As for AS Option C depicted in fig. 3, ASBR PEs of the service providers run IPv4 BGP session between them.

Over IPv4 BGP session, loopback interfaces of route reflectors and internal PEs are advertised. Next, I will explain why you need to advertise internal PE loopback addresses, not just route reflectors.

ASBR PEs know the loopback address of Route reflector through OSPF in the Service Provider A network, through IS-IS in the Service Provider B network.

Since the VPNv4 EBGP neighborship is set up between VPNv4 RR of the service provider, the next hop for the VPN route is the route reflector.

Traffic trombones on the RR unless ” no bgp next-hop unchanged ” configured on the Route Reflectors. This is one of the important caveats in Inter AS MPLS VPN Option C.

Once this feature is enabled, route reflector does not change the next hop to itself even though the peering is EBGP VPNv4, between the RRs. ( Whenever there is an EBGP session between two BGP speaker, next hop is changed since they are not client of each other but regular EBGP neighborship, otherwise RR doesn’t change the next hop in IBGP)

In this case, because internal PEs continue to be a next hop for the customer prefixes, internal PE loopback is advertised on IPv4 BGP session between ASBRs.

As for the VPN next hop, transport label should be assigned to MPLS VPN to work efficiently, and MPLS label is assigned to the IPv4 unicast routes, which are the loopback of RRs and Internal PEs of ASBR.

Inter-AS Option C Design Objectives:

Inter AS Option C is unique, since it requires internal addressing advertisement between the service providers.

Those addresses are leaked into the global routing table of the providers, a process that providers dislike.

It can still be used in large scale design if the company has multiple autonomous system. (Not between the service providers).

This is just a network design option. Please note that it is not the only option.

Inter AS Option C  is very difficult to implement because it requires meticulous redistribution at the edge of the networks, VPNv4 neighborship between the route reflectors, IPv4 BGP neighborship between the ASBRs, label advertisement, route target agreement between the service providers, and so on.

It is insecure because the internal address leaks between the providers.

I know you must have thought whether AS design is common between the service providers in real life.

In reality, I helped two times Inter-AS MPLS VPN design of the Service Providers, both of them concluded with Inter AS Option A.

Let me ask you these questions. 

Is Inter AS MPLS VPN enabled on your network?

Are you planning to design Inter AS VPN?

Let’s discuss this in the comment section.

 
0.00 avg. rating (0% score) - 0 votes
  • Alvaro

    Hello Orhan,

    Great post, I’m a big fan of your work. I work for a latin american carrier in the regional network department. The carrier has coverage in several countries, and we have a core regional network which connects to the network of every country. We have option C in the data core, and every country is seen as a separate carrier and has a couple of route reflectors connected to the core. It is a highly complicated implementation and the troubleshooting is also complex. But this way the core routers do not carry vpnv4 routes and are almost strictly for forwarding. This saves a lor of memory which was the goal, because these routers are also part of an internet topology, and have full tables from Tier 1 ISP’s providers. They are Juniper by the way. Hope to keep reading more of your great design articles.

    Best regards

    • Hi Alvaro,
      It is pleasure to see such an experienced follower.Your comment is proof of Option C in the large scale design.

      Different AS in different countries and RRs run VPNv4. Thus you save a lot of memory and CPU on the ASBRs. But as I understand you run multi service PE. So you have internet and VPN customers on the same PE, there are pros and cons of that design.Probably it is a good idea to write about it, hope to see your comment on the other posts as well.

      Cheers,
      Orhan

      • Alvaro Arriola

        Hello Orhan,

        Thank you for your comment. Actually the PEs are not multiservice, well not strictly, we use logical systems to separate the internet topology from the data topology, they’re connected as if they were totally different and independent networks. But they’re sharing the same memory, line cards, etc. This is why Option C was kind of an advantage to save memory. Although one interesting disadvantageof our topology is that when both logical systems need to converge in, for example, mpls at the same moment, convergence from one system affects the other, so we could see sometimes slow lsp switching to the protection (strictly by testing in the forwarding plane, everything in the control plane appears excellent).

        Regards,
        Alvaro

        • @Alvaro Thanks for explanation.
          So you have still multi service PE then. You are using the same physical box for both VPN and Internet customers.
          The downside of this is fate sharing.
          As you said in your comment , the problem in one service, it might be over utilisation of resource , it might be attack so on , will impact the other service.
          I assume you are separating them with VRF , although full internet table doesn’t have to be in VRF.
          So far do we agree ?
          What about Route reflectors , are you using same RR clusters for both Internet and VPN service or per service RR set ?
          Cheers

          • Alvaro

            Orhan,

            Yes it is the same physical box, but with two logical systems. As I mentioned these are Juniper routers, so the logical systems separate the topologies, it is not the same as with a VRF. The router spawns a new routing protocol daemon for every logical system. So it is completely separate in a logical way, but the hardware is still shared. In the case of the data core each pair of route reflectors does not reflect using a route reflector config, but because it receives the vpnv4 routes from ebgp, it advertises them to all its ibgp peers, and it seems as if it were a reflector.

            Kind regards,

            Alvaro

          • You can allocate CPU and memory resources per logical system though, right ?
            Yes RR in the data path in your case. Nexthop is for the EBGP prefixes are RR. At least for VPN prefixes. In this case how you send more than one path to IBGP peer, unique RD ?

  • Haroon

    Posted: Provider Edge routers of Service Providers don’t setup VPNv4 neighborship neither with AS nor between ASes.

    As per the diagram PE’ are establishing VPNv4 session with RR. So i think this is typo mistake this should “P”. Is that correct?

    • @Haroon, thanks to ask this question. I was thinking that It might create a confusion. No there is no Typo. I mean ASBR PE there.

      ASBR in many cases is also a PE router. If you are terminating a customer VRF on ASBR PE as well, so you run VPNv4 with Route reflector in that case.

      By the way, You do not run VPN or IP BGP session with P router. Never !. You enable MPLS to have BGP free core in the first place.

  • Haroon

    Posted: Provider Edge routers of Service Providers don’t setup VPNv4 neighborship neither with AS nor between ASes.

    As per the diagram PE’ are establishing VPNv4 session with RR. So i think this is typo mistake this should “P”. Is that correct?

  • Pingback: Carrier Supporting Carrier - CSC - Network Design and Architecture()

  • Alvaro

    Hello Orhan,

    I’m sorry for the late response. We don’t allocate specific memory/cpu for the logical systems. Each PE has a unique RD. To clarify the topology, it is actually a Carrier Supporting Carrier (or Carrier of Carriers as Juniper refers to it). I didn’t make that clear in my comments. Maybe because I tend to confuse a bit carrier supporting carrier with Inter-AS option C, because as I understand CSC utilizes Inter-AS Option C, or not?

    • Hi Alvaro

      Please check then CSC Carrier Supporting Carrier Architecture. I published it today. I wonder if someone is getting or providing this service.

  • Excellent explanation Orhan!

    • Thanks Jason, now time to become CCDE for you , let me know the result !

  • Anonymous

    Good post many concepts have been cleared!!!

  • Pingback: Inter AS Option B | Cisco Network Design and Architecture | CCDE Bootcamp | orhanergun.net()

  • Pingback: Inter-AS Option AB (Option D) - MPLS VPN Design Consideration | Cisco Network Design and Architecture | CCDE Bootcamp | orhanergun.net()