Seamless MPLS architecture can be used to create large scale MPLS networks, reduce operational touch points for service creation, reduce overall complexity and enable flexible service creation points in the Service Provider networks.
Seamless MPLS architecture is best suited to the very large scale service provider networks that have 10s or 100s of thousands access nodes and very large aggregation networks that want to have a predictable,proven control plane.
IP traffic increases rapidly due to video, cloud, mobile internet, multimedia services and so on. To cope with the growth rate of IP Traffic, we increase our networks capacity but at the same time we have to maintain operational simplicity.
Seamless MPLS architecture is also called as Unified MPLS but there are small differences between them.
Let’s look at what typical service provider networks look like.
Figure 1 : Typical Service Provider Network
Most of the service provider that may benefit from the seamless MPLS. I covered the business models of the 2017 Provider networks in my most recent Service Provider design workshop. You can access all the videos of that course from here.
Access, Pre aggregation, Aggregation, Edge and Core are the common hierarchies in the large scale service provider networks.
You have probably always dealt with them as Edge (PE) and Core (P) devices that you will see as ABR and P in my topology drawings throughout this post.
Typical core nodes are in the rage of 10s, edge nodes 10-100s, aggregation nodes, 100s-1.000s, pre aggregation nodes 1.000-10.000s and access nodes are 10.000s to maybe 100.000 nodes.
I will explain you how these amounts of nodes still run end to end MPLS with single IGP with the seamless MPLS architecture . We will extend MPLS all the way down to the Access nodes.
I explain Access, Metro , Core networks and the many other technologies, protocols and the architectures in Service Provider networks in my Service Provider Design Workshop.
Business Requirements for Seamless MPLS
Reduce the operational touch points
The picture below summarises the MPLS evolution in the Service Provider Networks.
For some service providers, MPLS might be just in the core, but some of them extend MPLS to the Aggregation domain,and some may have MPLS up to the access device but not using seamless MPLS architecture.
Let’s look at the picture below to better understand service creation points today in most service providers networks.
If you want to provision,let’s say a MPLS L2 VPN service, you need to provision an access device, Vlan is configured between access and aggregation. The Sonet/SDH, ATM access environment has already started be consolidated to Ethernet and IP/MPLS in the access network.
With the seamless MPLS, service provision is ideally achieved by configuring only two nodes.
If MPLS is used end to end from access nodes in one aggregation area to another access node in another aggregation area, a unified and predictable control plane reduces the troubleshooting time as well.
It is predictable because the MPLS control plane is robust,mature and highly available. If you don’t use MPLS in the access networks, you would deal with Spanning Tree, G.8032,REP protocols in the access layer.
Many of the access layer protocols for the ethernet have good resiliency characteristics. When the MPLS is introduced to the access networks, this ability should be maintained.
I will show you how they work later in this article.
Seamless MPLS architecture supports IPv6.
But in order to understand the applicability,use case, and requirements that I’ve covered so far this was needed.
Let’s take a look at Seamless MPLS Architecture.
Seamless MPLS Architecture
Our target with a seamless MPLS solution is to have an end to end label switch path from the access node in one aggregation domain to another access node in a different aggregation domain.
In this topology although I show two access and two aggregation domains, Seamless MPLS architecture can scale easily 100 Aggregation domains, 10.000 aggregation nodes in those domains.
As you can see from the below picture, we will divide the MPLS domains and connect them through BGP as a hierarchical manner and create an Inter- domain MPLS LSP.
Intra-Area LSP is created in each domain. IGP runs only on the Aggregation and Core domains.
For the inter-area LSP, all the nodes in the network have to reach loopback interfaces of each other. This is done via BGP.
We have a Single BGP Autonomous System with the seamless MPLS architecture.To have an end to end LSP, BGP + Label which is also known as RFC3107 is used.
In the picture above, you can see control plane operation in a seamless MPLS solution.
PE-1 and PE-2 represent access nodes. P devices are aggregation nodes, ABRs are Edge nodes that connect aggregation to the Core domain.
We use separate IGP areas to minimise IGP size in each domain. Assuming 10.000 aggregation nodes and every aggregation domain consist of 1.000 aggregation nodes, this network would have 10 IGP area + core backbone area.
As you see from the picture, we have three IGP areas (area is a logical concept in this picture, it can be IS-IS level or OSPF area)
The core MPLS domain is placed in a different area t0 the aggregation network areas. If the IGP protocol is IS-IS, then the aggregation domain is placed into an IS-IS level 1 domain and the core IGP is placed into an IS-IS level 2.
Between the areas, routing reachability is achieved through BGP. Nothing is redistributed in IGP. Within the area reachability is achieved through Intra Area IGP.
IBGP sessions is created between PE and ABRs and between ABRs. Since all the sessions are IBGP, we need Route Reflector, otherwise we would have scalability issues. We will have 1000 nodes in each aggregation domain and we would have 450000 IBGP session if you wouldn’t use Route Reflector.
These route reflectors are also an Area Border Router or Level 1/2 routers.
Alternative IGP design, to run separate IGP process on the ABRs for all attach aggregation domain.
Because if you have a OSPF with same process, even if you have NSSA in the aggregation, those routes would be sent into the core if you don’t filter them. Creating an NSSA areas and filtering might be seen as complex.
For the IS-IS case, if you put Aggregation network into IS-IS Level 1 domain, nothing is sent except ATT bit in L1 LSP from level 2 domain into level 1 but from level 1 to level 2 route propagation can be prevented with filtering.
On the ABRs, we redistribute IGP into BGP to send the loopbacks of the routers to the other domains. BGP +Label (AFI/SAFI 1/4) is sent between IBGP neighbors.
One might ask , what about the access devices ?
Answer is no. They will have static default route towards aggregation routers. But if you want to have end to end LSP , those devices have to support at least light weight LDP implementation.
LDP label advertisement can be done in two ways. Downstream Unsolicited or Downstream-on-Demand.
If Downstream Unsolicited label advertisement is enabled on the LSR interface, router sends a label binding for the prefixes to its neighbor without waiting neighbour’s request.
If the Downstream-on-Demand label advertisement mode is enabled router asks label information, only when it is needed.
For the access node, downstream-on-demand label advertisement is used. Aggregation nodes and access nodes label binding information for the necessary loopbacks.
One caveat of having default route on the access node is LDP specification. Normally LDP nodes install label information into LFIB for the /32 prefixes. RFC 5283 which is Inter area LDP extension can be used to overcome this.
Aggregation nodes should have static route for the access nodes loopbacks and those static route should be redistributed into the Aggregation domain IGP for the Intra domain (Transport Label) LSP.
Access device loopbacks , same as aggregation domain loopbacks, redistributed from IGP into IBGP on the aggregation nodes. In our topology aggregation node is the ABR but for the larger scale , we have separate Aggregation nodes in addition to ABRs.
Let’s see end to end packet flow with the dataplane interaction.
At the headend access node, we have 3 labels in the stack. Top most IGP label comes from LDP to carry the traffic to BGP next hop which is ABR1, ABR1 assign a BGP label for the PE2 loopback.
When the packet comes to the ABR1, ABR1 swaps its own BGP label with the BGP label assign by ABR2.Both ABR1 and ABR2 is BGP route Reflectors but we also set BGP next-hop self on this RR to achieve scalability.
If ABRs would be a route reflector PE devices would have end to end Transport LSP which is created by LDP for the loopbacks of every other PEs. But since each IGP domain is separated and not redistributed to each other, this is not possible.
When BGP next-hop self is configured on the Route Reflector, since the bgp next hop will be a Route Reflector which are also an ABR, every PE will have only two labels for the Route Reflectors. ( I assume for the redundancy we have more than one RR/ABR ).
When ABR1 swaps the BGP label, ABR1 pushes new transport label which is assigned by LDP to reach ABR2.ABR2 is the next hop for the remote PEs which are access devices in our topology.
ABR2 is the penultimate hop from the BGP point of view and it does PHP for the BGP assigned label. It pops the BGP label, push the transport label assigned by the local LDP.
PW label is kept as end to end.
With seamless MPLS if you want to create an MPLS Layer 3 VPN service end to end, it can be done by configuring only two PEs. No PW stitching,no different LSP segments, no VLAN traffic engineering and so on.
Otherwise bringing BGP + Label into design might be seen as complex.
There is no Standard or Informational RFC for the Seamless MPLS architecture but all the element especially for the availability requirements, are supported by Cisco. If other vendors have similar solution,let me know in the comments box below.
3 labels are enough to have LSP hierarchy for Seamless MPLS to support 10s of thousands of devices in the network.
But definitely it reduces the services touch points so to have end to end service, instead of configuring,managing,troubleshooting 5 to 10 device, seamless mpls ideally achieves to service creation with only two devices.
Having separated IGP areas is must. Only the loopback interfaces of PE,Aggregation and ABR devices are known.
P devices and transit link information is not required. At least not all the devices have to know P devices and link information. Still for the monitoring purposes they can be carried to the management system.
Do you think that your company can benefit from Seamless MPLS ?
Let’s talk as always in the comment box below.