Total 9 Blogs

Created by - Orhan Ergun

Introduction to MPLS - Fundamentals of MPLS

MPLS Multiprotocol Label Switching is one of the most popular and commonly used technologies in today's Service Provider and Enterprise networks. In this post, we will explain the most fundamental topics about MPLS. After reading this post, you will learn a lot about MPLS, why we should use MPLS to MPLS packet formats, USA cases of MPLS to MPLS advantages and MPLS disadvantages, some recommendations about MPLS books, MPLS training, some basics MPLS questions, and many other things will be covered. Sit tight and let's enjoy!. What is MPLS in Networking? Multiprotocol Label Switching - MPLS, is a networking technology that switch the network traffic using the shortest path based on “labels,” rather than IP destination addresses, to handle forwarding over a private Wide Area Network. MPLS is a scalable and protocol-independent solution, that can carry Layer 3 IP and Non-IP and Layer 2 traffic, PPP, HDLC, Frame-Relay, Ethernet, all are possible. MPLS provides transport and can be considered one of the tunneling mechanisms. MPLS transport protocols as of 2022, are LDP, RSVP, Segment Routing and BGP LU. An MPLS network is Layer 2.5, meaning it falls between Layer 2 (Data Link) and Layer 3 (Network) of the OSI 7 layer model hierarchy. When MPLS was invented for the first time, the reason was faster packet processing. The common belief was label switching would be faster compared to IP destination-based lookup. Businesses use MPLS to connect remote branch offices that require access to applications that reside in the organization's data center or company headquarters. Service Providers use MPLS in their network to scale their network and to connect thousands, if not tens of thousands of their customers' locations. What is MPLS used for, Why MPLS is used? MPLS is used to create a transport network actually. It provides an underlay medium for overlay services. The main services that we run with the MPLS are: Layer 2 MPLS VPN with Pseudowires (VPWS, VPLS) EVPN Layer 3 MPLS VPN Inter-AS MPLS VPN Carrier Supporting Carrier MPLS Traffic Engineering with RSVP and Segment Routing RSVP-FRR, TI-LFA Seamless MPLS/Unified MPLS These are some of the reasons/use cases we have MPLS in the networks. What MPLS network consists of? MPLS network consists of  three different types of devices: MPLS PE Router: PE is a Provider Edge device. In MPLS networks, all the intelligence is at the edge. The core is kept as simple as possible. KISS principle in network design comes from the ‘ Intelligent Edge, Dummy Core ‘ idea. PE device looks at the incoming frame or packet and identifies which egress PE device is used for transport. A second lookup is made to determine the egress interface on the egress device MPLS CE Router: CE is customer equipment that can be managed by Service Provider or Customer depending on SLA. It resides in the customer network and doesn't run MPLS with the PE router. The only exception is CSC - Carrier Supporting Carrier Architecture, in that case, LDP or BGP runs between CSC-CE and CSC-PE nodes, otherwise, in any other MPLS service, the CE router doesn't run MPLS. MPLS P Router: P is the Provider device and only has a connection to the MPLS-enabled devices. P device doesn’t have a connection to the customer network. Its main job is to connect the PE devices and provide reachability between the PE nodes. MPLS network can run without P nodes. In that case, the scalability of the MPLS network might be an issue. If the MPLS network runs without P routers, then the LSP - Label switch path is referred to as One-hop LSP.   Figure - MPLS Network nodes/elements MPLS Header MPLS Header is 4 bye - 32 bits field. First 20 bits for MPLS Label, 3 bits for EXP, 1 bit for Bottom-of-stack, and 8 bits for TTL purposes. Labels 16 - 100,000 are in the default range used by Cisco devices. Each router's label range can be specified with the 'mpls label range command'. MPLS Label stack has 4 parts! The MPLS Label consists of four parts: The Label - 20 bits The label holds all of the information for MPLS routers to determine where the packet should be forwarded. It is 20-bits long, thus 1,048,576 labels can be assigned in the MPLS network. Sometimes this amount of labels may not be enough but in this post, we won't cover it. MPLS Experimental - EXP bits - 3 bits Experimental bits are used for Quality of Service (QoS) to set the priority that the labeled packet should have. In DSCP we have 6 bits for QoS, in layer 2 802.1p we have 3 COS - Class of Service bits for QoS and in MPLS, we have 3 bits EXP field. When QoS is done in the MPLS network, COS to EXP or DSCP to EXP mapping is done. Based on the MPLS DiffServ Tunneling mode, Uniform, Short-Pipe, and Pipe Model, EXP bit mapping would be different. Bottom-of-Stack - 1 bit  The Bottom-of-Stack tells MPLS routers whether there are no more labels in the label stack. The bottom--of-stack bit is a field that is set to 1 for the last MPLS header. For example, with MPLS VPN the VPN label will have the bottom-of-stack label set to 1, which tells the MPLS router to process the embedded transport protocol. This bit in some resources is referred to as S-bit. MPLS Time-To-Live - 8 bits This identifies how many hops the packet can make before it is discarded. MPLS TTL, similar to IP header, is an 8-bit value. Similar to the IP header, the TTL field is used to prevent infinite forwarding loops of MPLS frames. Max value is 255 because it is 8-bits. The TTL field can be used for path tracking like MPLS Traceroute. MPLS Reserved Labels Reserved labels 0 - 15 have a special meaning in MPLS Label 0 - Explicit Null in IPv4 - The egress LSR tells the neighboring LSRs to forward the packet keeping the explicit null label (0). The egress router strips the label, paying attention to the QoS value, and makes the IP lookup, without doing a lookup on the label. The biggest advantage of explicit null is transferring the QoS information. Label 1 - Router Alert - the label that informs the LSR to look at the packet using software instead of forwarding in hardware. This is mainly used for traceroute purpose Label 2 - Explicit Null in IPv6 - Same as Label 0 in IPv4 but Label2 is for IPv6 Label 3 - Implicit Null label which is used for Penultimate Hop Popping - PHP purpose. The egress LSR tells the neighboring LSR to pop the topmost label before forwarding to the egress LSR. This also removes the EXP bits which may not be ideal when using MPLS DiffServ Tunneling modes(Uniform and Pipe models). The benefit of doing implicit Null is that egress LSR does not have to do the lookup on the label, strip it, and then lookup IP forwarding. It is done to improve the performance of the network but MPLS TP (Transport Profile) for example, we need the topmost/outer label end to end. MPLS OSI Layer MPLS in OSI Layer is considered as Layer 2.5 As you might know, Ethernet is Layer 2 in the OSI Layering model and IP is Layer 3 based on OSI. MPLS header is placed between Ethernet and IP, meaning between Layer 2 and Layer3, thus MPLS is commonly referred to as Layer 2.5 technology. What is MPLS Connection? Any circuit, layer 2 or layer3, that connects the device to another device for MPLS service to be carried is called MPLS Connection. Over the circuit MPLS, with LDP or RSVP doesn't need to run. The circuit might be an Ethernet and MPLS Layer 2 VPNs can run on top of it. Or Circuit (Link), can be Layer 3 and IP routing might run between the two end-points, and it can support MPLS Layer 3 VPNs. So, MPLS Connection is an underlay connection/transport which provides a medium for the overlay MPLS service. How does MPLS work? MPLS works based on 3 operations. MPLS Label Push, Swap, and POP. Ingress (First node) router does the IP destination-based lookup, assigns a label to the packet, and mid routers change this label towards the Egress router, and Egress router POP all the MPLS labels and forward the packet to the destination. The first device does a routing lookup, just like before in IP Routing But instead of finding a next-hop, it finds the final destination router. And it finds a pre-determined path, called Label switched path,  to that final router The router applies the MPLS label based on this information. Future routers use the label to forward the traffic Without needing to perform any additional IP lookups At the final destination router, the label is removed And the packet is delivered via normal IP routing. MPLS Router Roles Label Edge Router - LER or ingress node: The router first encapsulates a packet inside an MPLS LSP. Also, the route which makes the initial path selection. Label Switching Router - LSR or transit node: A router that only does MPLS switching in the middle of an LSP. Egress Node The final router at the end of a Label Switch Path - LSP, which removes the label What is MPLS Label Swapping? Mid-LSR, Label Switch Router only replaces the incoming label with the outgoing label. So it receives a label from its downstream node, to reach the final destination and advertises another label to its upstream for the same destination. Let's say it receives Label 10 to reach the destination/egress router, it assigns label 20 and advertises to its upstream router. Whenever its upstream router sends the packet with Label 20, LSR swaps/replace the label 20 with label 10 and sends it towards the final destination. What is MPLS Push Operation? Ingress PE router, which is the first router in the MPLS domain, does the IP lookup and assigns a label for the final destination. Assigning a label is called PUSH. Basically, it is adding a label to send the traffic towards the Egress PE router. What is MPLS POP Operation? MPLS POP means basically removing the MPLS labels. The topmost label can be removed if there is PHP, otherwise, MPLS Labels are carried all the way to the egress router and it POPs/removes the MPLS labels and forward the traffic towards the correct MPLS CE interface. What is MPLS Penultimate Hop Popping - PHP? Egress LSR, in order to improve the performance of the network, can send the Implicit Null labels which were explained earlier in the post. The benefit of doing implicit Null is that egress LSR does not have to do the lookup on the label, strip it, and then lookup IP forwarding. This process is called Penultimate Hop Popping. A weird name. But basically, next to the last-hop router, remove the topmost label. Only a service label/VPN label packet might have if MPLS VPN is enabled and the Egress router doesn't have to do a double lookup, one for MPLS and one for IP. Where MPLS PHP is used? Almost in any MPLS application, MPLS PHP is used by default. MPLS Layer 2 VPNs, MPLS Layer 3 VPNs, RSVP-TE, RSVP Fast Reroute, PHP is used. Where MPLS PHP is not used? MPLS Transport Profile - TP, requires an end-to-end label for OAM purposes. Also, when the topmost label needs to be carried for QoS information, Explicit Null is sent to preserve the topmost label header. Thus in general, QoS and MPLS TP don't have MPLS PHP. What is MPLS FEC? Wikipedia's explanation for MPLS FEC is, it is a forwarding equivalence class (FEC) is a term used in Multiprotocol Label Switching (MPLS) to describe a set of packets with similar or identical characteristics which may be forwarded the same way; that is, they may be bound to the same MPLS label. MPLS FEC can be identified by address, tunnel, or CoS - Class of Service. Typically, a device assigns the same label to one MPLS FEC. The traffic of one FEC is forwarded in the same mode and through the same path. However, not all packets with the same label belong to the same FEC. The EXP values of the packets may be different. Therefore, they are processed in different ways and belong to different FECs. Because the ingress LSR needs to classify packets and add labels to the packets, it is responsible for determining the FEC to which packets belong. MPLS FEC Examples: Unicast Packets with the destination IP addresses match the same prefix. Multicast packets belonging to a specific multicast group. Packets that are processed in the same mode based on the process or the IP DSCP field. MPLS Label Signalling Protocols: MPLS Labels can be assigned by 4 protocols currently. LDP, RSVP, BGP, and Segment Routing. For the Service layer/Overlay MPLS Label, SR and RSVP are not used. The service layer which is also referred to as Overlay, LDP, and BGP is used. When LDP is used for the service layer, it is called Targeted LDP - tldp. When LDP is used for transport, sometimes it is referred to as Directed LDP. Underlay/Transport MPLS Label signaling can be done based on LDP, RSVP, Segment Routing, and BGP. BGP here, basically a BGP LU - Labeled Unicast. MPLS Switch and MPLS Routing MPLS is a switching technology. Switching is done based on MPLS Label. But, MPLS with an IP control plane, requires a routing protocol to set up an underlay transport network. For MPLS nodes to communicate with each other, underlay routing needs to provide reachability. Static routing or any dynamic routing protocols can be an underlay routing for MPLS. MPLS Internet Many companies want to have Internet Access with SLA, but unfortunately, this is not possible. Internet is a best-effort service, meaning there can't be Packet Loss, Delay, and Jitter guarantee and Service Providers cannot give an SLA - Service Level Agreement to their customers. MPLS on the other hand can provide SLA for availability, packet loss, latency, jitter, and many other criteria. MPLS and Internet are totally different services. Over the Internet, VPN can be created let's say via GRE, mGRE, or DMVPN technologies and MPLS can run over those technologies. So, MPLS cannot run directly over the Internet, which is a public network, but it can run over some other private networks. Not all though. For example, MPLS cannot run over GETVPN, although GETVPN is an overlay VPN, there is no tunnel with GETVPN, thus MPLS or routing protocols cannot run over GETVPN. is MPLS Layer 2 or Layer3? MPLS doesn’t fit neatly into the OSI seven-layer hierarchy, thus MPLS is not Layer 2 or Layer 3 in OSI layering. Although the Network Engineering community has been discussing whether the OSI layering is suitable for the many protocols for definition, if we would fit MPLS somewhere in the OSI layer, it is considered Layer 2.5 Because the MPLS header is placed between Layer 2 MAC and Layer 3 IP Headers. Thus, MPLS is commonly referred to as Layer 2.5 protocol. Figure - MPLS is layer 2.5 Source: www.mplsinfo.org MPLS Extra StudyTutorials MPLS Recommended Books Network Convergence: Ethernet Applications and Next Generation Packet Transport Architectures Definitive MPLS Network Designs (Networking Technology) MPLS-Enabled Applications: Emerging Developments and New Technologies 3rd Edition MPLS Related Blog Posts Making the case for Layer 2 and Layer 3 VPNs Scalable VPLS Architecture Juniper MPLS Based Layer 2 VPNs Understanding MPLS VPNs Jeff Doyle MPLS Training Suggestions We strongly recommend MPLS Training with Cisco from Orhan Ergun for MPLS Training. This training comes with more than 30 hours of network design and 40 hours of hands-on practical labs using Cisco routers and switches. Network design examples in MPLS training are vendor-neutral, meaning applicable to every vendor. Also, MPLS VPN with Juniper Network Training, explains the MPLS Layer 3 VPN, MPLS Layer 2 VPNs, and EVPN by using Juniper network equipment. is MPLS Point-to-Point? Actually, MPLS depends on the protocol that we used for labeling can be a point to point, point to multipoint, or multipoint to point. If we use regular LDP, it is Multi to Point, which is used in IP Unicast transport networks. If MPLS is used with mLDP - Multipoint LDP, when it is Point to Multipoint or Multipoint to Multipoint and mLDP is used in MPLS Multicast. if regular RSVP is used, then MPLS is a point-to-point, and RSVP is used in IP Unicast transport as well. Last but not least, if RSVP is used for MPLS Multicast, then MPLS would be considered as P2MP - Point to Multipoint. How many Labels do MPLS Layer 3 VPNs have? MPLS Layer 3 VPN has two labels. Most MPLS operation requires a minimum of 2 labels. In MPLS Layer 3 VPN, Transport, and BGP label In Layer 2 VPN, based on pseudowire technology, MPLS labels are Transport and VC Label. What is the most common MPLS use case in 2022? As of 2022, the most common MPLS use case is MPLS VPNs. MPLS Layer 3 VPN and MPLS Layer 2 VPNs are the most common reasons that networks deploy MPLS technology. MPLS Fast Reroute would be considered the second most common use case for MPLS.

Published - Wed, 13 Apr 2022

Created by - Orhan Ergun

MPLS Benefits 4 Very important things to understand!

MPLS Benefits and Advantages, Network Engineers should understand MPLS. In this post, we will look at what are the benefits of deploying MPLS in the Network, and the advantages of having MPLS-enabled infrastructure. MPLS is Multi-Protocol Label Switching as you might know already. Multi-Protocol because we can carry many different types of traffic over MPLS. MPLS is Multi-Protocol Technology Layer 2 and Layer 3 network traffic Ethernet, Frame Frame-Relay, ATM, TDM different types of traffic was carried over MPLS. Because it provides an abstraction layer for the protocols, it is possible to carry many different types of traffic that couldn't be possible with other technologies easily. MPLS is a Scalable Protocol If we talk about MPLS benefits, probably one of the most important ones would be MPLS Scalability. There is a popular belief that MPLS was invented because the packet processing resource requirement and lookup speed are faster with MPLS, compare to IP destination-based lookup. Because MPLS is just a switching operation on the Mid-Label Switch Routers - LSR, and MPLS Label is 20 bits long, compared to IP which is 32 bits long with IPv4 and 128 bits long with IPv6, MPLS was considered a better performance protocol, thus allows to grow, without adding extra device resources. MPLS has many Applications/Use Cases One of the main advantages of MPLS is it has many different use cases and services. Ina Minei went through these services in his MPLS Enabled Applications book. Figure - MPLS Enabled Applications MPLS Layer 2 based VPNs, LDP and BGP based, then MPLS Layer 2 VPN with EVPN, Classical EVPN for both Layer 2 and Layer 3 VPNs, 2457-based MPLS Layer 3 VPN, Carrier Supporting Carrier, Seamless/Unified MPLS, Segment Routing, Inter-AS MPLS VPNs, GMPLS, RSVP based MPLS for Traffic Engineering, RSVP-TE FRR, SRTE and Topology Independent Loop-Free Alternate, and the list goes on. All of the above architectures, services, and frameworks work because the underlying mechanism is MPLS. Some of them can work with other mechanisms as well but could it be secure as MPLS'?. Could they be scalable as MPLS?. Are they Multi-Protocol?. MPLS can provide Fast Reroute One of the MPLS benefits can be considered Fast Reroute. Fast Network Convergence is a very critical and desired behavior as of 2022 by almost any type of Network and Fast Reroute as one of the data plane protection techniques, can provide 50 ms. convergence time. This can be possible with MPLS technologies such as RSVP and Segment routing. MPLS can support Constrained-based Explicit Routing By using RSVP and Segment Routing with Path Computation Element (PCE), MPLS can support explicit, constrained-based routing. Meaning, that on the head-end router, the entire path can be defined with the user/network administrator constraint. You can request, 100Mb LSP between Point A and Point B, maybe based on time of the day, etc. Many different types of constrained can be defined and these are not again easily, in a scalable way possible with other technologies. Although there are many other MPLS benefits, to keep the short post, I will end the post here, but please check the other posts on the website as we have thousands of them and many also in MPLS topics. pros and cons of mpls, advantages of mpls over internet, mpls pros and cons, mpls vs vpn pros and cons, mpls benefits for business, mpls network, route traffic, mpls routing, network congestion, mpls networks, point to point connectivity

Published - Fri, 08 Apr 2022

Created by - Orhan Ergun

What is Deadlock situation in MPLS Traffic Engineering ?

What is deadlock situation in MPLS Traffic Engineering ? What happens when deadlock occurs ? Is there any mechanism to prevent deadlock ? I will explain all the details in this post. Deadlock occurs when LSP needs to move to the other link but due to lack of available bandwidth cannot move to the other links. I will show you the case with the below topology. LSP Dead Lock Figure -1 Dead Lock Problem in Distributed MPLS Traffic EngineeringThere are two RSVP signaled MPLS TE LSP in the above figure. Red LDP is signaled with 400 Mbps and Blue LSP is signaled with 400 Mbps capacity. Link capacities are 1000 Mbps , except P1 – P3 , P2 – P3 , they are 500 Mbps. Reservation with RSVP is done at the control plane. RSVP doesn’t take actual link utilization in the dataplane. Which means, if you send 600Mbps traffic over 400Mbps signaled LSP, traffic is not dropped. If there is no utilization on the physical link, you wouldn’t see any problem. Control and data plane is not synchronized by default in MPLS Traffic Engineering deployment with RSVP signaling. On the above figure, let’s say actual traffic utilization reaches to the 800 Mbps on the Red LSP. And usage on Blue LSP is still 400 Mbps.  Two of these LSPs combine, 1200Mbps traffic is sent down over 1000Mbps link. Thus , the traffic over both of the LSPs will be affected. So, RED LSP’s traffic increase, affects Blue LSP as well.   In distributed Traffic Engineering, two futures are used to avoid deadlock problem.   LSP priorities and the Auto Bandwidth.   With Auto Bandwidth, routers check the actual interface usage, so the data plane traffic and adjust the RSVP control plane accordingly. So, LSP is resized. But without priority, even when resizing the Red LSP to 800 Mbps, Blue LSP is not moved to the bottom path.   Both priority and Auto bandwidth would work in this topology. But what if Blue LSP’s priority is better and it tries to force Red LSP to move to an alternate link. So solution gets very complex.   Thus, the better solution for the deadlock problem is centralized approach. If you would have a centralized node which knows the real time topology information , traffic demand and the active LSPs in the network, centralized nodes would move the LSPs accordingly to an alternate links. Centralized nodes would take the latency, bandwidth and the many other constraints for the LSPs into an account while placing the LSPs.

Published - Tue, 26 Nov 2019

Created by - Orhan Ergun

Bin Packing Problem of Distributed Traffic Engineering

Bin Packing Problem ? What is Bin Packing ? I will explain in this post Bin Packing Problem in MPLS Traffic Engineering. Very complex post normally but I will make it simple for you. And trust me, it is important to understand. Before I start explaining Bin Packing problem, let’s just remember the purpose of MPLS Traffic Engineering. Very easy, MPLS Traffic Engineering is deployed to use all available capacity on the network, or maximize the capacity usage of the entire network, so at the end ‘ Money ‘ ! If there will be any idle link, due to SPF shortest path selection maybe, MPLS Traffic Engineering gives us to ability to utilize those under utilized or no utilized links as well. I explain couple other main reasons of MPLS Traffic Engineering in this post, but just remember that, Maximizing the capacity usage of our networks is primary technical goal. Bin Packing problem is just the opposite. Although the purpose of MPLS Traffic Engineering is to maximize the usage by sending the traffic to the all possible paths optimally, as you will see from the below example, this is not always the case unfortunately.     Figure -1 LSP Blocking Due to Bin PackingIn the above topology, PE1 wants to signal RSVP LSP to the PE3. PE1 wants to signal with 300Mbps capacity. 300 Mbps from PE1 to PE3, can be signal through two paths. First one is PE1 – P1 – P2 – PE3 and the second one is PE1 – P1 – P3 – P2 – PE3 Obviously the second one is longer path (IGP cost is higher over the bottom path), thus PE1 to PE3 LSP is setup over the red LSP (PE1 – P1 – P2 – PE3) When this LSP signaled, over the top path, only 700 Mbps link capacity remains. Bottom path haven’t been used yet, thus bottom path has 500 Mbps capacity. So far, so good, no problem ! But, PE2 will not stay there without any traffic forever. And here it is. PE2 wants to setup 800 Mbps RSVP signaled LSP to PE3. From PE2 to PE3, physically two paths are available.These are; PE2 – P1 – P3 – P2 – PE3 and PE2 – P1 – P2 – PE3. Although there are two available paths from PE2 to PE3, none of them can be used. Because, top path capacity is only 700 Mbps, because 300 Mbps is used by the RED LSP (PE1 LSP). Bottom path cannot be used, because maximum available capacity is 500Mbps. This is Bin Packing Problem. If PE2 LSP request would come first, it could use only top path and when the PE1 request comes, PE1 LSP could be setup over the bottom path. But there is no coordination between the PE 1 and PE2. They don’t talk each other and say, hey here is my traffic demand, you should use the top path and I will use the bottom path, so on and so forth. PE1 request came first, it’s demand is satisfied over the IGP shortest path and Red LSP is signaled. If order would be different, at least in this topology, we wouldn’t have an issue. This request (Bandwidth in this case) ordering problem is called ‘ race condition ‘ problem. Whoever comes first, He gets the bandwidth baby ! How Bin Packing Problem can be avoided ? Do we have a solution for this ? Fortunately yes. First solution is LSP Priority. By giving a higher priority to the PE2 LSP, when the request comes, PE1 LSP is moved to the bottom path. Second one is changing the computation mechanism. Note: Below part will be a bit more technical, promised to keep it as simple, but might be harder, let’s try. Distributed path computation with MPLS is done with CSPF (Constrained Based Shortest Path First Algorithm). Each router knows the traffic demand of itself and it attacks to the available bandwidth. They don’t know the traffic demand of the other routers. But don’t confuse traffic demand with network topology. They of course know the each other topology, because every router in an Area or Level (OSPF and IS-IS respectively) has the same LSDB (Link State Database) and TED (Traffic Engineering Database, TED is only when Traffic Engineering is enabled) Since they don’t know each other traffic demands, if there would be a centralized node which talk with these routers and learn the traffic demands and the network topology, it could calculate the path on behalf of all the routers and tell to the routers which path they should use for their LSPs. I think some of you are saying that it is SDN approach and that’s correct. In case of MPLS networks, PCE (Path Computation Element) with some extensions, exactly does that. By having centralized traffic demand view, we would avoid Bin Packing Problem, thus would be able to signal required LSP, timely and over an optimal paths. End result would be as below for the above topology. Figure – 2 Optimal Bin Packing – No Problem for the LSP DemandsPE1 is signaled through the bottom path (Red LSP) and PE2 is signaled through the top path (Blue LSP) 200Mbps (1000- 800Mbps demand of Blue LSP) remains at the top path , 200 Mbps (500 – 300 Mbps demand of Red LSP) remains at the bottom path. No need to have complex priority schema in this case.

Published - Tue, 26 Nov 2019

Created by - Orhan Ergun

Inter AS Option A Design Considerations and Comparison

Inter-AS Option A is the easiest, most flexible, most secure Inter autonomous system MPLS VPN technology. I am explaining this topic in deep detail in my CCDE Training Bootcamp and CCDE course. In the below topology VPN Customers A and B are connected to two different service providers via MPLS Layer 3 VPN. In order to have end-to-end MPLS VPN service, Service Providers use special mechanisms. In this article, I will explain the most basic one which is Inter-AS Option A, though there are many other things you need to know about Inter-AS Option A. Our aim is to carry out all the customer routes between the service providers. There are many different ways of handling this case. In this post, I will explain Inter AS Option A MPLS/VPN, also known as the VRF-to-VRF approach. Figure 1: Inter-AS Option A I will use the topology depicted in fig. 1 throughout this post to explain Inter AS Option A operation. In the above diagram, we have two service providers and two customers which require Inter-AS MPLS VPN service. The PE routers that connect the two providers are also known as ASBR (Autonomous System Boundary Router). Inter-AS Option A: an ASBR router in one Autonomous System attaches directly to an ASBR router in another Autonomous System. The two ASBR routers are attached to multiple sub-interfaces; at least one of the VPNs whose routes need to be passed from one AS to the other AS is attached. In addition, those sub-interfaces associated with the VRF table. For each customer, service providers could use the separate physical connections, instead of a sub-interface. However, doing that would not produce an optimal results for resource utilization. PE routers connected to the CE devices run MP-IBGP, either through full mesh or through RR (route reflector). Inter-AS Option A allows ASBR routers to keep all the VRFs for customers who require Inter AS service. SP-A and SP-B ASBR routers maintain a VPN forwarding table in LFIB. Furthermore, they keep routing information in RIB and FIB. Compared to other AS options, ASBRs have high memory usage in Inter-AS Option A. However, other Inter AS VPN options does not have these capabilities. ASBRs can either run the same routing protocol with the customer on the VRFs or use just EBGP. For example, if the requirement for customer A, is to keep routing information, such as metric end to end, SP-A and SP-B run the same routing protocol on the ASBRs and the PE devices to where the Customer CE device is attached to. SP-A and SP-B: Inter AS Option A will have to manage redistribution at the ASBRs because the routes appear to the ASBR as BGP routes from remote PEs. For customer A, those routes need to be redistributed from BGP to Customer A PE-CE Routing protocol, and from the Customer A PE-CE routing protocol to BGP. ASBRs associate each such sub-interface with a VRF. Inter-AS Option A does not require MPLS at the ASBRs unlike the other Inter AS options. Since we need to have a separate VRF and sub-interface for each customer VPN, separate routing protocol, and deal with redistribution for each protocol, it is operationally cumbersome and thus hard to scale. Among all other Inter AS options since there is only IP routing between the AS, Option A is considered as a most secure one. In addition, it is the easiest option to implement between the AS because Option A does not require another control plane mechanism between service provider ASBRs such as LDP, BGP+label, or BGP-VPNv4. Between the service providers on ASBRs, either IGP protocols or EBGP are used. More importantly, other Inter AS Options (Inter-AS Option B and Inter-AS Option C) require additional control-plane protocols to advertise the customer or infrastructure prefixes between the ASBRs. Since only IP traffic passes (Not MPLS) between the service providers, most granular QoS implementation is achieved with Option A. (Per sub-interface and IP DSCP vs. MPLS EXP) For all Inter AS Options, it is very common that customers to have to trust the service provider for data integrity, confidentiality, and availability. MPLS does not encrypt the packets because if a customer needs end-to-end encryption, the user can deploy an IPSEC. The below Inter-AS MPLS VPN Options Comparison Table gives you the most comprehensive analysis. In real life and also for the design exams it will be very useful since the comparison is done from the design point of view. Figure - Inter-AS MPLS VPN Options Comparison Do you use MPLS VPN service? Is it from one provider or multiple providers? Let's talk about your design in the comment section. To have a great understanding of SP Networks, you can check my newly published SP Book. It covers the SP network Technologies with also explaining in detail a factious SP network.

Published - Tue, 26 Nov 2019

Created by - Orhan Ergun

Inter AS Option B – Design Considerations and Comparison

Inter-AS Option B is a highly scalable, reasonably secure, but operationally complex inter-autonomous MPLS VPN architecture. For those who want to understand the easiest, most flexible, and most secure Inter-AS solution, which is Inter-AS Option A please click here. I am explaining this topic in deep detail in my Onsite CCDE, Live/Webex CCDE, and Self-Paced CCDE course. First, I will explain how Inter-AS Option B works. After that, I will cover the design considerations for Inter-AS Option B. I will use fig.1 to explain Inter AS Option B. Figure 1 : Inter AS Option B In the topology shown above in fig.1, AS10 and AS20 are the service providers. There are two customers, VPNA and VPN B NOTE: All the customers of the service providers could run different routing protocols at different sites. For example, while customer VPNA connected to AS10 is running OSPF, VPN A connected to AS20 could run EIGRP. In the service provider network, PE routers are connected to the customer locations. PE routers run MP-BGP with route reflectors to advertise VPNv4 prefixes. PE router that connects the two providers at the edge is known as ASBR (Autonomous System Boundary Router). Since ASBR runs MP-BGP with RR as well, ASBR learns all the VPN routes from Inter-AS customers. As for Inter-AS Option A, separate sub-interfaces are enabled, and IGP protocols run between ASBRs. MP-BGP is redistributed into IGP. Inter-AS Option B does not require separate sub-interfaces and different IGP protocols or BGP per sub-interfaces between the ASBRs. Between the ASBRs, MP-BGP enables the VPNv4 address family. ASBRs (Autonomous System Boundary Routers) advertise the customer prefixes which are learned from the local BGP route reflector to the other service provider through MP-BGP VPNv4 session. You do not need to keep the VRF table for the customers on the ASBRs. That is why there is no VRF on PE, as shown in fig.1. You just need to keep the customer prefixes in the LFIB database. CPU and memory requirements for Inter-AS Option B are much less than those of Inter-AS Option A. PE devices have BGP VPNv4 sessions with the Service Provider's Route Reflectors. PE redistributes customer VPN prefixes fromPE-CE routing protocol to MP-BGP and vice versa. Route-target extended community is placed on the VPNv4 update. Route Target helps to place the VPN prefixes in the appropriate VRF. When PE receives the customer IP prefixes, it changes next hop as themselves, Customer IP prefixes are made VPN prefixes by adding Route Distinguisher and a VPN update is sent to the VPNv4 route reflector. The route reflector does not change the next hop; rather, it only reflects the route to the ASBR. (This is normal BGP RR behavior but it is different in Option C) ASBR does not place the MP-BGP route into VRF, since it does not have to keep the VRF table but to maintain the VPNv4 BGP table in Inter-AS Option B. By changing the next hop, ASBR from SP-A sends VPNv4 prefixes through the MP-BGP session to SP-B ASBR. SP-B ASBR sends the customer prefixes to its local route reflector. The next hop is changed on ASBR. (Normally EBGP to IBGP, next hop doesn't change). Thus, either ASBR to ASBR link is redistributed into the global IGP table of service providers, or "next-hop-self" is enabled in order to change the next hop as ASBR. Route reflector in the SP-B domain reflects the prefixes as is, send to the PE which is connected to Customer A2 location. SP-B PE sets the next hop again and sends the prefixes to the customer A2 router. As shown in the service provider domains, there are three LSP. Because changing the BGP next hop terminates the LSP, a new VPN label is assigned on that router for all VPN prefixes. Caveat : Normally ASBR would not accept the VPN prefixes because they do not have VRF for the customers. Since ASBRs do not have a VRF and the Route Target is not associated with the VRF in Option B, they would reject the prefixes. As shown in fig.1, "no bgp route-target filter" configuration knob is enabled. This configuration command allows the ASBRs to accept the VPN prefixes even though they do not have a route target associated with VRF. Inter-AS Option B requires MP-BGP between autonomous systems. The VPN address family is enabled on the MP-BGP session. Inter-AS Option B does not require LDP or IGP protocols; thus service providers do not need to understand the internal addressing structure of one another. LSP is not end-to-end but it is terminated at many points; as a result, it is very difficult to manage. Similar to Inter-AS Option A, you do not need to redistribute VPN prefixes at the ASBR. Route reflectors store all the VPN routing information for each customer, and they advertise those prefixes to ASBR. Operators need to manage MP-BGP on the ASBRs as well as on the route reflectors. What if by removing those prefixes from ASBRs, can we advertise all the VPN prefixes directly between route reflectors? To answer this question, peruse the next article on Inter-AS Option C. To have a great understanding of SP Networks, you can check my newly published SP Book. It covers the SP network Technologies with also explaining in detail a factious SP network.

Published - Tue, 26 Nov 2019

Created by - Orhan Ergun

Carrier Supporting Carrier – CSC

CSC Carrier Supporting Carrier is a hierarchical MPLS VPN architecture between the Service Providers.   Service is an MPLS VPN service mostly but doesn’t have to be as you will see throughout the post. I am explaining this topic in deep detail in my Instructor Led CCDE and Self Paced CCDE course. Customer carrier ( Provider ) receives an MPLS VPN service from the Core/Backbone carrier. Although CSC architecture is not common in real life, if you are studying for the CCDE , CCIE Service Provider, JNCIE exams, you should know this architecture. We have two Service Providers at least. Can it be three ? Yes If two different customer carriers setup their Inter-AS MPLS VPN over a Core/Backbone Carrier,although architecture get really complex, it is possible. (Inter-AS Option C would be an ideal in this case) Carrier Supporting Carrier also an Inter-AS MPLS VPN solution but it tries to solve different problem. Let’s remember what we were trying to accomplish with all the Inter-AS MPLS VPNs with the very high level overview picture.Customer’s 2 sites are connected to 2 different service providers. Let’s look at what problem Carrier Supporting Carrier Architecture solves.Customer’s 2 sites are connected to the POP locations of a Service Provider which receives an MPLS VPN service from another Service Provider. In the above picture from the SP-1 point of view, SP-2 just provides a reachability between the SP-1 devices. SP-2 doesn’t know anything about SP-1’s customer prefixes.This brings to solution a scalability. SP-2 just carries Service Provider -1 routers loopbacks, not even infrastructure addresses. I will talk about what would be the better solution rather than Carrier Supporting Carrier at the end of the article but let’s look at how Carrier Supporting Carrier works, in detail.We have two Service Providers, SP-1 and SP2. SP1’s customer has two location. R1 in location 1 is connected to SP-1 Site-1 router R1 which runs EIGRP. R11 in location 2 is connected to SP-1 Site-2 router R9 which runs EIGRP . SP-1 has OSPF in its both location as core IGP. SP-2 which is a backbone carrier has IS-IS as its core IGP. LDP is a label signalling protocol for both Service Providers. ( It could be RSVP as well ) SP-1 is called as a Customer Carrier and SP-2 is called as a Backbone or Core Carrier. R4 is a VPN route reflector in SP-1 Site 1. R10 is a VPN route reflector in SP-1 Site 2.They run MP-BGP VPNv4 session with the PEs. SP-1 provides an MPLS L3 VPN service to CE Site-1 and Site-2. SP-2 provides an MPLS L3 VPN service to SP-1 which is customer carrier. SP-2 treats SP-1 as its regular VPN customer thus on the edges SP-2 puts all SP-1 prefixes into VRF. SP-1 doesn’t use VRF since SP-1 doesn’t send its customer prefixes to SP-2.Instead only loopback addresses of SP-1 is sent to SP-2. Thus Global routing table of SP-1 is advertised to the SP-2. This can be achieved by running either IGP or BGP at the edges. In this topology R3-R5 and R7-R8 runs IGP, specifically OSPF as CSC-CE – CSC-PE protocol. Whole purpose of this architecture is to provide reachability between CE-`1 and CE-2 by using SP-2 as transparent from the SP-1 point of view. Thus SP-2 shouldn’t break end to end LSP. End to End LSP is setup between the SP-1’s edge routers.In our topology R2 and R9 has to reach each other’s loopback likewise Router reflector and the other PEs should reach each other. IBGP VPNv4 session is setup between the RRs in each domain. Since between the sites of SP-1, BGP session is an IBGP session, RR doesn’t change the next hop of the Edge PEs. If it is an EBGP session; on the RR you need to say that BGP next-hop shouldn’t be changed as in the Inter-AS Option C case. If you will remember one thing from this article then read below paragraph. In the Carrier Supporting Carrier Architecture,in order to hide end user prefixes ( SP-1’s customer prefixes), between CSC-CE and CSC-PE (SP-1 and SP-2 ASBRs), mpls is enabled through either LDP or BGP+label (RFC 3107). CSC-PEs ( R5 and R7 in our topology ) assign a VPN label for all of the loopbacks of SP-1. SP-2 advertises those label towards MP-BGP as a VPNv4 label to edge PEs, and also towards CSC-CE (R3 and R8 in our topology) through LDP or BGP. So on the R5 and R7, we have three labels in the label stack in the SP-2 domain. In R6 top most label can be removed with PHP operation. (If Pipe mode QoS enables Explicit Null could be signalled , so PHP is not done) Top most label (Transport label) is for the edge devices reachability for the BGP next hop (Loopback of R5 and r7) of SP-2 , second label is VPN label for the customer/SP-1 for SP-1 edge devices reachability of SP-1 and the third label is the VPN label of SP-1 for the end customer prefixes. If you use in SP-2 MPLS Traffic engineering and FRR for Carrier Supporting Carrier Service, you see up to 5 labels in the label stack thus MTU needs be considered but also hardware should support deep label stack. Although I explained only an MPLS VPN service in the Customer carrier network, Carrier Supporting Carrier architecture is used to provide an additional layer of hierarchy. If customer carrier wants to provide an Internet Service and don’t want to receive CSC service from the backbone carrier for its MPLS VPN customers, same operation (Advertising only loopbacks from the CSC-CE to CSC-PE and assign a label for the loopbacks) is done. Conclusion : Although backbone/core carrier provides an MPLS Layer 3 VPN service with the Carrier Supporting Carrier architecture, better solution for backbone carrier is to provide Layer0 service (DWDM),Layer 1 (Sonet/SDH,POTN and so on) or layer 2 service ( Ethernet, MPLS L2VPN and so on) since providing MPLS Layer 3 VPN in this case operationally more complex than providing basic connectivity. DWDM and SONET/SDH for the long haul links is an expensive solution,especially if the requirement is fast recovery.SONET/SDH can provide this but requires additional hardware such as APS (Automatic Protection Switching). Thus probably for both customer carrier and backbone carrier ideal solution would be Layer 2 MPLS VPN on the backbone carrier. To have a great understanding of SP Networks, you can check my new published “Service Provider Networks Design and Architecture Perspective” Book. It covers the SP network Technologies with also explaining in detail a factious SP network. Click here What about you ? Are you an Enterprise or Service Provider engineer ? Do you provide or receive Carrier Supporting Carrier service ?

Published - Tue, 26 Nov 2019

Created by - Orhan Ergun

MPLS Transport Profile (MPLS-TP) Basic Explanation and Key Points

MPLS-TP - Multi-Protocol Label Switching Transport Profile (MPLS-TP) is a new technology developed jointly by the ITU-T and the IETF. The key motivation is to add OAM functionality to MPLS in order to monitor each packet and thus enable MPLS-TP to operate as a transport network protocol. MPLS Technologies and Design are explained in deep detail in Instructor Led CCDE and Self Paced CCDE course. Motivations for MPLS Transport Profile Evolution of SONET/SDH transport networks to packet switching driven by growth in packet-based services (L2/L3 VPN, IPTV, VoIP, etc) and desire for bandwidth and also QoS flexibility in transport network was one of the main drivers of MPLS Transport Profile. Requirements of an MPLS Transport Profile was defined in RFC 5654. MPLS-TP is a new packet transport mechanism which has the same operational model with TDM based transport networks. It is useful to understand some of the key differences between MPLS-TP and classic MPLS. MPLS-TP vs. IP/MPLS MPLS-TP uses network management instead of signalling protocols for determinism (know where your packets are). GMPLS is defined as MPLS TP signalling protocol and provides control plane functionality to MPLS TP networks. MPLS TP doesn’t force GMPLS to be used though. No routing protocols or IP control plane in MPLS Transport Profile. NMS is used in MPLS TP. Also MPLS Transport Profile doesn’t have :Penultimate Hop Popping (PHP) – last hop IP meaning no MPLS OAM end to end. Thus there is no PHP in MPLS Transport Profile. Equal Cost Multi-Path (ECMP) – makes OAM difficult. That’s why there is no ECMP in MPLS Transport Profile Label Switched Path (LSP) merging – makes OAM difficult. Applications MPLS-TP is a packet transport protocol with capabilities that traditionally belong to transport networks such as SONET/SDH and OTN. The intention with MPLS-TP is to be able to replace such legacy networks as SONET/SDH, though not OTN, while still keeping the advantages of packet transport. Due to its extensive feature catalogue, MPLS-TP is very flexible and can be used for many different applications. One main advantage is that it uses Pseudowire (PW) as a transport entity. PWs are able to encapsulate any type of traffic, such as Asynchronous Transfer Mode (ATM), Frame Relay, Point-to-Point Protocol (PPP), Ethernet, etc. MPLS Transport Profile is applicable to situations where reliability, QoS and OAM are the main requirements. MPLS Transport Profile can be operated/controlled via network management or a control plane, the latter being a main advantage when dynamic provisioning is required. Moreover, MPLS-TP is fully compatible with IP/MPLS networks, which presents many possibilities for network solutions that demand MPLS/MPLS-TP interworking. MPLS Transport Profile can run over Ethernet, SONET/SDH (G.783) and OTN (G.709, G.872) using Generic Framing Procedure (GFP). In these studies OTN was chosen as the underlying layer and thus the delay from the GFP should be included for the MPLS-TP or the OTN layer. When MPLS network is controlled by IP/LDP or RSVP, LSP (Label Switched Path) is always unidirectional. Which mean, return traffic requires an additional LSP. Thus return traffic can pass completely different set of routers/nodes than forward traffic. MPLS TP LSP is a bidirectional LSP which provides deterministic path since the nodes in forward direction is exactly same as reverse direction. Same delay and jitter is provided since the forward and reverse traffic goes through same set of routers/nodes. MPLS Transport Profile doesn’t change the IP/MPLS data plane. Still 32 bits MPLS Header is used. 20 bits of MPLS Label space, 3 bits for EXP, 1 bit for bottom of stack indicator and 8 bits for TTL. I mentioned in the beginning of this post, MPLS TP can use GMPLS for control plane function. Generalized MPLS (GMPLS) provides deterministic and connection oriented behaviour using LSPs (Label Switched Paths). MPLS-Transport Profile also uses Targeted LDP (T-LDP) to set up pseudowires (PWs) over GMPLS LSPs, to provide VPWS (Virtual Private Wire Service) and VPLS (Virtual Private LAN Service). MPLS Transport Profile mandates running protocols such as BFD (Bidirectional Forwarding Detection) over GMPLS LSPs and PWs, to provide OAM functionality.   Last but not least MPLS Transport Profile LSPs are connection oriented. Connection oriented is a communication mode in telecommunications and computer networking, where a communication session or a semi-permanent connection is established before any useful data can be transferred, and where a stream of data is delivered in the same order as it was sent. I recommend below as extra study resources for those who are interested in MPLS Transport Profile Deploying Packet Transport with MPLS Transport Profile Requirements of an MPLS Transport Profile To have a great understanding of SP Networks, you can check my new published “Service Provider Networks Design and Perspective” Book. It covers the SP network Technologies with also explaining in detail a factious SP network. Click here

Published - Tue, 26 Nov 2019

Created by - Orhan Ergun

Inter AS Option C – Design Considerations and Comparison

Inter AS Option C is the most complex, insecure, uncommon, but extremely scalable inter provider MPLS VPN solution. I am explaining this topic in deep detail in my Self Paced CCDE course. In this post, I will explain how service providers can use Inter AS Option C to assist customers to have an end-to-end MPLS VPN service. In the Inter AS Option B post, I explained that ASBR routers between the service providers do not keep a VRF table for the VPN customers. As depicted in the fig.1 (shown below), as for Inter AS Option B, MP-BGP VPNv4 session is set up between service providers’ ASBR PEs. Figure 1: Inter-AS Option B As for Inter AS Option B, ASBR routers – the provider-edge devices between the service providers – maintain only the VPN prefixes of the customers in the BGP table. In fact, I have shown that VPNv4 BGP session has been set up between the ASBRs. The high-level operational differences between Inter AS Option C and Inter AS Option B are in two folds: one is that ASBRs do not have VRF table; the other is that unlike Inter AS Option B, Inter AS Option C do not keep VPN prefixes for the customer on the ASBR. ASBR is the edge routers which are used to connect the Service Provider each other. As depicted in fig. 2 (shown below), as for Inter AS Option C, the VPNv4 BGP session is set up between route reflectors of the service providers. Figure-2 Inter AS Option C In fig.2, there are two service providers: service provider A and service provider B. Service Provider A: Service provider A has two customers: customer A and B For scaling purposes, all the provider-edge routers run VPNv4 BGP session with VPNv4 route reflector. Service provider B: Service provider B has two customers: customer A and B A and B are companies which require Inter AS MPLS VPN service. Service provider B runs IS-IS internally ( It could run other IGP protocols as well for sure) PE-CE routing protocols are enabled on the VRF; thus service provider A has two separate VRF table. For scaling purposes, all the provider-edge routers run VPNv4 BGP session with VPNv4 route reflector. Inter AS Option C runs by enabling VPNv4 BGP session between the Route Reflectors of the Service Providers. Provider-edge routers of service providers do not set up VPNv4 neighborship neither with AS nor between ASes. In order to use the route reflectors to setup VPNv4 neighborship, they should move towards each other. i.e. BGP runs over TCP. For reachability, route reflector loopback interfaces are advertised on the global routing table of service providers. As for AS Option C depicted in fig. 3, ASBR PEs of the service providers run IPv4 BGP session between them. Over IPv4 BGP session, loopback interfaces of route reflectors and internal PEs are advertised. Next, I will explain why you need to advertise internal PE loopback addresses, not just route reflectors. ASBR PEs know the loopback address of Route reflector through OSPF in the Service Provider A network, through IS-IS in the Service Provider B network. Since the VPNv4 EBGP neighborship is set up between VPNv4 RR of the service provider, the next hop for the VPN route is the route reflector. Traffic trombones on the RR unless ” no bgp next-hop unchanged ” configured on the Route Reflectors. This is one of the important caveats in Inter AS MPLS VPN Option C. Once this feature is enabled, route reflector does not change the next hop to itself even though the peering is EBGP VPNv4, between the RRs. ( Whenever there is an EBGP session between two BGP speaker, next hop is changed since they are not client of each other but regular EBGP neighborship, otherwise RR doesn’t change the next hop in IBGP) In this case, because internal PEs continue to be a next hop for the customer prefixes, internal PE loopback is advertised on IPv4 BGP session between ASBRs. As for the VPN next hop, transport label should be assigned to MPLS VPN to work efficiently, and MPLS label is assigned to the IPv4 unicast routes, which are the loopback of RRs and Internal PEs of ASBR. Inter-AS Option C Design Objectives: Inter AS Option C is unique, since it requires internal addressing advertisement between the service providers. Those addresses are leaked into the global routing table of the providers, a process that providers dislike. It can still be used in large scale design if the company has multiple autonomous system. (Not between the service providers). This is just a network design option. Please note that it is not the only option. Inter AS Option C is very difficult to implement because it requires meticulous redistribution at the edge of the networks, VPNv4 neighborship between the route reflectors, IPv4 BGP neighborship between the ASBRs, label advertisement, route target agreement between the service providers, and so on. It is insecure because the internal address leaks between the providers. I know you must have thought whether AS design is common between the service providers in real life. In reality, I helped two times Inter-AS MPLS VPN design of the Service Providers, both of them concluded with Inter AS Option A. To have a great understanding of SP Networks, you can check my new published “Service Provider Networks Design and Architecture Perspective” Book.

Published - Tue, 26 Nov 2019