High Availability inside the Datacenter

High Availability inside the Datacenter: In Leaf/Spine VXLAN based data centers, everyone likes to provide HA with Active/Active in it, so choices are different. There are two types of HA in data centers, Layer 3 and Layer 2. For layer 3 HA, always there is more than one spine that can provide ECMP and HA at same time. However, Layer 2 redundancy for hosts and l4-l7 services that connected to leafs are more than an easy choice.

As Cisco provided vPC for nearly 10 years ago, almost this was the first (and only) choice of network engineers. Also, other vendors have their own technologies. For example, Arista provided Multi-chassis Link Aggregation (MLAG) for L2 HA in leafs. But, there is always a problem in implementation of them. One example in vPC is “peer-link” that is an important component in the vPC feature.

However, it can be a tough one in most cases like Dynamic Layer-3 routing over vPC or Orphan members that may cause local traffic switching between vPC peers without using Fabric links.   To address the “peer-link” issue, there is a “fabric-peering” solution that uses Fabric links instead of “peer-link” and convert it to “virtual peer-link”. With this solution there is no concern about local switching in specific cases.


This solution works better, but it cannot solve Dynamic Layer-3 routing over vPC or other issues (PIP, VIP, virtual-rmac) without enhancements. There is another HA solution that you can find it below.  


EVPN Multihoming

With introducing EVPN, EVPN Multihoming is a solution to bring HA in layer 2 links without any “peer-link” like dependency. EVPN uses Ethernet Segment Identifier (ESI) in Ethernet Auto-discovery (EAD) or route type 1 with 10 bytes value. ESI is configured under bundled link that sharing with multihomed neighbor. ESI can be manually configured or auto-derived.

Also, to prevent loop scenarios because of packet duplication in multihoming scenarios, Ethernet Segment Route (ESR) is the other route type (4) that mainly uses for designated forwarder (DF) election. DF is election for a forwarder that handles BUM traffic. Also, Split-horizon feature enables only remote BUM traffic allowed to be forwarded to a local site and BUM traffic from same ESI group will be drop. With using EVPN multihoming, traffic will be balance between both leafs because they are advertising a shared system MAC with same ESI.

This feature is called “Aliasing”. In failure scenarios, with “Mass Withdrawal” fast convergence will remove failed leaf from ECMP path list. Also, LACP can be turned on to prevent ESI misconfigurations.


Designated Forwarder



MAC Aliasing



MAC Mass Withdrawal


Regardless of which HA solution is better, there are some differences between HA solutions that some of them are listed below on Cisco technologies. You must keep in mind that this is Data Center comparison. Because vPC does not supported on Cisco routers but EVPN does.  
  vPC/vPC2 EVPN Multihoming
Hardware All Nexus platforms Nexus 9300 Only (until now)
FEX supported Yes No
Same OS version Yes Not mentioned
Multiple components Yes No
QoS needed For vPC2(fabric peering) No
ISSU Yes No
Maximum peers 2 2+
 

Load Balancing

vPC is using regular port-channel load balancing methods. On the other hand, EVPN provides modern load balancing. There is three EVPN load balancing method available: 1-Single Active 2- All active 3- Port Active. Assume two leafs are connected to one host, but only one of them we are considering to be active for a service. So, related MAC address is reachable via only one of leafs and it refers to per-VLAN (service) load balancing too. This method is suitable for billing some services or policing on specific traffic.


In All active mode, each leaf can forward traffic. In this mode, load balancing method is per-flow. This is regular load balancing mechanism that share traffic between leafs as below figure. This method is better for providing more bandwidth based on end hosts.


Port Active is a mechanism that brings Active-Standby scenario to EVPN multi-homing and only one leaf forwards traffic. However, in failure scenarios with fast convergence, traffic will switch to standby leaf. This method is your choice when you want to force traffic on a specific link that is cheaper or you want to use only one link.


This is important that every EVPN feature in this post is not implemented on all platforms and vendors. To recapitulate, both solutions have cons and pros. Depend on Data Center design and requirement, you can choose one of solutions. Keep in mind that you can not enable both feature on a switch at a same time. Also, LACP is an additional tool to improve these features functionality and it helps avoiding misconfigurations.   note: all figures are taken from ciscolive presentations

Created by
Orhan Ergun

Orhan Ergun, CCIE/CCDE Trainer, Author of Many Networking Books, Network Design Advisor, and Cisco Champion 2019/2020/2021

He created OrhanErgun.Net 10 years ago and has been serving the IT industry with his renowned and awarded training.

Wrote many books, mostly on Network Design, joined many IETF RFCs, gave Public talks at many Forums, and mentored thousands of his students.  

Today, with his carefully selected instructors, OrhanErgun.Net is providing IT courses to tens of thousands of IT engineers. 

View profile

Daniel Lardeux
Daniel Lardeux Senior Network Consultant at Post Telecom

I passed the CCDE Practical exam and Orhan’s CCDE course was very important contributor to my success. I attended the CCDE course of Orhan Ergun in July and it was exactly what I needed, Orhan is taking the pain to break down the different technologies.

Roy Lexmond
Roy Lexmond Senior Network Designer at Routz CCDE #20150017 & CCIE R&S; #26557

After I attended Orhan Ergun’s CCDE course I passed the CCDE practical exam.I really enjoyed the course a lot ...

Nicholas Russo
Nicholas Russo Network Consulting Engineer (CCDE/CCIEx2), Cisc

I signed up for Orhan’s CCDE training. This training is very technically detailed and the use-cases, quizzes, scenarios, and mind maps are all great resources in the overall training program. Orhan teaches his students to think like a network designer ...

Slide Heading
Slide Heading Network Systems Engineer at Conscia A/S CCIE #42544 (SP) & CCDE #20160015

Orhan is forcing you to take off the implementation hat that most of us have been wearing for many years, instead he is providing a new fancy design hat, which makes you see and deal with the issues presented ...

Kim Pedersen
Kim Pedersen CCIE in RS and SP (#29189) CCDE#20170021

I’ve used Orhan’s self-paced CCDE training material. If you are interested in knowing how all the technologies go together in a coherent design i can highly recommend it.I also enjoyed the Quizzes which helped pick out my weak spots in selecting ...

Laurent Metzger
Laurent Metzger 3xCCIE/CCDE Senior Network Architect

Hi Orhan. I passed the CCDE exam on February 22. I read everything that you put on your Self Paced CCDE Training course and it was very helpful in my success. Thank you very much.

Martin J. Duggan
Martin J. Duggan Network Architect at AT&T;, Ciscopress Author CCDE #20160006 & CCIE#7942

I attended Orhan’s April 201610 days CCDE Bootcamp. I am CCDE now !

You can tell Orhan has a great deal of experience, it really comes through when he presents his design case studies and the CCDE Practical scenarios.

Muhammad Abubakar
Muhammad Abubakar Lead Network Architect – CCDE #20160016 2xCCIE #26693 2xJNCIE VCIX

Your excellent CCDE materials and amazing Bootcamp helped me tremendously through my learning journey.Also thank you very much for being available whenever I have a design question or a complex design topics. I can’t compare your design skills ...

Jennifer Pai
Jennifer Pai Network/Security Engineer at KNET Technology

Thanks Orhan very much for this course. It helped strengthen my “Network design mind”.

Ruslan Silyayev
Ruslan Silyayev Solution Architect at R.I.S.K Company

Training by Orhan is not a CCDE preperation training only. It will be useful for engineers which are dealing with design. You want to pass CCDE exam or learn network design, then don’t look at anywhere else!

Sameer Meher
Sameer Meher Solutions Architect at 23 Wards/Japan

Orhan Ergun’s CCDE course was really very good. CCDE Level Intelligence was delivered very well and with very useful case studies and the scenarios, I am thankful to Orhan for all his help!

Ken Young
Ken Young Senior Technical Architect Province of Nova Scotia, 2xCCIE #41597 | CCDE #20170047

If anyone wants to understand network design and architecture, also pass CCDE exam , I recommend you to attend Orhan’s online courses! I am a CCDE now but learning is a journey, we will be together in your other courses too Orhan!

Matt Cross
Matt Cross Technical Architect at Heartland – CCDE #2019::7

Orhan did an excellent job of filling in the gaps of knowledge that I had that took me to the finish line of the practical exam CCDE. The community of people that Orhan facilitates are both engaging and supportive of the journey to CCDE. Orhan ...

Shiling Ding
Shiling Ding Sentinel Technologies – CCDE #2019::12

Just passed the CCDE Practical exam! I attended Orhan Ergun’s CCDE training program , used Orhan’s Instructor Led and Self-Paced CCDE training and Online CCDE Practical Scenarios during my CCDE journey. Orhan’s CCDE In Depth book is an excellent summary ...

Abelardo Basurto
Abelardo Basurto Solutions Architect at Cisco Systems – CCDE 2018::6

Hi everyone, I’ve just passed the CCDE Exam. My Number is CCDE 2018::6 I attended to Online CCDE Bootcamp of Orhan. I want to thank Orhan not only for the great book and bootcamp, but also for his commitment, availability and willingness to assist the ...

Hady Mohamed Abdellah
Hady Mohamed Abdellah Network Architect Hamad International Airport Qatar – CCDE 2018::1

Hi guys, I’m so happy that I passed the exam. I’ve already got my number CCDE 2018::1. Thanks to Orhan for being the best CCDE instructor in the world. I highly recomend Orhan’s CCDE Training and In-Depth-CCDE ...

Bryan Bartik
Bryan Bartik Sr. Systems Engineer at CompuNet – CCDE 20170059

Hi Orhan I passed CCDE Practical exam on November 2017 ! I really enjoyed your materials and quizzes and use cases. They were definitely helpful in my preparation. Thanks a lot !

Giedrius Trapkauskas
Giedrius Trapkauskas Network Solutions Architect at Liberty Global – CCDE 20180004

I attended Orhan’s CCDE Training in Istanbul and it was very helpful in my preparation. I passed the exam recently and I want to say Thank you Orhan! For those who want to pass the CCDE exam, definitely start with ...

Alaa Issa
Alaa Issa Sr.Solutions Architect – CCDE#20180033 3xCCIE ( Collab|DC|Security )#27146

I registered to Orhan’s training in Feb 2017. From that time, I attended Orhan’s training several times. The depth of knowledge which Orhan has is amazing, and how to present such consistent knowledge to the ...

Mazin Ahsan Design Lead Engineer | Solutions Engineer | CCDE License # 20160030 | CCIE Licence # 23892

I passed the CCDE Practical Lab exam on November 17,2016 from supplications of elders and dedication from my Sensei Mr. Orhan Ergun I took different CCDE bootcamps in the past. Orhan has the most depth and expertise ...

Jeff Patterson CCDE# 2018::11

Hi Orhan I wanted to pass along my appreciation for the outstanding training material. I used the online CCDE training provided by Orhan as well as the In-Depth-CCDE book and passed the exam in February 2018. Thank you Orhan!

Mehdi Sfar
Mehdi Sfar Network and Security Architect / CCDE #20210003 | CCIE R&S; #51583

I signed up for Orhan’s CCDE Self paced Course. This course, along with the CCDE In Depth book, helped me for my CCDE Practical as well as Written exams. It pushed me to ask the "WHY" questions and allowed ...

Related courses

Network Design Fundamentals Course

02:28:37 Hours
17 Lectures
Beginner

$50

CCIE Enterprise Infrastructure Training

117:12:56 Hours
287 Lectures
Expert

$995

Cisco Datacenter Networking Fundamentals Training

15:22:21 Hours
57 Lectures
Beginner

$50