BGP Peering is an agreement between different Service Providers. It is an EBGP neighborship between different Service Providers to send BGP traffic between them without paying upstream Service Provider.
To understand BGP peering, first we must understand how networks are connected to each other on the Internet. The Internet is a collection of many individual networks, which interconnect with each other under the common goal of ensuring global reachability between any two points.
BGP Peering and Transit Links
As in the above picture, there are three primary relationships in this interconnection:
- Provider: Typically someone who is paid and has the responsibility of routing packets to/from the entire Internet.
- Customer: Typically someone who pays a provider with the expectation that their packets will be routed to/from the entire Internet.
- Peers: Two networks that get together and agree to exchange traffic between each others’ networks, typically for free. There are generally two types of peering: public and private. Both will be explained in this session.
Reduced operating cost: A transit provider is not being paid to deliver some portion of your traffic. Peering traffic is free!
Improved routing: By directly connecting with another network with whom you exchange traffic, you are eliminating a middle-man and potential failure point.
Distribution of traffic: By distributing traffic over interconnections with many different networks, the ability to scale is potentially improved.
In the above diagram, Two Service Providers create a BGP Peering with each other and send only their customer and aggregate subnets. They don’t need to use upstream transit Service Provider to reach their customers.
Private BGP Peering
Private Peering is a direct interconnection between two networks, using a dedicated transport service or fiber.It is also known as bilateral peering in the industry.
- May also be called a Private Network Interconnect, or PNI.
- Inside a datacenter this is usually a dark-fiber cross-connect. May also be a Telco-delivered circuit as well.
- If there is big amount of traffic between two networks, private peering makes more sense than public peering
- Private peering can be setup inside Internet Exchange Point as well
- Larger companies generally uses Private peering rather than Public peering since they want to select who they are going to be peer with.
Typically public peering is done at the exchange point. BGP Route servers are used in public peering to improve scalability. It is very common BGP Router reflector which is used in IBGP topologies. The difference is BGP Route Server is used in EBGP.
Exchange point is commonly referred as IX. It is BGP prefix exchange between the Service Providers.
Internet Exchange Point network is basically a Layer 2 LAN.
Each Service Providers is assigned an IP address from the common LAN and BGP neighborship is setup between the SP and the Route Servers of IX.
Below is UAE-IX physical topology which consist of two Ethernet switches and trunk connection between them.
Larger IX might use more advanced topologies but the concept is the same. Layer 2 LAN and providing an IP address from the same Ethernet segment to each BGP speaker.
UAE-IX (Internet Exchange) Physical Topology
Some considerations in public vs. private BGP peering:
- Route servers are used in the Internet Exchange Points in the same way as route reflectors in IBGP design.
- Route servers provide scalability in the IX and don’t change the BGP next-hop of the prefixes.
- An exchange point is typically the optimal choice for a network maintaining a large number of small interconnections and little traffic.
- Not every company peers publicly with all others; some have closed peering agreements or require a number of POP locations or some amount of bandwidth or other criteria to be a peer with others.
Trying to maintain private interconnections with dedicated physical links for every peer is often financially or logistically prohibitive.
For example, maintaining one hundred GigE cross-connects to a peer with one hundred small peers would probably exceed the cost of an exchange point port, not to mention the overhead of provisioning and maintaining the ports.
A private peer is typically the optimal choice for two networks exchanging a large volume of traffic.
For example, if two networks exchange 10 Gbps of traffic with each other, it is probably cheaper and easier to provide a dedicated 10 GE between them, rather than have them each pay for another 10 GE exchange port.
Many networks maintain a mix of public and private peers.
There are other concepts such as Settlement Free Peering ,Paid Peering and Mandatory Multilateral peering as well but I will cover them in separate post.
BGP Peering Case Study
Network A is a customer of Network Z and Network B is a peer of Network Z.
Network A becomes a transit customer of Network B.
Network A announces 18.104.22.168/16 aggregate to Network Z and more specific prefixes, 22.214.171.124/24 and 126.96.36.199/24 to Network B. Network B sends more specifics to its peer Z.
Network Z only announces the aggregate to the world. What is the impact of this design?
How can it be fixed?
BGP Peering Case Study
As depicted in the above diagram, Network B doesn’t announce the specific to the world. As a result, traffic from Internet to Network A goes through Network Z and then through Network B over peer link.
Network A doesn’t have to pay its provider Network Z. This is known as the Jack Move. Here Network A and Network pull the Jack Move on Network Z. As previously seen in the peering section, most, if not all networks prefer customers over peers and this is implemented with local preference.
However, here the customer (Network A) is sending aggregates only to Network Z, but more specific routes are coming from Network B, which is a peer network.
Prefix length overrides the local preference during forwarding.
The only way to prevent this situation is Network Z should watch whether their peers advertise more specific announcements for the routes learned from its customers.