Is it just me or BGP is something I never learnt about until I heard about it causing issues? It seems it's essential to the internet, just like TCP/IP, but nevertheless I learnt about the latter in the university, during my career, I read many books about TCP/IP... but nothing about BGP (not in the university, not at work, not in books, nothing).
I can "play" with TCP/IP at home in dummy projects and learn more about it... but I have no idea how to "play" with BGP. In that regard, how does one learn about it at home?
Buy some routers that have BGP implementations (there are some cheap ones, Mikrotik for example), or use open source implementations. The article lists bird, another very popular one is FRR (free range routing). You can trivially stand up two docker containers, stand up a BGP session between them, and - for example - propagate static routes you set up within them.
It lets you setup multiple containers with direct connections between them in whatever topology you want. It allows you to run both Linux containers (with FRR for example) and emulated versions of popular router platforms (some of the ones mentioned in the article).
DESCRIPTION
bgpd is a Border Gateway Protocol (BGP) daemon which manages the network
routing tables. Its main purpose is to exchange information concerning
"network reachability" with other BGP systems. bgpd uses the Border
Gateway Protocol, Version 4, as described in RFC 4271.
At least to me one of the challenges is relating to the problems that BGP solves. You can get pretty far in network complexity before BGP (or OSPF etc) really does anything for you. What would be good scenarios one could encounter in "homelab" situation where BGP would be beneficial?
There are no scenarios where BGP contained within your home lab is beneficial for anything other than learning BGP. It's the routing protocol for the Internet. Its whole point is scaling globally, and - crucially - enabling making routing decisions that aren't just based on path weights. OSPF, IS-IS, EIGRP, whatever, they are all just path finding algorithms; OSPF is quite literally Dijkstra. That's great when you want to find the shortest or fastest path to somewhere, but that's not how the Internet works: it's quite reasonable for an operator to want to take the cheapest path (in terms of money), or to take the path that avoids specific foreign countries. BGP is expressive enough to write routing policy like that. You don't need that in your homelab, unless you want to learn BGP, either because you need to for work or to further your career, or because you're curious about it.
Running a few Kubernetes nodes with a network plane like Cilium that supports using BGP to inform your router about which container IP is on which node is a simple-ish one.
Not really, unless you have thousands of routes to manage across large numbers of gateways. Otherwise, running BGP inside your homelab is just a learning tool.
DN42[1] provides a playground for routing technologies. I wouldn't recommend digging in if you don't want to dedicate a lot of time into it. As someone fairly well versed in networking, WAN routing is still confusing to me.
GNS3 is probably the easiest way to get hands on experience with any networking technologies.
The other is to make a couple of virtual machines with a couple of network interfaces, make some sort of network betweeen them and then use some bgp routing deamon, eg:
My undergraduate networking course didn't touch BGP, my graduate networking course did touch BGP. We used a python package that acted as a simulator for different AS but I can't remember which one.
My undergrad networks course discussed a little BGP stuff but only on the blackboard.
To experiment with BGP you could use a network simulator like what the author of this blog did. In my class we used something called gini[1] which I think my profs grad student wrote but the author apparently used gns3 which seems to be a cisco specific ns3 version. I used ns3 once and found it had a steep learning curve. The gini simulator has a more basic user interface but is probably less powerful.
I think you need to manage a real (and large) network that's connected to global internet traffic in order to "play" with BGP. Well, you can tinker with it at home, but only by using a network simulator.
I worked an internship where I spent the summer setting up new equipment for a large corp that was replacing everything that AT&T had installed and managed with their own stuff. Nearly every office had their own ASN, everyone else got regular broadband or just a box of aircards depending on the number of users. I knew nothing about networks other than setting up my own consumer router at home so it was a pretty fun learning experience. I always got a smile on my face when I finally got vRouter to peer with our dummy AS in the office then we'd pack it all up and bring it out for installation over the weekend. I got offered a job to come back after I graduated but turned it down for something that paid better and was a lot more interesting. Honestly, I probably would be making more money as a network engineer now if I stuck with it.
You can set up local BGP routers and peer them and play with it.
Another fun thing is to log into publicly available looking glass servers. Most ISPs (including very, very, very large ones) operate routers that have their full view of the BGP routing tables. They either run web interfaces that let you query those tables (more common) or make public ssh or telnet credentials to log in with roles that have very limited access to the available commands, but have read rights to those tables.
I've used BGP internally at my company for a decade, using AS65xxx range. At home I use BGP between the house, garage and shed, I much prefer it to OSPF.
Same! At previous company I worked at we used BGP for all internal/external routing about 15 years ago despite all the poo-pooing by using BGP as an IGP. It was nice having no route redistribution and one command to monitor sessions.
Open Shortest Path First (OSPF) is an "internal" routing protocol. Basically, it is a protocol for routers to share routes when all routers are managed by the same organization.
Border Gateway Protocol (BGP) has the primary purpose of sharing routes between routers managed by different organizations. It can be used within an organization too. It has a lot more control over how and which routes it sends and receives.
Depends how much you want to know about how networks work. Never ceases to amaze me how ignorant modern software developers are of the underlying technology, I guess that's because I'm from the pre-2010s when "Information Technology" was a general field.
I took some comp-sci and majored in "IT" in the 2000s. Lower level CS did not go over routing protocols, and the IT side never got into compilation, linking, state machines, or pointers.
In the 2000 my team had to deal with everything from compilation problems to hardware answering arp answers with fake mac addresses. The team consisted of a wide range of skills and abilities and information obviously leaks. While the DBA didn't need to know anything about OSPF, just by being in the same team as the network person they pick up how things work.
Now it seems that teams seem to be far more specialised and there's less cross-specialist learning.
I have trained people on network technologies, including the younger generation. It never ceases to amaze me how much they can get done without a clue about the underlying technologies. Sometimes it feels like they have some super power, because I can't operate without that knowledge.
You don’t need a large network to participate to BGP. You just need a /24 (IPv4) or /48 (IPv6) allocation, AS number, and a business class Internet connection that can do BGP. Might be out of reach for most hobbyists but not impossible.
When has BGP not been implicated in causing issues though?
The first widespread incident I found was from 1997 [1], but I didn't look too hard.
I don't think there's really a satisfying way to play with BGP as a small network. Traffic engineering is where I think the fun would be, but you've got to have lots of traffic and many connections for it to be worthwhile. Then you'd be trying to use your announcements to coax the rest of the internet to push packets through the connections you want. As well as perhaps adjusting your outgoing packets to travel the connections you prefer when possible. Sadly, nobody lets me play with their setup.
One of the ways to get a sense of emergent routing behavior is if you have hosting in many places, you'll likely see a lot of differences in routes when you start going to far off countries. If you run traceroutes (or mtr) from your home internet and your cell phone and various hosting, and if you can trace back... you'll likely see a diversity of routes. Sometimes you'll see things like going from west coast US to Brazil, where one ISP will send your packets to florida, then Brazil, and one ISP will send your packets to Spain, then Brazil, with a lot more latency.
You can play with BGP by joining https://dn42.eu/ - a fake internet with a few thousand participants who are mostly as clueless as you, and none of whom will lose millions of dollars per hour if it breaks (which is not infrequently).
TCP/IP affects every networked application and endpoint on the internet.
BGP runs the internet routing "in the background" and you only need to know it if you're an internet service provider or work in a large org managing the network. If you didn't learn network routing, you aren't going to learn BGP.
Put two or three VMs (OpenBSD has OpenBGPD daemon) onto a shared virtual switch and addresses in 172.31.255.0/24, connect the VMs. Also each of the VMs should have at least one other interface onto unique virtual switches with their own network (172.31.1.0/24, 172.31.2.0/24, etc).
So back when I did Wisp stuff I'd set up simulates networks between multiple machines with real and virtual networks. VyOS which was similar to the UBNT equipment we were using is light weight and supports multiple protocols.
In my opinion, containerlab is one of the easier tools to setup a lab environment for networking. You define a network with yaml which consists of nodes and links between them and it creates these using docker. They also have a BGP peering example lab: https://containerlab.dev/lab-examples/peering-lab/
Well, what did you study in the university? I did learn about BGP and routing in university since one of my subjects was information networks and protocols. But haven't really used it outside of some lab exercises since there's been no need at work nor at home.
I learned (and later taught) BGP (and routing in general), albeit at a superficial level, in high school already. Then I actually got to work with it during labs in university.
I remember Helsinki CS having quite a bit of BGP, TCP and both ipv4 and ipv6. No guarantees that every student aced those classes, but the teaching definitely was there
Cisco offers some simulator tooling. It basically virtualizes a lot of networking devices and allows you to play LEGO/SimCity with them: Cisco Packet Tracer
Now, we built toy networks from scratch while I was working toward my certification. Surely larger-scale simulation files could be loaded into Packet Tracer. And perhaps, vendors have simulators on a larger scale than the free downloads?
When I worked at a regional ISP, my supervisor was the BGP wizard. He referred to exterior routing as "a black art". Even more, the telcos were deploying their own technologies like Frame Relay and SMDS, which are Layer 1/Layer 2 protocols beyond the standard "point-to-point" leased lines.
We once experienced a fiber cut on our T-3 backbone (construction workers didn't dial 811). So my supervisor arranged the BGP routes to send everything over a 56k line, IIRC. He gloated about it. The packet loss rate was absurd, but our customers had connectivity!
>When I worked at a regional ISP, my supervisor was the BGP wizard. He referred to exterior routing as "a black art".
Yep this seems like a very common experience. I tend to find most environments have one guy making BGP changes outside of project work.
>We once experienced a fiber cut on our T-3 backbone (construction workers didn't dial 811). So my supervisor arranged the BGP routes to send everything over a 56k line, IIRC. He gloated about it. The packet loss rate was absurd, but our customers had connectivity!
The modern version of this: At a small national ISP, we had our intercarrier lines cut. Megaport has this billing model where you only pay for the capacity you use, so our backup intercapital was a 1MB megaport service. Intercapital goes down, everyone kicks over to the megaport and we just log on to the megaport portal and raise the bandwidth to a few gig temporarily. Cost almost nothing to keep it sitting there ready for use. And yeah the engineer responsible was extremely and deservedly smug.
>And perhaps, vendors have simulators on a larger scale than the free downloads?
My experience is that you need both the exact hardware/firmware AND the exact config to perfectly simulate some of the weird and wonderful stuff. Largely because so much of the protocols issues, like the OP suggests, is down to individual vendor implementations of the protocol.
For instance, I used to consult for a small ISP that had a very unreliable peer. That peer would send them routes for everything, but occasionally their PE's routing plane would collapse and stop forwarding traffic to/from their other peers.
We still received enough packets to not trip any failover, and routes were still being advertised. So until they realised and rebooted their hardware, we had to withdraw our routes.
This is the specific behaviour between (IIRC) Cisco IOS-XR on our end, their predominantly mikrotik environment, and their other peers who I believe were mostly juniper.
I cant imagine simulating that without the relevant hardware and configs.
if you're a linux person consider a routing on host setup with FRR with /32s. As every host is a /32 network you can focus more on the aspects of BGP rather than TCP/IP.
I can "play" with TCP/IP at home in dummy projects and learn more about it... but I have no idea how to "play" with BGP. In that regard, how does one learn about it at home?