Just got back online a day later. Progress on side projects aside, the day was a write off.
In terms of causes, I think it was one of a few possibilities in order of percieved plausibility and likelihood:
1. newb engineer updated router configs using a CI pipeline that caused them to stop talking to each other.
2. standard mistake of redistributing iBGP with a bunch of static routes for load balancing into eBGP causing their internet facing ASN to announce the internet, and it took down their internal MPLS routes for backhauling their cellular traffic over IP.
3. geopolitical situation where someone was making a deniable example of their capability against a target who can't fight back. That was just a friday in Canada, it's just as easy to do D.C., Chicago, or New York.
4. a political stunt and pretext for "a safer internet" with "more oversight," orchestrated by party hacks, or an exercise to see how effectively they could do this. Least likely, but that's the level of trust there is in this govt right now. If you don't want conspiracy theories, try being credible enough people don't have to invent explanations because nobody believes you are in control.
I'd bet on 2 or 3. Items 1 and 4 are weak ideas, but 4 will likely exacerbate confirmation bias. What will likely not come out of this is liberalization of the telecoms market to add competition. However, regardless of teleological fallacies, I don't see this as a crisis that will be left to go to waste.
It has been a good fire drill for business continuity and preparedness anyway.
In terms of causes, I think it was one of a few possibilities in order of percieved plausibility and likelihood:
1. newb engineer updated router configs using a CI pipeline that caused them to stop talking to each other.
2. standard mistake of redistributing iBGP with a bunch of static routes for load balancing into eBGP causing their internet facing ASN to announce the internet, and it took down their internal MPLS routes for backhauling their cellular traffic over IP.
3. geopolitical situation where someone was making a deniable example of their capability against a target who can't fight back. That was just a friday in Canada, it's just as easy to do D.C., Chicago, or New York.
4. a political stunt and pretext for "a safer internet" with "more oversight," orchestrated by party hacks, or an exercise to see how effectively they could do this. Least likely, but that's the level of trust there is in this govt right now. If you don't want conspiracy theories, try being credible enough people don't have to invent explanations because nobody believes you are in control.
I'd bet on 2 or 3. Items 1 and 4 are weak ideas, but 4 will likely exacerbate confirmation bias. What will likely not come out of this is liberalization of the telecoms market to add competition. However, regardless of teleological fallacies, I don't see this as a crisis that will be left to go to waste.
It has been a good fire drill for business continuity and preparedness anyway.