11–20 of 64 posts
I'll bet JGC can write his own ticket by now, but unretiring would be really bad optics. He's on the board though and still keeping a watchful eye. But a couple more of these and CFs reputation will be in the gutter.
I’m a huge fan of flapping when it’s really hard to do progressive rollouts. What this would mean here is you switch advertising the old and new routes back and forth automatically and this happens let’s say for 1 minute max before the old config is restored. Then a human looks at various metrics before they push a button to really make the new config permanent. It gives you a cheap way to preflight what will happen…
Weak engineering. Both from the CloudFlare side and their peers.
> we pushed a change via our policy automation platform to remove the BGP announcements from Miami Is there any way to test these changes against a simulation of real world routes? Including to ensure that traffic that shouldn’t hit Cloudflare servers, continues to resolve routes that don’t hit Cloudflare? I have to imagine there’s academic research on how to simulate a fork of global BGP state, no? Surely there’s a…
You can get access to view of routes from different parts of networks but you do not have access to those routers policies, so no
> I have to imagine there’s academic research on how to simulate a fork of global BGP state, no? Surely there’s a tensor representation of the BGP graph that can be simulated on GPU clusters?
Just simulating your peers and maybe layer after is most likely good enough. And you can probably do it with a bunch of cgroups and some actual routing software. There are also network sims like GNS3 that can even just run router images
> we pushed a change via our policy automation platform to remove the BGP announcements from Miami Is there any way to test these changes against a simulation of real world routes? Including to ensure that traffic that shouldn’t hit Cloudflare servers, continues to resolve routes that don’t hit Cloudflare? I have to imagine there’s academic research on how to simulate a fork of global BGP state, no? Surely there’s a…
I assume it's not possible unless you know the in-memory state of all the other gateway routers on the internet, no? You can know what they advertise, but that's not the same thing as a full description of their internal state and how they will choose to update if a route gets withdrawn.
I've had to read the RCA a couple of times to (probably) get what happened, even if I'm reasonably familiar with BGP. Basically, my understanding (simplified) is: - they originally had a Miami router advertise Bogota prefixes (=subnets) to Cloudflare's peers. Essentially, Miami was handling Bogota's subnets. This is not an issue. - because you don't normally advertise arbitrary prefixes via BGP, policies were used. T…
Surely the notion of who owns an AS should be cryptographically held so that an update has to be signed. Updates should be infrequent so the cost is felt on the control plane, not on the data plane.
I'm sure there's a BGPSec or whatever like all the other ${oldTech}Sec but I don't know if there is a realistic solution here or if it's IPv6 style tech.
0: I looked it up before posting and it's 3000 leakers with 12 million leaks per quarter https://blog.qrator.net/en/q3-2022-ddos-attacks-and-bgp-inci...