21–30 of 87 posts
This is a great write up. It's also why the DNS root servers have a policy of surviving DDoS through massively over-provisioned, multi-org, anycasted redundancy rather this sort of smart DDoS mitigation that drops traffic: DNS is so critical that any risk of dropping real traffic is unacceptable. (obviously, such a scale is impractical for 99% of services) A good takeaway from this outage for the average user would b…
TL;DR: we should have used an IP that is not traditionally used for testing and internal stuff by everybody including Cisco.
Major credit to Cloudflare for publishing a clear, honest, and detailed description of what happened. I wish more companies would do this. One thing I’d be interested to know more about is why it took 17 minutes to fix. While you can and always should strive to make them less likely, outages are inevitable, so how you respond is crucial. Here the outage was very obviously caused by a deployment that I’d assume was su…
- shit is not working
- is this an attack?
- no it's us
- how?
- that's how
- let's go back
- have to get supervisor
- roll back huge thing
really that long?
Do they just use python as pseudo code or do they actually run their attack detection in python?
[1] https://speakerdeck.com/majek04/gatelogic-somewhat-functiona...
Major credit to Cloudflare for publishing a clear, honest, and detailed description of what happened. I wish more companies would do this. One thing I’d be interested to know more about is why it took 17 minutes to fix. While you can and always should strive to make them less likely, outages are inevitable, so how you respond is crucial. Here the outage was very obviously caused by a deployment that I’d assume was su…
Especially when you consider that they are getting DoS attacks every 2-3 minutes - so all deploys are going out into a hectic world and the dots maybe aren't that easy to connect under those circumstances.
This is a great write up. It's also why the DNS root servers have a policy of surviving DDoS through massively over-provisioned, multi-org, anycasted redundancy rather this sort of smart DDoS mitigation that drops traffic: DNS is so critical that any risk of dropping real traffic is unacceptable. (obviously, such a scale is impractical for 99% of services) A good takeaway from this outage for the average user would b…
Or 9.9.9.9 if you're not as comfortable with Google services.
Major credit to Cloudflare for publishing a clear, honest, and detailed description of what happened. I wish more companies would do this. One thing I’d be interested to know more about is why it took 17 minutes to fix. While you can and always should strive to make them less likely, outages are inevitable, so how you respond is crucial. Here the outage was very obviously caused by a deployment that I’d assume was su…
I'm not an expert, but is 17 minutes for: - shit is not working - is this an attack? - no it's us - how? - that's how - let's go back - have to get supervisor - roll back huge thing really that long?
Do they just use python as pseudo code or do they actually run their attack detection in python?
Earlier quoted context omitted.
9.9.9.10 is "quad nine without the threat detection" from memory.
Yeah, but the reason why 1.1.1.1 is so fast for sites that use Cloudflare as DNS is because Cloudflare is the authoritative DNS for them. The only way you get that in a more generic sense is if a specialist DNS CDN provider started up that provided DNS services for all the existing CDNs (or they all agreed to some type of federated standard that let them share the same recursive multicast IP addresses for DNS resolut…
Earlier quoted context omitted.
Cloudfare makes it weirdly difficult to find this. 1.1.1.1 is plastered over many pages but not concomitant with the secondary.
That is not my experience at all. By following the `install`-instructions on http://1.1.1.1 , all available DNS addresses are listed for both ipv4 and ipv6.