Another write up of the event: https://blog.cloudflare.com/bgp-leaks-and-crypto-currencies/
Hijack of Amazon’s domain service used to reroute web traffic for two hours
241–250 of 291 posts
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#242Earlier quoted context omitted.
I would assume they go far out of their way to resolve the address from a large number of distributed DNS resolvers - the question is if they're just round robin instead of multiple simultaneous queries, but I would hope that's part of why the DNS test usualy takes >60 seconds. If they're not treating DNS as a consensus protocol instead of as an immutable database, they'll always be open to poisoning attacks. If any…
Let's Encrypt does not currently validate from multiple perspectives but we're working on it. We're cooperating with a research team at Princeton to make sure our strategy is an effective mitigation against BGP attacks. Any given production validation right now may come from one of two of our datacenter locations, but not both. You may see multiple validation requests if you use our staging servers, but that's just a…
I'm trying really hard here not to be snide, but it is amazing to me that an organization that is responsible for securing the world wide web is basing that security on the hope that nobody can spoof 3 AS's at once.
Just give up on BGP. Strongly suggest people use DNSSEC. For TLDs that don't support DNSSEC, require a public key issued to the registrar. You (the Certificate Authority) can get the public key from [R]WHOIS, IRIS, RDAP, whatever method you like, directly from the registrar. This is standard practice using the thin WHOIS data model, where .com and .net require domain registrars to maintain their own customers' data. Other TLDs simply run thick WHOIS servers that store all the data for their domains.
Registrars are already required to pass up Delegation Signer records to a TLD to support DNSSEC. Passing on a similar record to a Certificate Authority would be practically the exact same thing. So we know the registrars can support it, and we know it would ensure that people would only be able to generate certificates for those domains that they actually own (and not "whomever can control the IP space currently").
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#243Earlier quoted context omitted.
Traffic to Route 53 was rerouted to an alternative DNS server. But that has nothing to do with my question re. AWS calling out Google in particular in their status update.
Some Route53 traffic was redirected somewhere else. Traffic can only be hijacked from neighbors that actually accept the routes. You would think Google of all networks would have effective ingress filtering to prevent this, but it seems like they did accept it in this case.
I don't think ingress filtering on Google's edge would have helped if the rerouting happened in any of the transit ASs between AWS and Google.
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#244Wonder if services like Let’s Encrypt were affected. I imagine a scenario where a small hijack of DNS could allow for properly signed certificates for domains that are not owned. If I operated a CA service, I would carefully examine the requests received during this time frame. Maybe someone can audit the Transparency Logs during this period for anomalous activity.
Isn't this exactly the problem that EV certificates solve? By going the extra mile to validate identity instead of just trusting DNS ownership?
Not that there's an easy way for LE to validate identity automatically - governments need to do a better job of providing electronic/smart identification, in which case a physical chip ID serves as a second factor (alongside DNS ownership) for automatic EV certs - but the weaknesses of only having DNS ownership are pretty much why EV certs exist.
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#245Earlier quoted context omitted.
I'm seeing HSTS on https://www.myetherwallet.com/ curl -I https://www.myetherwallet.com/ strict-transport-security: max-age=63072000; includeSubdomains; preload
Presumably that's fairly new. The domain hasn't made it on the HSTS preload list shipped by browsers yet, so my guess would be they started using HSTS in response to this attack.
It was addad at most a few hours earlier and as a response to the attack. That's when I checked and it did not have HSTS at that point.
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#246Wonder if services like Let’s Encrypt were affected. I imagine a scenario where a small hijack of DNS could allow for properly signed certificates for domains that are not owned. If I operated a CA service, I would carefully examine the requests received during this time frame. Maybe someone can audit the Transparency Logs during this period for anomalous activity.
I'm going to assume that Lets Encrypt wasn't impacted for the reason that there are several discussions that the hijacked myetherwallet website apparently had a self signed SSL cert running - and people were actually clicking through the warnings before handing over passwords.
Surely they would have realised they could make the whole thing a lot more profitable if any SSL provider was impacted.
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#247Time to use the Ethereum Name Service (ENS). As long as you keep your private key safe, your domain name is safe.
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#248Earlier quoted context omitted.
Let's Encrypt does not currently validate from multiple perspectives but we're working on it. We're cooperating with a research team at Princeton to make sure our strategy is an effective mitigation against BGP attacks. Any given production validation right now may come from one of two of our datacenter locations, but not both. You may see multiple validation requests if you use our staging servers, but that's just a…
I'm trying really hard here not to be snide, but it is amazing to me that an organization that is responsible for securing the world wide web is basing that security on the hope that nobody can spoof 3 AS's at once. Just give up on BGP. Strongly suggest people use DNSSEC. For TLDs that don't support DNSSEC, require a public key issued to the registrar. You (the Certificate Authority) can get the public key from [R]WH…
We should not "give up on BGP". What we should do is to improve security in all layers. This includes BGP, and as you mention DNS. DNSSEC should be mandatory, just as TLS, for any business that take themselves seriously.
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#249Earlier quoted context omitted.
So if BGP is trust based and hard to compromise then how were these Russians able to? > They re-routed DNS traffic using a man in the middle attack using a server at Equinix in Chicago.
Because large networks do not register all their prefixes or enforce policies on what they accept from other large networks. So a large network can do this. Allowing a small network to do this demonstrates incompetence. Frankly, I am having a hard time believing that HE.NET failed that badly. AS7007 incident demonstrated this being a bad idea in the nineties.
> Frankly, I am having a hard time believing that HE.NET failed that badly.
Believe it. HE has many, many netiron-based routers in their network and many, many ASNs as transit customers. the hardware doesn't have the capacity to implement complete filtering for everyone.
between hardware-constrained equipment being replaced and the tireless work Job Snijders (& others) are doing with route security, these kinds of episodes will become far more difficult and occur much, much less.
Re: Hijack of Amazon’s domain service used to reroute web traffic for two hours
#250How would you see this in ThousandEyes? I see he has a screenshot of it in the post, and doesn't appear to be logged in. Was troubleshooting this for the two hours, until I started reading reports of the BGP leak. Not in networking, so don't fully understand how it works. I'd just like to know how to troubleshoot this in the future, and check for leaks, if any. Are there any other online tools I could use? Obviously…
> How would you see this in ThousandEyes? Checkout this video analysis from the team on the hijack: https://youtu.be/YXm4GJMUlP0
Thanks! Do you know how I can access that page he is on? I'd like to have a click around myself.