Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heartbleed Update v3

blogs.akamai.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Heartbleed Update v3

#31

Post by daeken »

blahedo wrote: post #27
daeken wrote: post #18

Earlier quoted context omitted.

No, this was not a good response. It was well written and took responsibility for bad code, but it failed to address the elephant in the room: why were keys not immediately rotated? I've said this a number of times lately, but it bears repeating that the golden rule of security is: If a compromise occurs, assume that everything is compromised. Even if their approach was tested to be perfect, they shouldn't have made…

No, that goes too far the other way. If I have legitimate reason to believe that an exploit does not affect me (especially if it's because I acknowledged its possibility and programmed defensively against it), why would I automatically act as if I were breached too? Your argument is totally correct if I am simply unsure---if I don't know whether an exploit affects me but I suspect it might, I assume the worst and act…

> If I have legitimate reason to believe that an exploit does not affect me (especially if it's because I acknowledged its possibility and programmed defensively against it), why would I automatically act as if I were breached too?

If you have a memory read out of a process that contains sensitive data, you should always assume that the sensitive data was breached, even if you "programmed defensively against it". Period. No exceptions.

They had good reason to believe they were unaffected, which is why they shouldn't have hit the "oh shit, the world is ending" button. But the decision to not roll their keys in the face of a breach was a massively irresponsible one. We have how many years of major bugs to look at? And in how many cases are the likes of Akamai (that is, organizations with theoretical mitigations in place) actually safe when they think they are?

Always, always assume your protections are faulty in a way that a motivated attacker will figure out.

Re: Heartbleed Update v3

#32

Post by daeken »

Narkov wrote: post #29
daeken wrote: post #18

Earlier quoted context omitted.

No, this was not a good response. It was well written and took responsibility for bad code, but it failed to address the elephant in the room: why were keys not immediately rotated? I've said this a number of times lately, but it bears repeating that the golden rule of security is: If a compromise occurs, assume that everything is compromised. Even if their approach was tested to be perfect, they shouldn't have made…

They believed that they were covered and that no compromise could/did occur. It's a mistake they now acknowledge but they have owned up to it and are taking action - just as you suggest. I believe their actions are honest and completely reasonable. If they had known they still had an issue I seriously doubt they would have publicised the patch and risk the issue they now have.

> I believe their actions are honest and completely reasonable. If they had known they still had an issue I seriously doubt they would have publicised the patch and risk the issue they now have.

I completely agree that their actions are honest, and I seriously doubt they would have played things this way if they knew their protections were faulty. I'm not saying that they're acting maliciously; I'm saying that they acted in an arrogant and irresponsible way.

If an attacker can read memory out of a process containing sensitive data, the assumption should always be that they got the keys to the kingdom even if they didn't. I'm not saying they didn't do their best to defend their service -- on the contrary, I think that their defense is a good one -- but that they played chicken with an exploit and ended up on the wrong side of it.

Their customers deserved better.

Re: Heartbleed Update v3

#33

Post by brians »

daeken wrote: post #24
[deleted] wrote: post #23

[deleted]

When a company says "we're not changing our potentially compromised keys because we're protected by this magic box" and you punch a million holes in the magic box, your goal should be to make it well known that the protection is faulty. He accomplished that quickly, and now they're finally doing the right thing.

We're not doing anything we wouldn't have done in response to mail to security@akamai.com encrypted to

pub 4096R/5B9283CE 2007-04-04 [expires: 2017-04-01] Key fingerprint = 4A76 FD9B 3603 C9F5 33F8 1490 49E0 5C45 5B92 83CE uid Akamai Security Team (General Public) sub 2048R/DF985919 2013-03-29 [expires: 2015-03-29]

Re: Heartbleed Update v3

#34

Post by sitkack »

csoandy wrote: post #22
beambot wrote: post #15

Willem's letter to Akamai addressed three areas where their patch was likely insufficient: (1) First question is then, are 'p', 'q' and 'd' the only sensitive parameters? NO. (The Chinese Remainder Theorem parameters are included in OpenSSL private keys by default and they are enough to recover the private key). (2) Second question: Does OpenSSL ever copy the private key, or parts, after the key has been read? YES. (…

We are, in fact, evaluating those other claims, and testing them in our labs. We are also evaluating claims made by other researchers, both publicly and privately; and we hope to end up with both a better library out of this, as well as a better understanding of the hazards we do and don't provide safeties against. Validating his first claim was sufficient to undermine our belief in key safety; the others are therefo…

I think it is important to remember when validating flaws, that we show that attacks are possible not they are necessarily demonstrated. Just because one cannot construct an exploit doesn't mean that something is secure.

Re: Heartbleed Update v3

#35

Post by jevinskie »

jbg_ wrote: post #25
Sami_Lehtinen wrote: post #20

Why people are only talking about SSL keys? Because Heartbleed allowed to fetch any data, that could have also included SSH keys and or any other authentication credentials / information the server got, for itself or to other services. As example if your site got Facebook API keys it's possible that those have been leaking too.

Unless your web server makes SSH connections from some kind of in-process application, why would this bug allow SSH keys to leak?

Is memory wiped with zeros by malloc/free in libc or in the kernel? If not, you're at risk.

Re: Heartbleed Update v3

#36

Post by daeken »

brians wrote: post #33
daeken wrote: post #24

Earlier quoted context omitted.

When a company says "we're not changing our potentially compromised keys because we're protected by this magic box" and you punch a million holes in the magic box, your goal should be to make it well known that the protection is faulty. He accomplished that quickly, and now they're finally doing the right thing.

We're not doing anything we wouldn't have done in response to mail to security@akamai.com encrypted to pub 4096R/5B9283CE 2007-04-04 [expires: 2017-04-01] Key fingerprint = 4A76 FD9B 3603 C9F5 33F8 1490 49E0 5C45 5B92 83CE uid Akamai Security Team (General Public) sub 2048R/DF985919 2013-03-29 [expires: 2015-03-29]

If the Heartbleed bug was still viable against your service, I'd say that he made the wrong call and should've emailed your team about it. But here, we're basically talking forensics.

I've written at length about the decision to not roll the keys as soon as Heartbleed was discovered, so I won't repeat myself, but in short: him publicizing this, as well as the Cloudflare challenge immediately getting slapped down, should make it clear to everyone that if you have a memory read, you should assume everything is compromised.

Hopefully this is a good thing for security in general.

Re: Heartbleed Update v3

#37

Post by kevinr »

sweettea wrote: post #14
sobkas wrote: post #7

I would like to know how many security related patches they are keeping for themselves? And I think not only Akamai did this. I hope that this situation will prove that security needs a bit more patch reviews and a bit less of secret sauce.

I suspect their patches are oriented toward paranoia. Their particular patch (keeping what they believed to be all critical values in a special region of memory used only for critical things) had no obvious necessity -- it didn't obviously provide more security -- but was just paranoia. Paranoia isn't always necessary, or even desirable; hence, the remainder of the patches have no obvious external utility, so they pr…

As Andy tweeted, we built that particular patch to keep unencrypted secrets off disk -- it actually was intended to provide tangible security benefit, it wasn't just paranoia: https://twitter.com/csoandy/status/455307255895060480

Re: Heartbleed Update v3

#38

Post by bzbarsky »

jevinskie wrote: post #35
jbg_ wrote: post #25

Earlier quoted context omitted.

Unless your web server makes SSH connections from some kind of in-process application, why would this bug allow SSH keys to leak?

Is memory wiped with zeros by malloc/free in libc or in the kernel? If not, you're at risk.

Yes, it is. Leaking memory contents across processes would be a pretty serious bug in the kernel. So when your process asks the kernel for memory (via sbrk or mmap) the memory it gets is zeroed out.

Re: Heartbleed Update v3

#39

Post by brians »

daeken wrote: post #36
brians wrote: post #33

Earlier quoted context omitted.

We're not doing anything we wouldn't have done in response to mail to security@akamai.com encrypted to pub 4096R/5B9283CE 2007-04-04 [expires: 2017-04-01] Key fingerprint = 4A76 FD9B 3603 C9F5 33F8 1490 49E0 5C45 5B92 83CE uid Akamai Security Team (General Public) sub 2048R/DF985919 2013-03-29 [expires: 2015-03-29]

If the Heartbleed bug was still viable against your service, I'd say that he made the wrong call and should've emailed your team about it. But here, we're basically talking forensics. I've written at length about the decision to not roll the keys as soon as Heartbleed was discovered, so I won't repeat myself, but in short: him publicizing this, as well as the Cloudflare challenge immediately getting slapped down, sho…

There I agree with you. Thanks.

Re: Heartbleed Update v3

#40

Post by daeken »

brians wrote: post #39
daeken wrote: post #36

Earlier quoted context omitted.

If the Heartbleed bug was still viable against your service, I'd say that he made the wrong call and should've emailed your team about it. But here, we're basically talking forensics. I've written at length about the decision to not roll the keys as soon as Heartbleed was discovered, so I won't repeat myself, but in short: him publicizing this, as well as the Cloudflare challenge immediately getting slapped down, sho…

There I agree with you. Thanks.

For what it's worth, I think that the patch you guys threw out there (and have been using for a while) is a great idea, and I really hope it ends up in mainline OpenSSL. I think you guys have acted 100% properly from a tech perspective in this whole thing, even if a bug did pop up (when do they not?).

I disagree strongly with the business decision made w.r.t. keys, but I hope people don't take that as me ragging on the tech team over there; you guys are doing good work.

Post reply on HN