Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Technical Details on the Recent Firefox Add-On Outage

hacks.mozilla.org

Post reply on HN
Print view View original

51–60 of 279 posts

Re: Technical Details on the Recent Firefox Add-On Outage

#51

Post by DangerousPie »

My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well.

The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN. My extensions hadn't even been disabled yet by the time the patch came out. And pushing out the hotfix through studies followed by a new version probably ensured that a large fraction of the "average joe" userbase didn't even realize there was a problem.

So obviously there are some improvements to make for the future but I think some of the criticism over the last few days has been a bit harsh. Firefox is still my preferred browser by far.

Re: Technical Details on the Recent Firefox Add-On Outage

#52

Post by Faark »

> In theory, fixing a problem like this looks simple: make a new, valid certificate and republish every add-on with that certificate. Unfortunately, [..]

I'd expect addon usage to follow a pareto distribution. Thus resigning the most important ones would have helped a lot of users. Why didn't they start going this route anyway? Not enough manpower for this to not diverted resources from the other more important fixes?

Re: Technical Details on the Recent Firefox Add-On Outage

#53

Post by callahad »

Faark wrote: post #52

> In theory, fixing a problem like this looks simple: make a new, valid certificate and republish every add-on with that certificate. Unfortunately, [..] I'd expect addon usage to follow a pareto distribution. Thus resigning the most important ones would have helped a lot of users. Why didn't they start going this route anyway? Not enough manpower for this to not diverted resources from the other more important fixes…

I expect the full postmortem will dive into this in more detail, but one reason mentioned in ekr's post is that many add-ons are distributed independently, outside of AMO. So while we could re-sign those, we're not in a place to re-distribute them. Instead, "users would have to manually update any add-ons that they had installed from other sources, which would be very inconvenient."

Re: Technical Details on the Recent Firefox Add-On Outage

#54

Post by gpm »

THIS IS WRONG: Preserved to make below reply not seem out of context.

> Second, we immediately pushed a hotfix which suppressed re-validating the signatures on add-ons.

Wait, that's not right is it!?

The only hotfix I'm aware of (and I was following this pretty closely, as you might guess by my dozens of comments on the topic) was the addon installed via the studies system.

That addon didn't suppress the re-validation of signatures, it installed a new certificate and then triggered the re-validation of signatures immediately. It left the validation that happens on a 24 hour cycle alone.

Re: Technical Details on the Recent Firefox Add-On Outage

#56

Post by KwanEsq »

gpm wrote: post #54

THIS IS WRONG: Preserved to make below reply not seem out of context. > Second, we immediately pushed a hotfix which suppressed re-validating the signatures on add-ons. Wait, that's not right is it!? The only hotfix I'm aware of (and I was following this pretty closely, as you might guess by my dozens of comments on the topic) was the addon installed via the studies system. That addon didn't suppress the re-validatio…

There was a hotfix before the one that installed a new cert:

> hotfix-reset-xpi-verification-timestamp-1548973•Complete

> This study sets app.update.lastUpdateTime.xpi-signature-verification to 1556945257.

Re: Technical Details on the Recent Firefox Add-On Outage

#57

Post by johnp_ »

jzl wrote: post #11

Looks like a good read. I haven't finished reading it yet, but there's something I still don't get ... Windows and macOS both have a signing infrastructure for apps. The rules of that infrastructure dictate only that apps must have been signed by a valid certificate at the time they were signed. That way old app downloads don't need to be periodically re-signed just to account for expiring certificates. I can downloa…

It seems Mozilla is in the process[0] of moving the signature scheme to COSE, which allows timestamping[1]. A code comment[2] says that the current package format doesn't allow it.

[0] https://bugzilla.mozilla.org/show_bug.cgi?id=1545836

[1] https://tools.ietf.org/html/rfc8152#section-4.5

[2] https://searchfox.org/mozilla-central/rev/b9da45f63cb5672449...

Re: Technical Details on the Recent Firefox Add-On Outage

#58

Post by callahad »

jeffchien wrote: post #45

I would've liked to see some reflection or even just acknowledgment about the fact that they intentionally disabled the "xpinstall.signatures.required" setting on Windows and OSX. I hope it's at least in the formal postmortem.

That's covered at https://blog.mozilla.org/addons/2015/04/15/the-case-for-exte..., which is linked from ekr's post. I encourage you to read the rationale as a whole, but the specific question you're asking is addressed here:

> Many developers have asked why we can’t make this a runtime option or preference. There is nowhere we could store that choice on the user’s machine that these greyware apps couldn’t change and plausibly claim they were acting on behalf of the user’s “choice” not to opt-out of the light grey checkbox on page 43 of their EULA. This is not a concern about hypotheticals, we have many documented cases of add-ons disabling the mechanisms through which we inform users and give them control over their add-ons. By baking the signing requirement into the executable these programs will either have to submit to our review process or take the blatant malware step of replacing or altering Firefox. We are sure some will take that step, but it won’t be an attractive option for a Fortune 500 plugin vendor, popular download sites, or the laptop vendor involved in distributing Superfish. For the ones who do, we hope that modifying another program’s executable code is blatant enough that security software vendors will take action and stop letting these programs hide behind terms buried in their user-hostile EULAs.

Re: Technical Details on the Recent Firefox Add-On Outage

#59

Post by gpm »

KwanEsq wrote: post #56
gpm wrote: post #54

THIS IS WRONG: Preserved to make below reply not seem out of context. > Second, we immediately pushed a hotfix which suppressed re-validating the signatures on add-ons. Wait, that's not right is it!? The only hotfix I'm aware of (and I was following this pretty closely, as you might guess by my dozens of comments on the topic) was the addon installed via the studies system. That addon didn't suppress the re-validatio…

There was a hotfix before the one that installed a new cert: > hotfix-reset-xpi-verification-timestamp-1548973•Complete > This study sets app.update.lastUpdateTime.xpi-signature-verification to 1556945257.

Oh, you're completely right. Oops.

Re: Technical Details on the Recent Firefox Add-On Outage

#60

Post by eps »

johncolanduoni wrote: post #32
jzl wrote: post #11

Looks like a good read. I haven't finished reading it yet, but there's something I still don't get ... Windows and macOS both have a signing infrastructure for apps. The rules of that infrastructure dictate only that apps must have been signed by a valid certificate at the time they were signed. That way old app downloads don't need to be periodically re-signed just to account for expiring certificates. I can downloa…

For the time stamp method to work, you need a trusted mechanism to attest that the timestamp is correct, otherwise the mechanism is useless (an attacker with an outdated private key can just backdate the timestamp in the executable and then sign it). Windows code signing uses a server Microsoft runs to provide this, and Mozilla would need to do the same. I’m not saying they shouldn’t, but it is a significant piece of…

Every major CA runs a trusted timestamping service. Mozilla doesn't need to maintain their own timestamping infrastructure, they could delegate to one of the CAs, probably based on some sort of formal agreement with them.

Though the way things stand, all CAs have no problem timestamping sigs made with certs that are from other CAs, so perhaps even no explicit agreement is required.

Post reply on HN