Live data from Hacker News

Technical Details on the Recent Firefox Add-On Outage

hacks.mozilla.org

171–180 of 279 posts

Re: Technical Details on the Recent Firefox Add-On Outage

#171

Earlier quoted context omitted.

In this day and age you're living on borrowed time using an old version of a web browser. This state of affairs has its good and bad, but your modern-day browser vendors don't typically maintain branches of old versions to make security and bug fixes (especially when they change things for security reasons, as with this case of old-style addons being removed). If you're going to insist on using an old unsupported ver…

Well I've just been educated on a new mechanism by which things can be remotely broken.

Just to be clear: supported version or not, this sucks, and I hope we'll have a fix for you.

I wanted to point out this wasn't remotely broken, however. Even if you had no internet connection, your addons would have stopped working when the certificate expired.

(Disclosure: I work for Mozilla)

Re: Technical Details on the Recent Firefox Add-On Outage

#172

My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well. The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN. My extensions hadn't even been disabled yet by the time the patch came out. And pushing out the hotfix…

> My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well.

I think they handled it terribly… or rather, they handled the same event three years ago terribly. There's a reason this is called armagadd-on 2.0… that cert had expired once already, the previous fix didn't work. I think last time they actually had a week of lead time, too.

Re: Technical Details on the Recent Firefox Add-On Outage

#173
post #114
post #69

Earlier quoted context omitted.

I think this is covered in the blog post, but I'll take another stab at explaining it from my perspective: Republishing was one of the options we were investigating early on. However, the problem is that it only fixes things once you check for and install the updated version signed with the new certificate. Firefox would still have disabled your installed version that had an expired certificate. Firefox checks for ad…

Thanks for your response. An update being deactivated doesn't trigger an update check for said addon? Well, complexity of a graceful update attempt for a not loadable addon probably outweighs the benefit of such rare cases. But the main reason I had a hard time with you guys discarding this path is probably addon stats like [0]. The spike in downloads made me think it could have helped at least some users. But on sec…

This is a good point; I'll try to make sure it's raised when we conduct our postmortem.

(Disclosure: I work for Mozilla)

Re: Technical Details on the Recent Firefox Add-On Outage

#174

Earlier quoted context omitted.

I'm sorry that you felt like you were held hostage by Telemetry/Studies. With the exception of the hotfix, we disabled rolling out new Studies during the incident, and will not be re-enabling them until some time after Monday next week. We are also completely deleting all Telemetry and Studies data received in the week following the incident to ensure we respect people who had concerns like yours, but enabled Studies…

"I'm sorry that you felt like you were..." is the worst form of apology, because it admits no guilt or responsibility. "I'm sorry that you were..." or "I'm sorry that we..." would be a legitimate apology. That said, nuking this data is the first good thing Mozilla has done in this whole fiasco. It's a small but real act of contrition, so kudos for that.

> "I'm sorry that you felt like you were..." is the worst form of apology, because it admits no guilt or responsibility. "I'm sorry that you were..." or "I'm sorry that we..." would be a legitimate apology.

GP used the word "felt" and was expressing that he felt a certain way about enabling Studies. You're nit-picking a conversation and it has gone like this:

A: I felt that $x.

B: I'm sorry that you felt that $x.

C: "I'm sorry that you felt ..." is an insincere apology.

Yes, some people use this trick to get out of admitting guilt or responsibility but this is not an example of that.

Re: Technical Details on the Recent Firefox Add-On Outage

#175
post #66

Earlier quoted context omitted.

It was a couple of days before the Android fix was released. For many users this was a multiday affair. Some lost data (containers). You were lucky you extentions were not disabled before the fix, but for many people this was a major problem.

I unfortunately lost my container data (luckily I only have a couple set up). To get the fix though, I had to opt in to the Firefox studies. Apparently I had opted out at some point in the past.

Opt back out if you haven't already.

Re: Technical Details on the Recent Firefox Add-On Outage

#176

Related to this: Mozilla has deleted Telemetry data for those users who enabled Telemetry to get the hot-fix [1] [1] https://twitter.com/firefox/status/1126593558490693632

Thanks for sharing this. I didn't know they were doing that. This helps restore some of my faith.

Re: Technical Details on the Recent Firefox Add-On Outage

#177
post #97

> users should be able to opt-in to updates (including hot-fixes) but opt out of everything else Finally some good news. This is what I suggested in one of the previous threads: there should be a delivery channel for important updates, and a channel for experiments/telemetry/whatnot. Some other HNer said it was an unrealistic expectation "because manpower". Guess what, it isn't. This is how things should always be.

They did have that capability (see https://wiki.mozilla.org/Firefox/Go_Faster/System_Add-ons/Pr... ), at least the linked repository had commits in 2016. I can't tell from a 30 second search why that no longer works, just that it is a replacement of a similar previous capability.

Re: Technical Details on the Recent Firefox Add-On Outage

#178
post #11

Looks like a good read. I haven't finished reading it yet, but there's something I still don't get ... Windows and macOS both have a signing infrastructure for apps. The rules of that infrastructure dictate only that apps must have been signed by a valid certificate at the time they were signed. That way old app downloads don't need to be periodically re-signed just to account for expiring certificates. I can downloa…

For the time stamp method to work, you need a trusted mechanism to attest that the timestamp is correct, otherwise the mechanism is useless (an attacker with an outdated private key can just backdate the timestamp in the executable and then sign it). Windows code signing uses a server Microsoft runs to provide this, and Mozilla would need to do the same. I’m not saying they shouldn’t, but it is a significant piece of…

[deleted]

Re: Technical Details on the Recent Firefox Add-On Outage

#179

Hate on Google / Chrome / Proprietary / Lock-In / Whatever buzzword (tm) you want, but this would _never_ happen in Chrome. And it's the reason why Firefox will die to Chrome. BRING ON THE IN-DENIAL DOWNVOTES.

That's because they actually care about users. Mozilla only cares about all its millions of inconsequential hobby projects.

Re: Technical Details on the Recent Firefox Add-On Outage

#180

My heartfelt condolences to everyone who has had to browse the internet without an ad blocker. Nobody deserves that.

This is why you should have multi-tier defenses. An ad blocker in the browser, blackholed domains in your hosts file, and use DNS servers that also blackholes ad domains.
Post reply on HN