Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Technical Details on the Recent Firefox Add-On Outage

hacks.mozilla.org

Post reply on HN
Print view View original

61–70 of 279 posts

Re: Technical Details on the Recent Firefox Add-On Outage

#61

Post by callahad »

gilrain wrote: post #30
mmanfrin wrote: post #18

Earlier quoted context omitted.

I think using the system they used to provide a hotfix for a browser-breaking issue does not clash with the spirit of their prior pledge. I feel a complaint like this verges unhelpfully in to the pedantic.

As someone who opted out of Studies after the last abuse, I felt betrayed that my addons were, effectively, held hostage behind enabling both telemetry and Studies. I decided to wait. It’s bad optics at the very least. Users who opted in for the update were in fact entered into studies they explicitly wouldn’t have wanted to be in without the lure of an earlier update.

I'm sorry that you felt like you were held hostage by Telemetry/Studies. With the exception of the hotfix, we disabled rolling out new Studies during the incident, and will not be re-enabling them until some time after Monday next week.

We are also completely deleting all Telemetry and Studies data received in the week following the incident to ensure we respect people who had concerns like yours, but enabled Studies in order to receive the hotfix.

Specific details and timestamps are in the post at https://blog.mozilla.org/blog/2019/05/09/what-we-do-when-thi...

Re: Technical Details on the Recent Firefox Add-On Outage

#62

Post by cpeterso »

gilrain wrote: post #30
mmanfrin wrote: post #18

Earlier quoted context omitted.

I think using the system they used to provide a hotfix for a browser-breaking issue does not clash with the spirit of their prior pledge. I feel a complaint like this verges unhelpfully in to the pedantic.

As someone who opted out of Studies after the last abuse, I felt betrayed that my addons were, effectively, held hostage behind enabling both telemetry and Studies. I decided to wait. It’s bad optics at the very least. Users who opted in for the update were in fact entered into studies they explicitly wouldn’t have wanted to be in without the lure of an earlier update.

From another Mozilla blog post about the incident response:

> "In order to respect our users’ potential intentions as much as possible, based on our current set up, we will be deleting all of our source Telemetry and Studies data for our entire user population collected between 2019-05-04T11:00:00Z and 2019-05-11T11:00:00Z."

https://blog.mozilla.org/blog/2019/05/09/what-we-do-when-thi...

Re: Technical Details on the Recent Firefox Add-On Outage

#63

Post by bwat49 »

chrisseaton wrote: post #48
clucas wrote: post #46

Earlier quoted context omitted.

The SHIELD policies they promulgated after that incident has to do with studies. This was not a study. This was an emergency band-aid to help as many users as possible get their browsers back to the way they (the users) chose to make them work by re-enabling their add-ons, using the fastest avenue available. This was a user-friendly move, and while it's unfortunate that it was necessary in the first place, your criti…

> This was not a study. I don’t get it. Distributing something that was not a study as a study was what upset people last time. So they promised not to do it again. And they have just done it again.

The outrage over looking glass was because it was an unwanted advertisement/promotion being pushed as a 'study'

In this case, they were pushing a hotfix for a critical issue effecting all firefox users. It was only pushed through the studies system because it was the best option to get the fix to as many users as possible while they worked on a firefox update.

Implying that this is in any way equivalent to looking glass is completely disingenuous.

Re: Technical Details on the Recent Firefox Add-On Outage

#64

Post by SECProto »

DangerousPie wrote: post #51

My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well. The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN. My extensions hadn't even been disabled yet by the time the patch came out. And pushing out the hotfix…

> The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN

Maybe this is a timezone thing, but I was in East Asia, and I had to deal with the internet for close to 36 hrs (android) with no ublock. It was almost enough to look for a new browser (but browsers with adblock on Android are few and far between - so instead I just didn't use the internet as much for a day or two.). Part of that delay was play store being slow to push it, as I recall seeing the binaries somewhere a while sooner.

Re: Technical Details on the Recent Firefox Add-On Outage

#65

Post by jancsika »

chrisseaton wrote: post #48
clucas wrote: post #46

Earlier quoted context omitted.

The SHIELD policies they promulgated after that incident has to do with studies. This was not a study. This was an emergency band-aid to help as many users as possible get their browsers back to the way they (the users) chose to make them work by re-enabling their add-ons, using the fastest avenue available. This was a user-friendly move, and while it's unfortunate that it was necessary in the first place, your criti…

> This was not a study. I don’t get it. Distributing something that was not a study as a study was what upset people last time. So they promised not to do it again. And they have just done it again.

> Distributing something that was not a study as a study was what upset people last time.

People got upset because an add-on for a commercial TV series auto-magically installed itself in their browser without their manual intervention.

You can't just abstract that out into an ethics framework of, "If Firefox sends non-X over a channel reserved for X, then users get upset." I mean, you can, but you're going to be rightly confused and misunderstood.

Re: Technical Details on the Recent Firefox Add-On Outage

#66

Post by steve19 »

DangerousPie wrote: post #51

My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well. The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN. My extensions hadn't even been disabled yet by the time the patch came out. And pushing out the hotfix…

It was a couple of days before the Android fix was released. For many users this was a multiday affair. Some lost data (containers).

You were lucky you extentions were not disabled before the fix, but for many people this was a major problem.

Re: Technical Details on the Recent Firefox Add-On Outage

#67

Post by bwat49 »

fedups wrote: post #16
chrisseaton wrote: post #8

After the last PR disaster - the Mr Robot tie-in - one of the ways Mozilla tried to make it right was that they promised that the survey system would never again be used for something that was not an experiment. https://blog.mozilla.org/firefox/retrospective-looking-glass... > A SHIELD study must be designed to answer a specific question. Why have they abused it again here to deploy a hot fix, breaking their promise…

Do they have a better channel for deploying hot-fixes? Maybe I'm a bit of a squish but I'm pretty agnostic of how they deploy fixes to me so long as my addons start working again.

They don't currently, but adding a better channel to deploy hotfixes is something that's specifically mentioned in this article...

Re: Technical Details on the Recent Firefox Add-On Outage

#69

Post by sciurus »

Faark wrote: post #52

> In theory, fixing a problem like this looks simple: make a new, valid certificate and republish every add-on with that certificate. Unfortunately, [..] I'd expect addon usage to follow a pareto distribution. Thus resigning the most important ones would have helped a lot of users. Why didn't they start going this route anyway? Not enough manpower for this to not diverted resources from the other more important fixes…

I think this is covered in the blog post, but I'll take another stab at explaining it from my perspective:

Republishing was one of the options we were investigating early on. However, the problem is that it only fixes things once you check for and install the updated version signed with the new certificate. Firefox would still have disabled your installed version that had an expired certificate.

Firefox checks for addon updates every 24 hours, but it checks in with Normandy every 6 hours. Thus once we had stopgap fixes shipping to users via Normandy, and a proper fix of a new Firefox release in progress, republishing addons wasn't necessary.

(Disclaimer: I work for Mozilla)

Re: Technical Details on the Recent Firefox Add-On Outage

#70

Post by dave5104 »

steve19 wrote: post #66
DangerousPie wrote: post #51

My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well. The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN. My extensions hadn't even been disabled yet by the time the patch came out. And pushing out the hotfix…

It was a couple of days before the Android fix was released. For many users this was a multiday affair. Some lost data (containers). You were lucky you extentions were not disabled before the fix, but for many people this was a major problem.

I unfortunately lost my container data (luckily I only have a couple set up).

To get the fix though, I had to opt in to the Firefox studies. Apparently I had opted out at some point in the past.

Post reply on HN