Live data from Hacker News

Technical Details on the Recent Firefox Add-On Outage

hacks.mozilla.org

221–230 of 279 posts

Re: Technical Details on the Recent Firefox Add-On Outage

#221
post #11

Looks like a good read. I haven't finished reading it yet, but there's something I still don't get ... Windows and macOS both have a signing infrastructure for apps. The rules of that infrastructure dictate only that apps must have been signed by a valid certificate at the time they were signed. That way old app downloads don't need to be periodically re-signed just to account for expiring certificates. I can downloa…

For the time stamp method to work, you need a trusted mechanism to attest that the timestamp is correct, otherwise the mechanism is useless (an attacker with an outdated private key can just backdate the timestamp in the executable and then sign it). Windows code signing uses a server Microsoft runs to provide this, and Mozilla would need to do the same. I’m not saying they shouldn’t, but it is a significant piece of…

It depends on the level of security you want.

Let's say you compromise by letting old signatures stay valid, but only for a year. And you rotate the intermediate cert every 90 days.

This system is more secure than the old one, because you only have to worry about key leaks for 15 months, instead of years.

But at the same time, it's impossible to have this giant wave of everything failing at once. Instead the best case is nobody can sign extensions for a couple days, and the worst case is extensions that updated exactly a year ago start to fail in real time.

Re: Technical Details on the Recent Firefox Add-On Outage

#222
post #57

Earlier quoted context omitted.

It seems Mozilla is in the process[0] of moving the signature scheme to COSE, which allows timestamping[1]. A code comment[2] says that the current package format doesn't allow it. [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1545836 [1] https://tools.ietf.org/html/rfc8152#section-4.5 [2] https://searchfox.org/mozilla-central/rev/b9da45f63cb5672449...

If that is true, they should mention this in the post-mortem. Code signing is a well understood problem with a well known solution, but the blog post discusses everything except the well known solution. Right now you have a problem caused directly by lack of time stamping, and the article doesn’t even acknowledge that. That’s not inspiring confidence. I’m genuinely still not sure if they have understood what the actu…

> If that is true, they should mention this in the post-mortem.

They might:

> We’ll be running a formal post-mortem next week and will publish the list of changes we intend to make

The lessons noted down here are just some thoughts by the author of this blog post:

> but in the meantime here are my initial thoughts about what we need to do.

Re: Technical Details on the Recent Firefox Add-On Outage

#223

Earlier quoted context omitted.

We do not take an editorial stance on add-on content with regard to signing, but we do have the ability to block add-ons that are malicious or which violate user privacy and security. We hope those choices will be few and far between, and that our users will agree with them. Our Add-on Policy is discussed further at https://blog.mozilla.org/addons/2019/05/02/add-on-policy-and...

FWIW, I was genuinely shocked that this could happen, and it has severely damaged my trust in Firefox. I am extremely wary, having been bitten more than once, about software that automatically updates itself or is otherwise subject to remote interference. I don't run Windows 10. I don't use Chrome for anything important. I avoid subscription-based or activation-required software as much as possible. And in Firefox, I…

> I find the argument that it's impossible to make this configurable because malware could then circumvent it very weak. If we're talking about that level of interference, anyone with access to the Firefox executable could in theory replace it, and given the open source nature of the Firefox codebase this wouldn't be particularly difficult technically for anyone willing to go to such lengths in the first place.

As mentioned in GGP's quote, Firefox was specifically modified by popular software by Fortune 500 companies and laptop companies, which feel safe enough to modify user preferences, but not to replace the Firefox executable. This does specifically fix that real-world attack vector.

> In any case, even if the argument about hard-coding protection into the executable did stand up to scrutiny, there are alternatives possible instead of retrospectively disabling addons with no possible workaround. Perhaps most obviously, you could show a warning message and require explicit user approval at startup before activating the addon, for example, as is already done with various other useful features that are also potentially open to abuse.

I think that would be categorised as an option "that these greyware apps [could] change and plausibly claim they were acting on behalf of the user’s “choice” not to opt-out of the light grey checkbox on page 43 of their EULA".

> As things stand, far from protecting our privacy and security from malicious addons, the current system in fact deactivated all of our privacy- and security-protecting addons, without warning, right in the middle of browsing sessions.

That this happened was a risk, and they're taking active measures in the future. The other was a certainty, and this risk was the active measure they were taking against it.

Re: Technical Details on the Recent Firefox Add-On Outage

#224
post #90

Earlier quoted context omitted.

I find it hard to marry up "Firefox user that cares about privacy" with being happy about the study mechanism. Don't you, from a privacy perspective, find it more than a little disturbing that the study mechanism has so much access to internal APIs in Firefox that it can install certificates without your involvement? That seems like a crazy security risk, let alone privacy risk. It's built in and enabled _by default_…

It’s funny how the mind works. We tend to accept the far greater risk of allowing automated software updates (either in-app or blindly trusting your package manager e.g. apt-get upgrade ) but the Firefox Studies mechanism which does a minuscule, strict subset of what any automatic update can do is somehow a “crazy security risk”.

I chose to run apt-get to install packages. I have never opted-in to, nor even known about, how Firefox was running "studies" on me before this event. It's like finding out the TV you bought last year has a hidden camera and it's been recording you the whole time.

Re: Technical Details on the Recent Firefox Add-On Outage

#225

Earlier quoted context omitted.

Well I've just been educated on a new mechanism by which things can be remotely broken.

What did you expect to happen? You're using an online service connected to signing authorities. Of course it can be remotely broken at any time, that's how the internet works, it's all remote.

He's using a product. A product. Having it attached to on-line service is not a feature.

I think part of the reason this fiasco stirs up so much emotions is because normal people, including tech professionals, still expect the browser to behave like a product, not a service. And products aren't supposed to randomly break like that, they aren't supposed to ship with a time bomb attached.

Re: Technical Details on the Recent Firefox Add-On Outage

#226
post #11

Looks like a good read. I haven't finished reading it yet, but there's something I still don't get ... Windows and macOS both have a signing infrastructure for apps. The rules of that infrastructure dictate only that apps must have been signed by a valid certificate at the time they were signed. That way old app downloads don't need to be periodically re-signed just to account for expiring certificates. I can downloa…

How can you tel an app has actually been signed in the past? It could just as well been signed today with a fake date. This defeats the purpose of expiry. You could just as well have certificates that never expire and just start signing with a new one if you feel like it. Note the Mozilla ‘solution’ has the same problem: their root now authorized a new certificate with the old public key. If the original expectation…

  It could just as well been signed today with a fake date.
The traditional option (used by things like Java and ActiveX since the early 2000s) is Trusted Timestamping [1], where a trusted third party provides a signature (such as via RFC 3161 Time Stamp Protocol). AFAIK every certificate authority that sells code-signing certificates also provides a free timestamping server. If you trust a CA to issue code-signing certificates, the theory is you'd trust them to timestamp too.

If you don't like having the trusted third party, you can also publish the hash in some write-once medium, such as on the blockchain, in certificate transparency logs, or in the small ads of a reputable newspaper.

[1] https://en.wikipedia.org/wiki/Trusted_timestamping

Re: Technical Details on the Recent Firefox Add-On Outage

#227

My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well. The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN. My extensions hadn't even been disabled yet by the time the patch came out. And pushing out the hotfix…

Handled it very well? That must be the worst hyperbole I have ever seen in a while. It broke extensions suddenly for millions of users. If you went on Firefox.com there was absolutely no news about it. Completely awful lack of communication. You had to find a minuscule banner on the "find a fix" page in order to hear about the problem. Add to that that the recommended fix was to activate the backdoor of Firefox, that…

> And no admission of guilt anywhere.

> We strive to make Firefox a great experience. Last weekend we failed, and we’re sorry. (...) We let you down

https://blog.mozilla.org/blog/2019/05/09/what-we-do-when-thi...

Re: Technical Details on the Recent Firefox Add-On Outage

#228

Earlier quoted context omitted.

Does Windows et al really not provide a mechinism for saving privileged settings in a tamper-resistant way? I frankly find that hard to believe. How does other software solve this problem? There are of course always workarounds on an open platform like Windows/Mac/Linux, but the threshold isn’t “impossible”, it’s just “as difficult as injecting into the browser’s code.” Edit: For example, what if the config file cont…

I mean, the Windows UAC and the MacOS password prompt that elevate user access, but people often click through those. I can see their argument here.

No amount of security measures can stop users from being tricked by social engineering attack into self-pwnig themselves. At some point we need to stop taking control away from users, because this road ultimately leads to turning computers into cable TV.

Re: Technical Details on the Recent Firefox Add-On Outage

#229
post #90

My point of view as a long-time Firefox user that cares about privacy but also knows we live in an imperfect world: It obviously sucks that this happened but I think they handled it very well. The bug was fixed so quickly that I wouldn't even have realized it had happened if it hadn't been for the thread here on HN. My extensions hadn't even been disabled yet by the time the patch came out. And pushing out the hotfix…

I find it hard to marry up "Firefox user that cares about privacy" with being happy about the study mechanism. Don't you, from a privacy perspective, find it more than a little disturbing that the study mechanism has so much access to internal APIs in Firefox that it can install certificates without your involvement? That seems like a crazy security risk, let alone privacy risk. It's built in and enabled _by default_…

I guess I am just a bit more optimistic about Mozilla as a company than you might be. Sure they could abuse the study mechanism to spy on people and hack my browser. I just assume that they have neither the motivation nor the lack of ethics to do so. And I have automatic updates enabled anyway, so they can already install whatever binaries they want on my machine.

The other thing is that it's not like there are many other options for me. The only mainstream browsers I could realistically use are Chrome, Firefox and Edge. Out of those I think Firefox probably cares the most about my privacy.

Re: Technical Details on the Recent Firefox Add-On Outage

#230
post #11

Looks like a good read. I haven't finished reading it yet, but there's something I still don't get ... Windows and macOS both have a signing infrastructure for apps. The rules of that infrastructure dictate only that apps must have been signed by a valid certificate at the time they were signed. That way old app downloads don't need to be periodically re-signed just to account for expiring certificates. I can downloa…

How can you tel an app has actually been signed in the past? It could just as well been signed today with a fake date. This defeats the purpose of expiry. You could just as well have certificates that never expire and just start signing with a new one if you feel like it. Note the Mozilla ‘solution’ has the same problem: their root now authorized a new certificate with the old public key. If the original expectation…

The code-signing needs to happen through a trusted / certified timestamp server, it's not using the local timestamp. This process basically guarantees that the application used a valid certificate at the time it was signed, and that it wasn't tampered with since then.

With this, the time when code-signed application is executed doesn't matter at all, fake date or not.

Post reply on HN