Heavy SSD Writes from Firefox • Hacker News Board

Re: Heavy SSD Writes from Firefox

#191

Post by Dylan16807 » Fri, Sep 23, 2016, 6:23 PM UTC

Yoric wrote: post #156

nograpes wrote: post #130

Earlier quoted context omitted.

What would an updated version of Session Restore look like? I mean, what would be your vision for a new version?

I haven't worked on Session Restore in some time. But from the top of my head, it would be as follows. On the back-end, I believe that Session Restore should be backed by a database, with each tab updated independently, rather than a big bunch of JSON data. The rationale being that: - it's performance-critical; - it's safety-critical; - we have users with 300+ Mb of data in their Session Restore and JSON isn't meant…

How does that much data get into Session Restore? Mine are only about 3KB per tab, and Firefox starts melting at more than a thousand tabs.

Re: Heavy SSD Writes from Firefox

#192

Post by matt4077 » Fri, Sep 23, 2016, 6:27 PM UTC

lucb1e wrote: post #84

lighttower wrote: post #8

Chrome, on my system, is even more abusive. Watch the size of the .config/google-chrome directory and you'll see that it grows to multi-GB in the profile file. There is a Linux utility that takes care of all browsers' abuse of your ssd called profile sync daemon, PSD. It's available in the debian repo or [1] for Ubuntu or [2] for source. It uses `overlay` filesystem to direct all writes to ram and only syncs back to…

Launchpad is such a shitty website, aimed at Ubuntu and only Ubuntu, links to source code or more information are nowhere to be found... Searching on Github, this seems to be it. Turns out, there's releases for Arch, Debian, etc. and it's even in the repositories. No need to add a ppa. https://github.com/graysky2/profile-sync-daemon For Debian and co: $ apt-cache show profile-sync-daemon $ sudo apt-get install profil…

I checked the Activity Monitor on OS X to see if it's the same and if I should do something. It's quite surprising:

    bird	         11,38 GB   119,8 MB		
    kernel_task	          8,82 GB    23,0 MB
    launchd	          8,46 GB    77,7 MB	
    revisiond	          8,35 GB   105,8 MB	
    Tweetbot	          3,01 GB    50,2 MB	
    Safari	        879,6 MB     10,0 MB	
    mds_stores          726,2 MB    570,1 MB		
    systemstatsd        695,0 MB    775,0 MB
    nsurlstorag         685,3 MB      9,5 MB	
    Google Chrom        681,4 MB    303,5 MB		
    iTerm2	        653,7 MB      1,6 GB	
    cfprefsd	        509,1 MB     45,8 MB	
    Papers 3.4.7        465,9 MB    169,8 MB
    cloudd	        441,3 MB      9,7 MB
    ruby	        224,6 MB      1,7 MB	
    coreduetd	        217,5 MB		
    Reeder	        207,2 MB     28,5 MB 
    apsd	         65,7 MB     11	MB    
    Safari	         55,3 MB     12,6 MB

I've long gotten used to "bird" doing it's thing (something with the cloud I guess). But how can a twitter client write 3GB (while I'm not even actually using it?)

Re: Heavy SSD Writes from Firefox

#193

Post by rackforms » Fri, Sep 23, 2016, 6:27 PM UTC

gcp wrote: post #155

rackforms wrote: post #117

Earlier quoted context omitted.

Progress at what cost? Does maintaining security have to also mean breaking the systems that pay my bills?

I do not think you want to be paying your bills on a browser that's been compromised, no.

It's a dev box that only touches local resources.

Re: Heavy SSD Writes from Firefox

#194

Post by bsdetector » Fri, Sep 23, 2016, 6:28 PM UTC

Yoric wrote: post #142

fulafel wrote: post #134

Earlier quoted context omitted.

How about this: 29 out of 30 times, save only a diff to the previous data. 1 out of 30 times, save the complete data in compressed form. (I'm guessing there must already be functionality to diff a bunch of JSON somewhere in the millions of lines of code). Though I'm sure this doesn't make usually make a dent in a SSD's lifetime. But there are still people running Firefox on low end Android phones with meager flash, a…

> How about this: 29 out of 30 times, save only a diff to the previous data. 1 out of 30 times, save the complete data in compressed form. > > (I'm guessing there must already be functionality to diff a bunch of JSON somewhere in the millions of lines of code). That is actually a good idea we haven't considered yet. A bit too brute force for my tastes, but relatively easy to implement. We would need to determine how…

You could at least just compress the data with LZF or any of the really fast text compressors.

It'll compress to about 30% of the size, it's easy to do, and it shouldn't add more than a tiny CPU overhead over formatting the JSON itself.

It solves half the problem with like 15 minutes of work.

Re: Heavy SSD Writes from Firefox

#195

Post by zbuf » Fri, Sep 23, 2016, 6:28 PM UTC

icebraining wrote: post #167

zbuf wrote: post #67

I have been running Firefox for a long time with an LD_PRELOAD wrapper which turns fsync() and sync() into a no-op. I feel it's little antisocial for regular desktop apps to assume it's for them to do this. Chrome is also a culprit, a similar sync'ing caused us problems at my employer's, inflated pressure on an NFS server where /home directories are network mounts. Even where we already put the cache to a local disk.…

I don't think the use of SQLite is a problem, since it can be configured to be very careful or very reckless. http://www.sqlite.org/pragma.html#pragma_synchronous

It's a problem if it's not configured appropriately. Remember the use of SQLite is entirely internal to the application; it's not for users to know how it's used, or even that it's used at all.

In the case of a Chrome (many versions ago; may have changed), we manually tweaked that pragma in all our users profiles. I recall something like an order of magnitude reduction of pressure on the /home server.

Re: Heavy SSD Writes from Firefox

#196

Post by towerbabbel » Fri, Sep 23, 2016, 6:28 PM UTC

I observed something similar several years ago: http://www.overclockers.com/forums/showthread.php/697061-Whe...

I still think the worry about it wearing out an SSD is overblown. The 20GB per day of writes is extremely conservative and mostly there to avoid more pathological use cases. Like taking a consumer SSD and using it as the drive for some write heavy database load with 10x+ write amplification and when you wear it out demand a new one on warranty.

Backing up the session is still sequential writes so write amplification is minimal. After discovering the issue I did nothing and just left Firefox there wearing on my SSD. I'll still die of old age before Firefox can wear it out.

Re: Heavy SSD Writes from Firefox

#197

Post by Animats » Fri, Sep 23, 2016, 6:29 PM UTC

Firefox is relying too much on session restore to deal with bugs in their code. Firefox needs to crash less. With all the effort going into multiprocess Firefox, Rust, and Servo, it should be possible to have one page abort without taking down the whole browser. About half the time, session restore can't restore the page that crashed Firefox anyway.

Re: Heavy SSD Writes from Firefox

#198

Post by cbhl » Fri, Sep 23, 2016, 6:29 PM UTC

gcp wrote: post #157

fulafel wrote: post #143

Earlier quoted context omitted.

Firefox does generally assume that the filesystem is reliable (as long as you use fsync properly etc). Witness eg its reliance on SQLite for data storage.

SQLite is used exactly because it is so robust even on bad hardware. And you still need to check the data that SQLite is giving you.

Until you have a NFS homedir...

Re: Heavy SSD Writes from Firefox

#199

Post by digi_owl » Fri, Sep 23, 2016, 6:30 PM UTC

I do wonder if their mobile version have a similar problem. I have noticed it chugs badly when opened for the first time in a while on Android, meaning i have to leave it sitting or a while so it can get things done before i can actually browse anything.

Re: Heavy SSD Writes from Firefox

#200

Post by Dylan16807 » Fri, Sep 23, 2016, 6:33 PM UTC

froydnj wrote: post #173

blinkingled wrote: post #14

Earlier quoted context omitted.

Right but that doesn't happen quite often either. Also if it does how important is restoring the tabs and windows? Probably not to a lot of people other than serious creators that use a web editor of some sort that'll lose work if the browser disappears. Even then it is flaky to rely on something like this. Much better to have periodic backups of whatever you're doing.

You say "doesn't happen quite often", which I'm assuming means something like "doesn't happen quite often to me". But if you consider hundreds of millions of users using Firefox every day, then even a relatively rare event is happening to multiple users every single day. It is absolutely worth it to them that Firefox does its best to preserve their data.

You have a point, but power outages are very correlated.