Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tmpfs considered harmful

rwmj.wordpress.com

Post reply on HN
Print view View original

21–30 of 85 posts

Re: Tmpfs considered harmful

#21

Post by RobAley »

sehrope wrote: post #9
praptak wrote: post #4

Minor disagreement about the following: "Everyone must worry about whether their files need to survive a reboot" I believe there was never such expectation of /tmp. On many systems it's being wiped periodically or even when the user's last session ends. The example app(s) that use /tmp for drafts need(s) to be fixed.

This is exactly what I use it for on my desktop. I have any app that I don't want to maintain longterm history for it's internal files (ex: Firefox browser cache) pointed to /tmp (mounted as tmpfs). As an added bonus if you're running your OS on an SSD (which you should be) you don't have it thrashed by constant writes/deletes. The memory size issue is basically a non-issue at this point (pair of desktop 16 GB DIMMS…

The memory size issue is very much still an issue. I happen to have globs of memory on my machine, but for the very reason that I have software to use it. If large tmp files start eating into the resources available to my programs, its an issue for me. Also, older hardware that has less memory available, and even newer hardware that can have more memory installed but with non-tech users who don't know how to upgrade it or even that its possible (or can't afford $80).

Re: Tmpfs considered harmful

#22

Post by RobAley »

raverbashing wrote: post #7

"Everyone must now be careful never to store a file in /tmp that might grow large" If you're going to trash my /tmp with gigantic files please don't. Really "it’s better to fix the filesystem to make it faster" Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place So basically, he contradicts himself…

>If you're going to trash my /tmp with gigantic files please don't. Really

I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session and my disks kept filling up, so I decided to use /tmp and let the OS manage the space. Is there a better way to manage large tmp files than this that I'm missing?

Re: Tmpfs considered harmful

#23

Post by buster »

Oh, i hope they revert this change!

The big trouble for me would be "Everyone must now be careful never to store a file in /tmp that might grow large". I often use /tmp for all kinds of temporary storage. Be it a small text file or a GB big tar file.

Re: Tmpfs considered harmful

#24

Post by raverbashing »

RobAley wrote: post #22
raverbashing wrote: post #7

"Everyone must now be careful never to store a file in /tmp that might grow large" If you're going to trash my /tmp with gigantic files please don't. Really "it’s better to fix the filesystem to make it faster" Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place So basically, he contradicts himself…

>If you're going to trash my /tmp with gigantic files please don't. Really I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session a…

Depends on your use case and how much of that complexity you wish to manage

In your case, what could work is having the new run use the same file names as the old run, or coming up with a rotation schedule (so you would keep the latest N runs)

But if having them on /tmp as it is working for you, great, no need to complicate things.

There are also speed considerations, if you're using these files as actual intermediate storage (as opposed to a log/debug dump) it may be interesting to have them written to a different disk (like an SSD)

Re: Tmpfs considered harmful

#25

Post by rwmj »

colanderman wrote: post #19
rwmj wrote: post #11

Earlier quoted context omitted.

Sure, but if the power goes down, there's at least a chance that I may be able to find that long email I was writing. With /tmp on tmpfs, it's gone.

That's why the GP considers mutt broken. Drafts should be stored in ~/.mutt or /var, not /tmp. Files in /tmp should not be expected to survive a process.

This is exactly the point of the posting. Every program has to be examined for this problem, and it forces a complex decision about the storage hierarchy onto every programmer.

Re: Tmpfs considered harmful

#26

Post by masklinn »

raverbashing wrote: post #7

"Everyone must now be careful never to store a file in /tmp that might grow large" If you're going to trash my /tmp with gigantic files please don't. Really "it’s better to fix the filesystem to make it faster" Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place So basically, he contradicts himself…

> If you're going to trash my /tmp with gigantic files please don't.

Where else are temporary files supposed to go, exactly? It's no abuse to put in /tmp temporary "scratch" data which have no reason to survive a process restart let alone a machine reboot, that's the mount point's explicit purpose.

Re: Tmpfs considered harmful

#27

Post by mpyne »

RobAley wrote: post #22
raverbashing wrote: post #7

"Everyone must now be careful never to store a file in /tmp that might grow large" If you're going to trash my /tmp with gigantic files please don't. Really "it’s better to fix the filesystem to make it faster" Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place So basically, he contradicts himself…

>If you're going to trash my /tmp with gigantic files please don't. Really I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session a…

Given that they should persist, the better way would be /var/tmp I would think, at least if persistence means "should survive reboot".

Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents of /tmp at startup (and I know Gentoo/OpenRC is).

There might be an even more appropriate FHS-compliant directory (/var/run perhaps?)

Re: Tmpfs considered harmful

#28

Post by RobAley »

mpyne wrote: post #27
RobAley wrote: post #22

Earlier quoted context omitted.

>If you're going to trash my /tmp with gigantic files please don't. Really I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session a…

Given that they should persist, the better way would be /var/tmp I would think, at least if persistence means "should survive reboot". Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents of /tmp at startup (and I know Gentoo/OpenRC is). There might be an even more appropriate FHS-compliant directory (/var/run perhaps?)

Apologies, by persist I meant just beyond the end of that run of the software (i.e. the software itself couldn't manage/delete the files), to see what went "wrong". Deleting at reboot is perfectly fine.

/var/run "This directory contains system information data describing the system since it was booted" which doesn't seem to fit my use case that well. Not to say there isn't another directory that does fit, but I haven't found it.

Re: Tmpfs considered harmful

#29

Post by pezz »

w0utert wrote: post #18
raverbashing wrote: post #7

"Everyone must now be careful never to store a file in /tmp that might grow large" If you're going to trash my /tmp with gigantic files please don't. Really "it’s better to fix the filesystem to make it faster" Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place So basically, he contradicts himself…

> Mounting tmp as a memory backed fs is great (to which there may be better options than tmpfs), avoids (potentially) spinning up disks covered in rust for minor tasks. I mount /tmp as a tmpfs for almost exactly this reason. I have an always-on home server/HTPC that runs the OS off of a small (40 GB) SSD, and I don't want to wear down the SSD with writes to /tmp, /var/tmp, /var/run, /var/log, etc. In theory, I don't…

Yep - make your box do what you want.

Nothing stops you from changing fstab to make any mount tmpfs or disk.

Post reply on HN