Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tmpfs considered harmful

rwmj.wordpress.com

Post reply on HN
Print view View original

31–40 of 85 posts

Re: Tmpfs considered harmful

#31

Post by keeperofdakeys »

> "Everyone must now be careful never to store a file in /tmp that might grow large"

Be warned that when he says this, he doesn't mean it will eat all your memory, it will only grow upto a maximum size (by default half of your ram). When this is reached, it acts like any other filesystem that is full: you get write errors. There is actually another implementation called ramfs, which doesn't have such a size restriction. However, tmpfs is what is actually being implemented here. The ramfs itself is more of a 'toy' filesystem, also being quite limited, including not being able to use swap space (unlike tmpfs).

Re: Tmpfs considered harmful

#32

Post by RobAley »

raverbashing wrote: post #24
RobAley wrote: post #22

Earlier quoted context omitted.

>If you're going to trash my /tmp with gigantic files please don't. Really I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session a…

Depends on your use case and how much of that complexity you wish to manage In your case, what could work is having the new run use the same file names as the old run, or coming up with a rotation schedule (so you would keep the latest N runs) But if having them on /tmp as it is working for you, great, no need to complicate things. There are also speed considerations, if you're using these files as actual intermediat…

In my case, the runs are often run in parallel and in batches of varying sizes, so "same name" or rotation schedules wouldn't work well. They are a kind of hybrid log/storage file (they contain intermediate data sectioned/logged into the actions that created them, like an extremely verbose log). Dedicated SSDs would be very nice, but out of the price budget (i.e. its a zero cost project, run on existing scavanged hardware!).

As you say, /tmp is working well for me, my example was to perhaps illustrate that there are (what I think are) legitimate uses for large files in /tmp.

Re: Tmpfs considered harmful

#34

Post by snorkel »

Bad title. tmpfs is good. Fedora maintainers assuming that all of /tmp should be in tmpfs is bad.

Apps are already in the habit of writing a lot of expendable garbage to /tmp that doesn't require the fast I/O of tmpfs. Clogging RAM with this junk is bad idea.

Re: Tmpfs considered harmful

#36

Post by gwillen »

rwmj wrote: post #25
colanderman wrote: post #19

Earlier quoted context omitted.

That's why the GP considers mutt broken. Drafts should be stored in ~/.mutt or /var, not /tmp. Files in /tmp should not be expected to survive a process.

This is exactly the point of the posting. Every program has to be examined for this problem, and it forces a complex decision about the storage hierarchy onto every programmer.

Do you consider distros that rm -rf /tmp on startup to be broken? If not, why not?

Re: Tmpfs considered harmful

#37

Post by rwmj »

gwillen wrote: post #36
rwmj wrote: post #25

Earlier quoted context omitted.

This is exactly the point of the posting. Every program has to be examined for this problem, and it forces a complex decision about the storage hierarchy onto every programmer.

Do you consider distros that rm -rf /tmp on startup to be broken? If not, why not?

Firstly Fedora (Secondly, no I don't think it's broken. But it doesn't in any way affect the fact that as a last resort I can reboot in single user mode and grab the precious mutt file before it gets deleted.

Re: Tmpfs considered harmful

#38

Post by peterwwillis »

Example of exactly when to use hard disk and when to use tmpfs:

User browsing the web for a month without rebooting. Flash stores temporary files in /tmp/. For a month straight, Pandora is pumping out songs to /tmp/ as a cache. 43829 minutes in a month, imagine an average of 3 minutes per song that's 14609 songs, at an average 1.5 megabytes per song that's 7304 megabytes that's 7.13 gigabytes. Probably shouldn't be stored in RAM.

On the other hand, you have ganglia clusters of about 2,000 hosts. You have multiple gmetad's collecting data from different data sources and pooling it into your web interface box. It's updating ~10,000 tiny files every 30 seconds. Your i/o wait and system time is through the roof. You move the place gmetad writes to a tmpfs mount, and rsync to the disk every minute. Suddenly the i/o and system times are at 0.01%.

Tmpfs for /tmp/ is considered harmful, but extremely helpful in other cases.

Re: Tmpfs considered harmful

#39

Post by qznc »

colanderman wrote: post #19
rwmj wrote: post #11

Earlier quoted context omitted.

Sure, but if the power goes down, there's at least a chance that I may be able to find that long email I was writing. With /tmp on tmpfs, it's gone.

That's why the GP considers mutt broken. Drafts should be stored in ~/.mutt or /var, not /tmp. Files in /tmp should not be expected to survive a process.

I'd vote for ~/.cache

Re: Tmpfs considered harmful

#40

Post by ps2000 »

buster wrote: post #23

Oh, i hope they revert this change! The big trouble for me would be "Everyone must now be careful never to store a file in /tmp that might grow large". I often use /tmp for all kinds of temporary storage. Be it a small text file or a GB big tar file.

Rare use case...

I remember CD burning softwares and torrent clients asking for special temp directories to be used for that.

Allowing every day applications to create GBs of tmp files would mean you would need to worry once your partition has less than 100 GB free.

However, nobody stops you from mounting /tmp anywhere else or even disable tmpfs.

Post reply on HN