Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

lowendbox.com

Post reply on HN
Print view View original

51–60 of 137 posts

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#51

Post by PhilipRoman »

greatgib wrote: post #49

I'm surprised to discover that it was not already the case for a long time for tmpfs to be used for /tmp, and that change is nice. But the auto-cleanup feature looks awful to me. Be it desktop or servers, machine with uptime of more than a year, I never saw the case of tmp being filled just by forgotten garbage. Only sometimes filled by unzipping a too big file or something like that. But it is on the spot. It used t…

I agree about the auto cleanup, I discovered it a few days after using /tmp as a ramdisk for Yocto build. Lost a few patches but nothing significant.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#52

Post by rwmj »

gigatexal wrote: post #20

Why this change? Writing to it will be faster than disk but idk if am is a precious commodity I’d rather it was just a part of the disk I was writing to.

It's a dumb idea that came from the systemd people. They've never explained properly why it's a good idea, but it's the systemd default and for some reason distros defer to that.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#53

Post by nolist_policy »

rwmj wrote: post #50

'systemctl mask tmp.mount' - the most important command to run in these situations. It's a really bad idea to put /tmp into memory. Filesystems already use memory when necessary and spill to the filesystem when memory is under pressure. If they don't do this correctly (which they do) then fix your filesystem! That will benefit everything.

You'd think that, but in ext4 the first write to a new file will hit the disk (the code mentions it is a workaround for something). Btrfs does it correctly.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#54

Post by loa_in_ »

greatgib wrote: post #49

I'm surprised to discover that it was not already the case for a long time for tmpfs to be used for /tmp, and that change is nice. But the auto-cleanup feature looks awful to me. Be it desktop or servers, machine with uptime of more than a year, I never saw the case of tmp being filled just by forgotten garbage. Only sometimes filled by unzipping a too big file or something like that. But it is on the spot. It used t…

I tried out variations on this on my daily driver setups. The design choices here were likely threefold:

Store tmpfs in memory: volatile but limited to free ram or swap, and that writes to disk

Store tmpfs on dedicated volume: Since we're going to write onto disk anyway, make it a lightweight special purpose file system that's commited to disk

On disk tmpfs but cleaned up periodically: additional settings to clean up - how often, what should stay, tie file lifetime to machine reboot? The answers to these questions vary more between applications than between filesystems, therefore it's more flexible to leave clean up to userspace.

In the end my main concern turned out to be that I lost files that I didn't want to lose, either to reboot cleanup, on timer cleanup, etc. I opted to clean up my temp files manually as needed.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#55

Post by hnlmorg »

greatgib wrote: post #49

I'm surprised to discover that it was not already the case for a long time for tmpfs to be used for /tmp, and that change is nice. But the auto-cleanup feature looks awful to me. Be it desktop or servers, machine with uptime of more than a year, I never saw the case of tmp being filled just by forgotten garbage. Only sometimes filled by unzipping a too big file or something like that. But it is on the spot. It used t…

If you’ve got swap set up then stale files will get written back to disk so at least you’re not RAM indefinitely just because they’re stored on tmpfs.

It’s still not an ideal solution though.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#56

Post by marginalia_nu »

edoceo wrote: post #33
m463 wrote: post #30

Earlier quoted context omitted.

what swap partition? I meant this sort of jokingly. I think have a few linux systems that were never configured with swap partitions or swapfiles.

I'm with you. I don't swap. Processes die. OOM. Linux can recover and not lose data. Just unavailable for a moment.

The Linux OOM killer is kinda sketchy to rely on. It likes to freeze up your system for long periods of time as it works out how to resolve the issue. Then it starts killing random PIDs to try to reclaim RAM like a system wide russian roulette.

It's especially janky when you don't have swap. I've found adding a small swap file of ~500 MB makes it work so much better, even for systems with half a terabyte of RAM this helps reduce the freezing issues.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#57

Post by rwmj »

nolist_policy wrote: post #53
rwmj wrote: post #50

'systemctl mask tmp.mount' - the most important command to run in these situations. It's a really bad idea to put /tmp into memory. Filesystems already use memory when necessary and spill to the filesystem when memory is under pressure. If they don't do this correctly (which they do) then fix your filesystem! That will benefit everything.

You'd think that, but in ext4 the first write to a new file will hit the disk (the code mentions it is a workaround for something). Btrfs does it correctly.

Sounds like fixing that ext4 problem could make a lot of things go faster.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#58

Post by marginalia_nu »

ars wrote: post #8

File is tmpfs will swap out if your system is under memory pressure. If that happens, reading the file back is DRAMATICALLY slower than if you had just stored the file on disk in the first place. This change is not going to speed things up for most users, it will slow things. Instead of caching important files, you waste memory on useless temporary files. Then the system swaps it out, so you can get cache back, and t…

This doesn't really make sense. If /tmp was an on-disk directory the same memory pressure that caused swapping would just evict the file from the page cache instead, again leading to a cache miss and a dramatically slower read.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#59

Post by mrweasel »

greatgib wrote: post #49

I'm surprised to discover that it was not already the case for a long time for tmpfs to be used for /tmp, and that change is nice. But the auto-cleanup feature looks awful to me. Be it desktop or servers, machine with uptime of more than a year, I never saw the case of tmp being filled just by forgotten garbage. Only sometimes filled by unzipping a too big file or something like that. But it is on the spot. It used t…

> I never saw the case of tmp being filled just by forgotten garbage.

It might have more to do with the type of developers I've worked with, but it happens all the time. Monitoring complains and you go into check, and there it is gigabytes of junk dumped there by shitty software or scripts that can't cleanup after themselves.

The issue is that you don't always knows what's safe to delete, if you're the operations person, and not the developer. Periodically auto-cleaning /tmp is going to do break stuff, and it will be easier to demand that the operations team disable auto-cleanup than getting the issue fixed in the developers next sprint.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#60

Post by wahern »

marginalia_nu wrote: post #56
edoceo wrote: post #33

Earlier quoted context omitted.

I'm with you. I don't swap. Processes die. OOM. Linux can recover and not lose data. Just unavailable for a moment.

The Linux OOM killer is kinda sketchy to rely on. It likes to freeze up your system for long periods of time as it works out how to resolve the issue. Then it starts killing random PIDs to try to reclaim RAM like a system wide russian roulette. It's especially janky when you don't have swap. I've found adding a small swap file of ~500 MB makes it work so much better, even for systems with half a terabyte of RAM this…

Yeah. I always disable overcommit (notwithstanding that Linux cannot provide perfectly accurate strict memory accounting), and I'd prefer not to use swap, but Linux VM maintainers have consistently stated that they've designed and tuned the VM subsystem with swap in mind. Is swap necessary in the abstract? No. Is swap necessary on Linux? No. But don't be surprised if Linux doesn't do what you'd expect in the absence of swap, and don't expect Linux to put much if any effort into improving performance in the absence of swap.

I've never ran into trouble on my personal servers, but I've worked at places that have, especially when running applications that tax the VM subsystem, e.g. the JVM and big Java apps. If one wonders why swap would be useful even if applications never allocate, even in the aggregate, more anonymous memory than system RAM, one of the reasons is the interaction with the buffer cache and eviction under pressure.

Post reply on HN