Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

lowendbox.com

Post reply on HN
Print view View original

121–130 of 137 posts

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#121

Post by blueflow »

Scaevolus wrote: post #120
blueflow wrote: post #97

Earlier quoted context omitted.

First, having no swap means anonymous pages cannot be evicted, named pages must be evicted instead. Second, the binaries of your processes are mapped in as named pages (because they come from the ELF file). Named pages are generell not understood as "used" memory because they can be evicted and reclaimed, but if you have a service with a 150MB binary running, those 150MB of seemingly "free" memory are absolutely cruc…

Swapping anonymous pages can bring the system to a crawl too. High memory pressure makes things very slow with swap, while with swap off high memory pressure is likely to invoke the oom killer and lets the system violently repair.

The "bug" with the OOM killer that i implied is that what you describe does not happen. Which is not surprising because disk cache thrashing is normal mode of operation for serving big files to the network. An OOM killer acting on that alone would be problematic, but without swap, that's where the slowdown will happen for other workloads, too.

Its less a bug but an understood problem, and there aren't any good solutions around yet.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#122

Post by 1vuio0pswjnm7 »

I havent used a non-tmpfs (disk-based) /tmp in over 15 years

Didnt need it on NetBSD, memory could go to zero and system would (thrash but) not crash. When I switched to Linux the OOM issue was a shock at first but I learned to avoid it

I use small form factor computers, with userland mounted in and running from memory, no swap; I only use longterm storage for non-temporary data

https://www.kingston.com/unitedkingdom/en/blog/pc-performanc...

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#123

Post by rubatuga »

apexalpha wrote: post #101

I thought /dev/shm was ramdisk? Does /dev/shm stay? Surely it does but it is also capped at 50% RAM. Does that mean /dev/shm + /tmp can now get to 100% RAM? Or do they share the same ram budget?

Same budget

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#124

Post by justsomehnguy »

mnw21cam wrote: post #69
marginalia_nu wrote: post #56

Earlier quoted context omitted.

The Linux OOM killer is kinda sketchy to rely on. It likes to freeze up your system for long periods of time as it works out how to resolve the issue. Then it starts killing random PIDs to try to reclaim RAM like a system wide russian roulette. It's especially janky when you don't have swap. I've found adding a small swap file of ~500 MB makes it work so much better, even for systems with half a terabyte of RAM this…

Install earlyoom or one of its near-equivalents. That mostly solves the problem of it freezing up the system for long periods of time. I haven't personally seen the OOM killer kill unproductively - usually it kills either a runaway culprit or something that will actually free up enough space to help. For your "even for systems with half a terabyte of RAM", it is logical that the larger the system, the worse this beha…

> I haven't personally seen the OOM killer kill unproductively

Ah, the classical linux fan adage: "never happened to me means never happens ever to anyone".

My favourite things to see with OOM:

killing mysql on the machine which hosts only mysql and is THE production;

and the best one - killing sshd. Of course I can report on that only after seeing it on the tty0 through the BMC/IPMI console or KVM console of a VM.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#125

Post by justsomehnguy »

m463 wrote: post #31
fh973 wrote: post #23

Swap on servers somewhat defeats the purpose of ECC memory: your program state is now subject to complex IO path that is not end-to-end checksum protected. Also you get unpredictable performance. So typically: swap off on servers. Do they have a server story?

I can see it now: pro ecc sata and m.2 ssds

Well, SATA do have a basic CRC and you would see an increase in CRC transfer errors in SMART if the path (usually the cables) aren't good.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#126

Post by NekkoDroid »

Suzuran wrote: post #115
nolist_policy wrote: post #48

Earlier quoted context omitted.

Use `/var/tmp` of you want a disk backed tmp. Not sure why the article omits that.

/var/tmp is not allowed if you wish to pass security audits, it MUST be a bind mount to /tmp

What is the rational behind requiring it being a bind mount (or also whatever other options are allowed)?

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#127

Post by yjftsjthsd-h »

tatref wrote: post #77
cromka wrote: post #72

Earlier quoted context omitted.

So maybe another filesystem with heavy checksums could be used? Btrfs or dm-crypt with integrity over ext4?

Why not dm-integrity?

https://wiki.archlinux.org/title/Dm-integrity

> It uses journaling for guaranteeing write atomicity by default, which effectively halves the write speed.

That seems like a poor fit for swap IMO.

https://www.kernel.org/doc/html/latest/admin-guide/device-ma... says,

> There’s an alternate mode of operation where dm-integrity uses a bitmap instead of a journal. If a bit in the bitmap is 1, the corresponding region’s data and integrity tags are not synchronized - if the machine crashes, the unsynchronized regions will be recalculated. The bitmap mode is faster than the journal mode, because we don’t have to write the data twice, but it is also less reliable, because if data corruption happens when the machine crashes, it may not be detected.

It's not clear to me if that would be okay for swap (as long as you don't hibernate, maybe) or if it's sufficiently protected from corruption.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#128

Post by Trixter »

blueflow wrote: post #121
Scaevolus wrote: post #120

Earlier quoted context omitted.

Swapping anonymous pages can bring the system to a crawl too. High memory pressure makes things very slow with swap, while with swap off high memory pressure is likely to invoke the oom killer and lets the system violently repair.

The "bug" with the OOM killer that i implied is that what you describe does not happen . Which is not surprising because disk cache thrashing is normal mode of operation for serving big files to the network. An OOM killer acting on that alone would be problematic, but without swap, that's where the slowdown will happen for other workloads, too. Its less a bug but an understood problem, and there aren't any good solut…

earlyoom is what we use to address this. We can't tolerate any kind of swapping at all in our workloads, where it is better for the system to kill one process to save the others, than for the system to slow down or lock up.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#129

Post by tgtweak »

This is precisely what /dev/shm is for... And it can be used explicitly without any gotchas. If someone really wanted tmp to be in memory (to reduce SSD endurance writes or speed it up nominally) they can edit their mounts.

This feels like a very unnecessary change and nothing in that article made a convincing argument for the contrary.

Re: A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

#130

Post by ars »

blueflow wrote: post #118
Suzuran wrote: post #116

Earlier quoted context omitted.

The swap partition does not have a filesystem, it is a linear list of blocks.

Neither i nor parent said that. Confusion?

You wrote your comment like it was a rebuttal of the person above you, but the text supports what they said: A filesystem is faster than swap for this.

What was your intent?

Post reply on HN