Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debunking Zswap and Zram Myths

chrisdown.name

Post reply on HN
Print view View original

61–70 of 70 posts

Re: Debunking Zswap and Zram Myths

#62

Post by BoredomIsFun »

messe wrote: post #60
MaxCompression wrote: post #26

One underappreciated aspect of zswap vs zram is the compression algorithm choice and its interaction with the data being compressed. LZ4 (default in both) is optimized for speed at the expense of ratio — typically 2-2.5x on memory pages. zstd can push that to 3-3.5x but at significantly higher CPU cost per page fault. The interesting tradeoff: memory pages are fundamentally different from files. They contain lots of…

LLM written comments are not permitted on this site.

1. What makes you think it is written by an LLM

2. Where is that rule, could you cite it?

3. How dow I know you did not use LLM for your comment?

Re: Debunking Zswap and Zram Myths

#63

Post by networked »

bilegeek wrote: post #32
prussian wrote: post #15

With zram, I can just use zram-generator[0] and it does everything for me and I don't even need to set anything up, other than installing the systemd generator, which on some distros, it's installed by default. Is there anything equivalent for zswap? Otherwise, I'm not surprised most people are just using zram, even if sub-optimal. [0]: https://crates.io/crates/zram-generator

Kernel arguments are the primary method: https://wiki.archlinux.org/title/Zswap#Using_kernel_boot_par... Snag: I had issues getting it to use zstd at boot. Not sure if it's a bug or some peculiarity with Debian. Ended up compiling my own kernel for other reasons, and was finally able to get zstd by default, but otherwise I'd have to make/add it to a startup script.

I had the same issue with LZ4. I found a thread about it on the Linux Mint Debian Edition forum and posted my fix there: https://forums.linuxmint.com/viewtopic.php?p=2767087#p276708....

In short: add the kernel modules and update GRUB as usual, then install sysfsutils and add the following line at the end of `/etc/sysfs.conf`:

  module/zswap/parameters/compressor = lz4
  # For zstd:
  #module/zswap/parameters/compressor = zstd
Perhaps some kernel change between Linux 6.8 and 6.12 caused the old approach to no longer work.

Re: Debunking Zswap and Zram Myths

#64

Post by [deleted] »

patrakov wrote: post #9
cdown wrote: post #5

Earlier quoted context omitted.

Thank you for reading and your critique! What you're describing is definitely a real problem, but I'd challenge slightly and suggest the outcome is usually the inverse of what you might expect. One of the counterintuitive things here is that _having_ disk swap can actually _decrease_ disk I/O. In fact this is so important to us on some storage tiers that it is essential to how we operate. Now, that sounds like patent…

1. Thanks for partially (in paragraph 4 but not paragraph 5) preempting the obvious objection. Distinguishing between disk reads and writes is very important for consumer SSDs, and you quoted exactly the right metric in paragraph 4: reduction of writes, almost regardless of the total I/O. Reads without writes are tolerable. Writes stall everything badly. 2. The comparison in paragraph 4 is between no-swap and zswap,…

[deleted]

Re: Debunking Zswap and Zram Myths

#65

Post by suncore »

zozbot234 wrote: post #29
patrakov wrote: post #3

User here, who also acts as a Level 2 support for storage. The article contains some solid logic plus an assumption that I disagree with. Solid logic: you should prefer zswap if you have a device that can be used for swap . Solid logic: zram + other swap = bad due to LRU inversion (zram becomes a dead weight in memory). Advice that matches my observations: zram works best when paired with a user-space OOM killer. Bol…

> Many consumer SSDs ... under synchronous writes, will regularly produce latency spikes of 10 seconds or more Surely "regularly" is a significant overstatement. Most people have practically never seen this failure mode. And if it only occurs under a heavy write workload, that's not something that's supposed to happen purely as a result of swapping.

Very easy to reproduce: 1. Buy cheap QLC drive. 2. Fill with Steam games. 3. Delete some steam games and download new games. 4. Watch write speeds tank to zero for long periods when downloading.

It's due to garbage collecting on very slow QLC NAND. You won't see it until the drive starts to get 60%+ full. Until then, the drive pretends it is an SLC with very fast writes, but then it starts to show its true colors. Yuck.

Re: Debunking Zswap and Zram Myths

#66

Post by suncore »

patrakov wrote: post #3

User here, who also acts as a Level 2 support for storage. The article contains some solid logic plus an assumption that I disagree with. Solid logic: you should prefer zswap if you have a device that can be used for swap . Solid logic: zram + other swap = bad due to LRU inversion (zram becomes a dead weight in memory). Advice that matches my observations: zram works best when paired with a user-space OOM killer. Bol…

Cheap QLC drives become super slow when it is starting to get fairly full and starts garbage collecting (collecting SLC writes into QLC at maybe 10MB/s). IMHO this is not good enough for an OS drive.

Re: Debunking Zswap and Zram Myths

#68

Post by adrian_b »

Numerlor wrote: post #56
patrakov wrote: post #3

User here, who also acts as a Level 2 support for storage. The article contains some solid logic plus an assumption that I disagree with. Solid logic: you should prefer zswap if you have a device that can be used for swap . Solid logic: zram + other swap = bad due to LRU inversion (zram becomes a dead weight in memory). Advice that matches my observations: zram works best when paired with a user-space OOM killer. Bol…

> The assumption is simply false, and not due to the "SSD wear" argument. Many consumer SSDs, especially DRAMless ones (e.g., Apacer AS350 1TB, but also seen on Crucial SSDs), under synchronous writes, will regularly produce latency spikes of 10 seconds or more, due to the way they need to manage their cells. This is much worse than any HDD. If a DRAMless consumer SSD is all that you have, better use zram. Do mind th…

The behavior that you see depends a lot on your workload.

I frequently write big multi-gigabyte files and this overflows any kind of buffers, so I often see pauses of many seconds for garbage collection on Samsung Pro NVMe SSDs.

Someone who only writes small files is unlikely to see such pauses, but when writing big amounts of data, pauses are guaranteed on any SSD.

Re: Debunking Zswap and Zram Myths

#69

Post by messe »

BoredomIsFun wrote: post #62
messe wrote: post #60

Earlier quoted context omitted.

LLM written comments are not permitted on this site.

1. What makes you think it is written by an LLM 2. Where is that rule, could you cite it? 3. How dow I know you did not use LLM for your comment?

1. Word choice, phrasing, and sentence structure make it seem likely. Ironically, one has to go on vibes. One gets a feel for the voice and tone used by LLMs after a while. It's also a new account with one comment.

2. "Don't post generated comments or AI-edited comments. HN is for conversation between humans." From https://news.ycombinator.com/newsguidelines.html

3. You don't.

Re: Debunking Zswap and Zram Myths

#70

Post by BoredomIsFun »

messe wrote: post #69
BoredomIsFun wrote: post #62

Earlier quoted context omitted.

1. What makes you think it is written by an LLM 2. Where is that rule, could you cite it? 3. How dow I know you did not use LLM for your comment?

1. Word choice, phrasing, and sentence structure make it seem likely. Ironically, one has to go on vibes. One gets a feel for the voice and tone used by LLMs after a while. It's also a new account with one comment. 2. "Don't post generated comments or AI-edited comments. HN is for conversation between humans." From https://news.ycombinator.com/newsguidelines.html 3. You don't.

1 and 3 contradict each other. Last thing people need is anti-AI hysteria.
Post reply on HN