Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I helped fix sleep-wake hangs on Linux with AMD GPUs

nyanpasu64.gitlab.io

Post reply on HN
Print view View original

51–60 of 218 posts

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#51

Post by bachmeier »

xondono wrote: post #36
Apofis wrote: post #23

Earlier quoted context omitted.

Remarkable that it's 2025 and laptop sleep/suspend still doesn't work right on linux. I think the first time I encountered this was probably 15 years ago now?

Sleep & suspend doesn’t work on Windows either. Power control is the kind of stuff that benefits from very tight integration, and PCs just don’t have that. Firmware is seen by most vendors as a pure cost to minimize, so you get a fragmented market full of subcontractors delivering the bare minimum that is considered “working”. Manufacturers also know most people aren’t going to use a big part of the functions they’re…

> Sleep & suspend doesn’t work on Windows either.

What's strange is that it never used to be a problem. There are five Windows laptops floating around our house at various times (mixture of work and personal) and suspend works properly on none of them. Oddly, it works on my personal laptop with Debian Stable almost every time, failing maybe 1/25 times. Other distros are about the same as Windows.

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#52

Post by meowkit »

whatever1 wrote: post #28

Apple became a trillion dollar company by mastering sleeping / waking up of electronic devices. why nobody else sees this?

Because they own the now hardware, and MacOS iOS run exclusively on their hardware

Source: I work on windows power management and I know system engineers at apple.

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#53

Post by nyanpasu64 »

sidkshatriya wrote: post #25

TL;DR: During suspend, for graphics cards, GPU VRAM needs to be transferred to system RAM. However, during high memory usage scenarios the VRAM + RAM usage could exceed system memory -- this would ordinarily involve system swap coming into play and handling the temporarily out of memory issue. However system swap was already deactivated when it came time to suspending the AMD card causing all sorts of problems. The f…

Technically it was the suspend notifier; even suspend prepare executes after swap is disabled. See replies.

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#54

Post by jorvi »

VMG wrote: post #17
jorvi wrote: post #3

> Through some digging, I found that when a desktop enters S3 sleep, the system cuts power to PCIe GPUs I am not sure how correct this assumption is. S3 is supposed to cut power to everything but RAM, but for example Gigabyte Aorus motherboards are notorious for an NVMe SSD sleep bug that randomly prevents the system from properly sleeping or waking. This is fixed by adding the following udev rule: # Generic PCIe fix…

As somebody with an Aorus motherboard who has probably burned a few kWh on this issue, I was really excited to try these solutions - no luck. Thank you anyway!

Did you try the general fix? And reload udev rules?

You also have to make sure it applies after the default rules.

You can check if the rule applies once you have everything set up by doing an `udevadm` attribute walk of your SSD device (not partition), and then following it up all the way up the device tree until you see your specific device port (target fix) or PCIe driver subsystem (general fix). Then check if "power/wakeup" is set to "disabled". If it is set to disabled, something else is keeping your device awake on sleep.

For that you can check /proc/acpi/wakeup, and there's also a specific systemd invocation (that I forgot) you can do that shows if your device slept, how long it slept, how much battery was drained, and if your device woke-up, slept or failed to resume, it'll give you a reason.. to the best of its ability.

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#55

Post by _cenw »

Some AMD integrated GPUs are surprisingly fragile with this. I have a GPD Win Max 2 8840U (a "concept car" handheld laptop hybrid) and when I got it last year, it would fail to wake from suspend and hibernate about half the time in Windows, with Linux actually being more reliable (but also not perfect), and only this year did an AMD GPU driver fix this.

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#56

Post by jchw »

Memory management and specifically OOM conditions remain an unbelievably painful nightmare on Linux. It's not like I run into these issues constantly, but I've definitely tried to debug issues like these (unsuccessfully). Ultimately if I OOM a machine I usually wind up installing more RAM, which is wasteful/expensive, but it's pretty clear that handling OOM conditions gracefully is going to be a hard problem for Linux to solve into the future.

This is really great work and will serve as a reference point for debugging similar issues in the future. Pretty happy about systemd's debug-shell feature, I had no idea that existed. I don't think my X670E Steel Legend board has a serial header anywhere on it, though. How do modern built-in serial ports work, anyway? Are they attached off of the chipset PCIe lanes?

Something that's also very useful when trying to dive into the Linux kernel is that there's a bunch of great talks discussing Linux kernel subsystems from conferences like FOSDEM and Linux Plumber's Conference which you can usually find recordings of online. For example, there's this one for TTM, the memory subsystem that most of the desktop GPU DRM drivers use:

https://www.youtube.com/watch?v=MG7_tUNKSt0

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#57

Post by _cenw »

bachmeier wrote: post #51
xondono wrote: post #36

Earlier quoted context omitted.

Sleep & suspend doesn’t work on Windows either. Power control is the kind of stuff that benefits from very tight integration, and PCs just don’t have that. Firmware is seen by most vendors as a pure cost to minimize, so you get a fragmented market full of subcontractors delivering the bare minimum that is considered “working”. Manufacturers also know most people aren’t going to use a big part of the functions they’re…

> Sleep & suspend doesn’t work on Windows either. What's strange is that it never used to be a problem. There are five Windows laptops floating around our house at various times (mixture of work and personal) and suspend works properly on none of them. Oddly, it works on my personal laptop with Debian Stable almost every time, failing maybe 1/25 times. Other distros are about the same as Windows.

Modern Standby. Windows wanted to do the Apple "power nap" stuff, but never realized how painful it'd be if you don't control all the hardware and have millions of different hardware permutations (with a lot of terrible drivers) instead of just a few. Not that it would've helped, half the time my machine is either overheating or off it seems to be wake timers doing windows updates (which yes, you can disable, but most wouldn't).

I don't get why S3 sleep had to die for this, but it did.

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#58

Post by nyanpasu64 »

yellow_lead wrote: post #7

I have an Nvidia GPU and a sporadic crash (black screen) with no logs on Linux. I suspect it's a driver issue too. Going to try some of these tips to enable the debug shell, but I'm not sure if they'll be effective. Anyone have other tips for this type of thing? I did try upgrading drivers/kernels already

Does `sudo systemctl enable nvidia-suspend` help?

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#59

Post by nyanpasu64 »

empiricus wrote: post #8

I notice I am confused how the code needed for the GPU to sleep was implemented. It was failing when simply saving/copying gigabytes of flat memory, but on the other hand it was able to recover successfully the previous complex hw and sw state and data structures?! I guess it probably makes sense if after waking up that data is actually dropped and the gpu and ui is reinitialized and redrawn.

As I understand if the GPU fails to save VRAM it drops the RAM copy(?) before restoring the GPU data structures or abandoning sleep entirely. But if it saves VRAM then crashes after the GPU is suspended, it will often fail to wake up the GPU, resulting in no monitor signal. Though I'm fuzzy on the details (and the order of operations depends on which kernel you're running).

Re: I helped fix sleep-wake hangs on Linux with AMD GPUs

#60

Post by Skunkleton »

jchw wrote: post #56

Memory management and specifically OOM conditions remain an unbelievably painful nightmare on Linux. It's not like I run into these issues constantly, but I've definitely tried to debug issues like these (unsuccessfully). Ultimately if I OOM a machine I usually wind up installing more RAM, which is wasteful/expensive, but it's pretty clear that handling OOM conditions gracefully is going to be a hard problem for Linu…

I’ve had good luck containing ooms with cgroups. I’m not sure if there is a state of the art for handling oom conditions beyond what Linux does. If anyone knows and can recommend some reading I would appreciate it.
Post reply on HN