Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux 7.3 improves performance when running out of vRAM

pixelcluster.dev

Post reply on HN
Print view View original

141–150 of 328 posts

Re: Linux 7.3 improves performance when running out of vRAM

#141

Post by marky1991 »

TiredOfLife wrote: post #63
d3Xt3r wrote: post #11

Gosh, 7.2 literally just dropped with a bunch of awesome of performance/gaming related improvements (large folios, cache-aware scheduling, improved MGLRU reclaiming, Fair GPU Scheduler etc)... and I already can't wait for 7.3 to come out. Meanwhile in the Windows world, users hate updates... Like I genuinely can't think of a single instance that made users exclaim, "oh boy I just can't wait for the next Patch Tuesday…

I use arch based linux. This year updates have broken video decoding twice. Font rendering once (still not fixed without manually deleting files.). And broken the whole system due to there not being a mechanism that makes sure all QT libraries upate at the same time once or twice. That is all one system. Meanwhile 4 separate windows pcs have not had problems with updates for at least 6 years

How would you detect broken video decoding or font rendering? I don't know what those are. I don't think i've had any issues this year (Maybe i had do do some pacman-wrangling once this year, but i'm slow to update, so that was probably my fault...)), that I noticed anyway, but maybe everything's broken and i just can't tell.

Re: Linux 7.3 improves performance when running out of vRAM

#142

Post by bee_rider »

sunaookami wrote: post #101
mrec wrote: post #85

Earlier quoted context omitted.

The best thing about Win10 reaching EOL is not being at the mercy of Windows Update any more. It was by far the biggest risk to having a functioning system.

Running an unsupported OS is even more risky, especially with the excessive number of security updates needed while AI finds RCE after RCE.

MS should either fix the RCEs in Windows 10 (it is unsupported but the defects were already there when it was still supported, just undiscovered), or they should refund their customers (since they never managed to release a defect-free version).

Re: Linux 7.3 improves performance when running out of vRAM

#143

Post by inventor7777 »

I see all the comments about how Windows and Linux handle OOM, but I suppose I'll also mention macOS.

On my M4 Max Mac Studio if I try to load too big of an AI model with protections off, the desktop starts glitching back and forth between the past few hundred frames. It looks bad when it happens but CtrlC still works to kill llama.cpp and if you were using LM Studio, SSH also works. Once you kill the offending process, or if macOS does it for you, the desktop comes back immediately.

Re: Linux 7.3 improves performance when running out of vRAM

#144

Post by aruggirello »

embedding-shape wrote: post #89
mrec wrote: post #85

Earlier quoted context omitted.

The best thing about Win10 reaching EOL is not being at the mercy of Windows Update any more. It was by far the biggest risk to having a functioning system.

Windows is such a horrible OS to use when you work in a small company that isn't big enough to have IT to shield you from all the horrible stuff, like in a small studio. Normally I work in Linux, sometimes need to boot Windows just to do some export or something, then goal is to boot into Linux again ASAP. But time and time again fucking Windows forces me to prepping updates when shutting down for next boot, with lit…

This sounds like the perfect use case for a simple, straightforward VM. Dual booting is dangerous, time consuming, and makes it inconvenient to share clipboard, stuff... why don't you use a Windows VM instead? You can freeze a VM (saving state) rather than restarting it. Windows won't bat an eye since it's not being shut down or suspended. No longer unexpected 15 minutes pauses - restart it when you have time to spare, and keep working in your host Linux OS while the Windows VM does its shenanigans.

Re: Linux 7.3 improves performance when running out of vRAM

#145

Post by leni536 »

krisknez wrote: post #34

I hope there will be an update where when my RAM gets full my PC doesn't freeze and becomes unusable... I remember that Linux and Windows do this in different ways and Windows doesn't have the problem.

My experience is that if you have swap you better have a lot of it. Swapping is bad, but swapping and thrashing is a disaster.

Re: Linux 7.3 improves performance when running out of vRAM

#146

Post by LoganDark »

inventor7777 wrote: post #143

I see all the comments about how Windows and Linux handle OOM, but I suppose I'll also mention macOS. On my M4 Max Mac Studio if I try to load too big of an AI model with protections off, the desktop starts glitching back and forth between the past few hundred frames. It looks bad when it happens but CtrlC still works to kill llama.cpp and if you were using LM Studio, SSH also works. Once you kill the offending proce…

On my M4 Max MacBook Pro, if I run or even load an LLM more than a couple times, even a small one, general desktop performance starts to get really, really bad until a reboot and I don't know why.

Genuinely, everything starts feeling super sluggish, even long after the process has terminated -- cmd+tab gets slow, apps constantly hang for a few seconds at a time, the cursor position starts lagging and videos are slow to pause/play -- and the only way I've found to fix it is to reboot the entire machine. Clearing memory, restarting apps etc. doesn't work.

Don't get me wrong, the performance and QoS system works really well while the model is still in memory, but after a few cycles of loading and unloading, performance just ends up getting bad in general. Doesn't matter if I'm loading it with llama.cpp, MLX, native Metal, if I start using the GPU for machine learning the machine just gets really upset.

This sucks man... macOS is not supposed to need reboots like that, my Intel Mac could run for months on end without slowing down at all and I could push it as hard as I wanted. As far as I can tell, this behavior is new as of either macOS 26 or the 27 beta. Apple really has been betraying their true Unix roots lately.

Re: Linux 7.3 improves performance when running out of vRAM

#147

Post by aruggirello »

toredash wrote: post #12
d3Xt3r wrote: post #11

Gosh, 7.2 literally just dropped with a bunch of awesome of performance/gaming related improvements (large folios, cache-aware scheduling, improved MGLRU reclaiming, Fair GPU Scheduler etc)... and I already can't wait for 7.3 to come out. Meanwhile in the Windows world, users hate updates... Like I genuinely can't think of a single instance that made users exclaim, "oh boy I just can't wait for the next Patch Tuesday…

Didn't we eagerly await on updates in the good old days of Windows?

We did, and there was some cool stuff coming out of MS now and then. I remember when the first Power Toys came out - there's nothing really magic in current ones though.

Re: Linux 7.3 improves performance when running out of vRAM

#148

Post by adrian_b »

krisknez wrote: post #34

I hope there will be an update where when my RAM gets full my PC doesn't freeze and becomes unusable... I remember that Linux and Windows do this in different ways and Windows doesn't have the problem.

I stopped using swap almost a quarter of century ago, and since then I have not seen a PC with Linux freezing because of full RAM.

Before the recent increase in RAM prices, it was not difficult to have a generous amount of RAM in a PC, that would make very unlikely the out-of-memory situations.

When OOM really happened, it was much better to become aware of this immediately, and reduce the number of concurrent threads for a running job, or whatever caused that, instead of having reduced performance or even freezing.

Re: Linux 7.3 improves performance when running out of vRAM

#149

Post by radlad »

LoganDark wrote: post #146
inventor7777 wrote: post #143

I see all the comments about how Windows and Linux handle OOM, but I suppose I'll also mention macOS. On my M4 Max Mac Studio if I try to load too big of an AI model with protections off, the desktop starts glitching back and forth between the past few hundred frames. It looks bad when it happens but CtrlC still works to kill llama.cpp and if you were using LM Studio, SSH also works. Once you kill the offending proce…

On my M4 Max MacBook Pro, if I run or even load an LLM more than a couple times, even a small one, general desktop performance starts to get really, really bad until a reboot and I don't know why. Genuinely, everything starts feeling super sluggish, even long after the process has terminated -- cmd+tab gets slow, apps constantly hang for a few seconds at a time, the cursor position starts lagging and videos are slow…

Oh wow, you might've just explained some symptoms I've run into a few times. Thanks for your comment.

Re: Linux 7.3 improves performance when running out of vRAM

#150

Post by jeffbee »

adrian_b wrote: post #148
krisknez wrote: post #34

I hope there will be an update where when my RAM gets full my PC doesn't freeze and becomes unusable... I remember that Linux and Windows do this in different ways and Windows doesn't have the problem.

I stopped using swap almost a quarter of century ago, and since then I have not seen a PC with Linux freezing because of full RAM. Before the recent increase in RAM prices, it was not difficult to have a generous amount of RAM in a PC, that would make very unlikely the out-of-memory situations. When OOM really happened, it was much better to become aware of this immediately, and reduce the number of concurrent thread…

Unfortunately Linux makes it difficult to fully get away from swapping because any file-backed page is eligible to be evicted, and even if you have "disabled swap" the kernel will cheerfully page-out your program, even if it needs to be immediately paged in again to continue. The only way to stop it from doing so is to remap all executable mappings into anonymous memory, a thing which is possible but which the loader and linker do not make easy.
Post reply on HN