Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local AI is driving the biggest change in laptops in decades

spectrum.ieee.org

Post reply on HN
Print view View original

121–130 of 273 posts

Re: Local AI is driving the biggest change in laptops in decades

#121

Post by ignoramous »

whazor wrote: post #115
jwr wrote: post #96

The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.

But economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI. However, I agree with the article that people will run big LLMs on their laptop N years down the line. Especially if hardware outgrows best-in-class LLM model requirements. If a phone could run a 512GB LLM model fast, you would want it.

> economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI

Uber is economical, too; but folks prefer to own cars, sometimes multiple.

And how there's market for all kinds of vanity cars, fast sportscars, expensive supercars... I imagine PCs & Laptops will have such a market, too: In probably less than a decade, may be a £20k laptop running a 671b+ LLM locally will be the norm among pros.

Re: Local AI is driving the biggest change in laptops in decades

#122

Post by benbristow »

grayhatter wrote: post #120
benbristow wrote: post #110

Earlier quoted context omitted.

I don't really PC game anymore, use my Xbox or a few older games my laptop's iGPU can handle, not at the moment anyway. Battlefield 6 is a big one recently that if I had a gaming PC set-up I'd probably want to play. I know Adobe are... c-words, but their software is industry standard for a reason.

> Battlefield 6 is a big one recently that if I had a gaming PC set-up I'd probably want to play. We definitely play very different games, I wouldn't touch it if you paid me. So I'm sure we both have a bit of sample bias in our expected rates of linux compatibility. Especially since EA is another company like Adobe. Also, the internet seems to think they have a cheating problem. I wonder how bad it really is, and if…

I think older versions do, like CS6 through WINE.

Photoshop CC 2024 apparently works somewhat, but no GPU support and the removal tool doesn't work apparently.

https://appdb.winehq.org/objectManager.php?sClass=version&iI...

Basically, no.

Re: Local AI is driving the biggest change in laptops in decades

#123

Post by cultofmetatron »

benbristow wrote: post #100
sandworm101 wrote: post #23

Earlier quoted context omitted.

>> I'd really rather my devices only do what I ask them to Linux hears your cry. You have a choice. Make it.

Unfortunately still loads of hurdles for most people. AAA Games with anti-cheat that don't support Linux. Video editing (DaVinci Resolve exists but is a pain to get up and running on many distros, KDenLive/OpenShot don't really cut it for most) Adobe Suite (Photoshop/Lightroom specifically, and Premiere for Video Editing) - would like to see Affinity support Linux but hasn't happened so far. GIMP and DarkTable aren't…

> AAA Games with anti-cheat that don't support Linux.

I really don't understand people that want to play games so badly that they are willing to install a literal rootkit on their devices. I can understand if you're a pro gamer but it feels stupid to do it otherwise.

Re: Local AI is driving the biggest change in laptops in decades

#124

Post by benbristow »

cultofmetatron wrote: post #123
benbristow wrote: post #100

Earlier quoted context omitted.

Unfortunately still loads of hurdles for most people. AAA Games with anti-cheat that don't support Linux. Video editing (DaVinci Resolve exists but is a pain to get up and running on many distros, KDenLive/OpenShot don't really cut it for most) Adobe Suite (Photoshop/Lightroom specifically, and Premiere for Video Editing) - would like to see Affinity support Linux but hasn't happened so far. GIMP and DarkTable aren't…

> AAA Games with anti-cheat that don't support Linux. I really don't understand people that want to play games so badly that they are willing to install a literal rootkit on their devices. I can understand if you're a pro gamer but it feels stupid to do it otherwise.

Most of the time they're not really informed that they are. I know Valorant does (Riot Games), one I've avoided in the past because of it.

But a lot of the time it's peer-pressure for wanting to play with friends who couldn't care less.

Re: Local AI is driving the biggest change in laptops in decades

#125

Post by layer8 »

Morromist wrote: post #9

I was in the market for a laptop this month. Many new laptops now advertise AI features like this "HP OmniBook 5 Next Gen AI PC" which advertises: "SNAPDRAGON X PLUS PROCESSOR - Achieve more everyday with responsive performance for seamless multitasking with AI tools that enhance productivity and connectivity while providing long battery life" I don't want this garbage on my laptop, especially when its running of its…

It’s true that the AI marketing is largely nonsense, but the NPUs also don’t hurt, and you don’t have to make use of them.

Re: Local AI is driving the biggest change in laptops in decades

#126

Post by andai »

dangus wrote: post #114
fancyfredbot wrote: post #111

Earlier quoted context omitted.

I think the author is aware of Apple silicon. The article mentions the fact Apple has unified memory and that this is advantageous for running LLMs.

Then idk why they say that most laptops are bad at running LLMs, Apple has a huge marketshare in the laptop market and even their cheapest laptops are capable in that realm. And their PC competitors are more likely to be generously specced out in terms of included memory. > However, for the average laptop that’s over a year old, the number of useful AI models you can run locally on your PC is close to zero. This stra…

So I'm hearing a lot of people running LLMs on Apple hardware. But is there actually anything useful you can run? Does it run at a usable speed? And is it worth the cost? Because the last time I checked the answer to all three questions appeared to be no.

Though maybe it depends on what you're doing? (Although if you're doing something simple like embeddings, then you don't need the Apple hardware in the first place.)

Re: Local AI is driving the biggest change in laptops in decades

#127

Post by swyx »

seunosewa wrote: post #65

"How many TOPS do you need to run state-of-the-art models with hundreds of millions of parameters? No one knows exactly." What's he talking about? It's trivial to calculate that.

> state-of-the-art models

> hundreds of millions of parameters

lol

lmao, even

Re: Local AI is driving the biggest change in laptops in decades

#128

Post by almosthere »

azuanrb wrote: post #119
jwr wrote: post #96

The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.

You still need ridiculously high spec hardware, and at Apple’s prices, that isn’t cheap. Even if you can afford it (most won't), the local models you can run are still limited and they still underperform. It’s much cheaper to pay for a cloud solution and get significantly better result. In my opinion, the article is right. We need a better way to run LLMs locally.

749 for an M4 air at Amazon right now

Re: Local AI is driving the biggest change in laptops in decades

#129

Post by layer8 »

wkat4242 wrote: post #5

This article is so dumb. It totally ignores the memory price explosion that will make large fast memory laptops unfeasible for years and states stuff like this: > How many TOPS do you need to run state-of-the-art models with hundreds of millions of parameters? No one knows exactly. It’s not possible to run these models on today’s consumer hardware, so real-world tests just can’t be done. We know exactly the performan…

The article is from mid-November (and probably was written even earlier), where the RAM price explosion wasn’t as striking yet.

Re: Local AI is driving the biggest change in laptops in decades

#130

Post by fancyfredbot »

dangus wrote: post #114
fancyfredbot wrote: post #111

Earlier quoted context omitted.

I think the author is aware of Apple silicon. The article mentions the fact Apple has unified memory and that this is advantageous for running LLMs.

Then idk why they say that most laptops are bad at running LLMs, Apple has a huge marketshare in the laptop market and even their cheapest laptops are capable in that realm. And their PC competitors are more likely to be generously specced out in terms of included memory. > However, for the average laptop that’s over a year old, the number of useful AI models you can run locally on your PC is close to zero. This stra…

Most laptops have 16GB of RAM or less. A little more than a year ago I think the base model Mac laptop had 8GB of RAM which really isn't fantastic for running LLMs.
Post reply on HN