Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Copilot+ PCs

blogs.microsoft.com

Post reply on HN
Print view View original

81–90 of 614 posts

Re: Introducing Copilot+ PCs

#81

Post by coder543 »

phkahler wrote: post #77

To really be an AI PC, I'd want to be able to run something like llama 3 70B locally, and that's going to need a lot of RAM. Even running a 7B will take 14GB if it's fp16. So these really just need to be able to run a few MS AI apps. A quick googling tells me things like the Gen-AI in photoshop is actually done in the cloud, so that should still work but doesn't really require a special "AI PC".

No, 7B LLMs only need about 4GB of RAM.

There is extremely little quality loss from dropping to 4-bit for LLMs, and that “extremely little” becomes “virtually unmeasurable” loss when going to 8-bit. No one should be running these models on local devices at fp16 outside of research, since fp16 makes them half as fast as q8_0 and requires twice as much RAM for no benefit.

If a model is inadequate for a task at 4-bit, then there's virtually no chance it's going to be adequate at fp16.

Microsoft has also been doing a lot of research into smaller models with the Phi series, and I would be surprised if Phi3 (or a hypothetical Phi4) doesn’t show up at some point under the hood.

Re: Introducing Copilot+ PCs

#82

Post by DeathArrow »

andersa wrote: post #74
dingle_thunk wrote: post #39

Earlier quoted context omitted.

Microsoft has been consistently trying to give ARM a try since the surface RT. Consumers are not going to bite. marginal power saving is not meaningful.

No one cares for power saving. Turn it into higher performance at same power usage and people will bite. Of course it has to actually be a real upgrade like the Apple Silicon chips were.

>No one cares for power saving.

I bought a MacBook Pro just to have 20 hours battery life, even if I don't like macOS too much.

Re: Introducing Copilot+ PCs

#83

Post by jsheard »

nerdjon wrote: post #73
jsheard wrote: post #42

Earlier quoted context omitted.

The first iteration of Windows ARM didn't have any x86 emulation layer, so that one was doomed from the start. The second iteration did, but it initially couldn't run 64bit apps and the performance was poor. They do have 64bit support now and it sounds like the emulation performance has come a long way.

Here is my question though, comparing how this works on Mac. Will Windows have the opposite? ARM running on x86? I continue to wonder how Microsoft expects to work long term. Are they expecting that every developer is just going to keep x86 and ARM based app perpetually or users be stuck always using that emulation layer if they are running ARM? Microsoft won't be able to 100% transition to ARM like Mac did. At some…

I think the official line from Microsoft would be that most software should be using .NET anyway, and in that case the same binary should Just Work on either architecture. In reality there is still a lot of native software though, so who knows how that will play out. Games in particular will always be native.

Re: Introducing Copilot+ PCs

#84

Post by [deleted] »

binkethy wrote: post #59

I suspect that there will be a LOT of TPU e-waste in about 3-5 years. One can call me reactionary, but I see a greater benefit in providing SIMD primitives for known bottlenecks, co-processing for known inner loops, than to continually jump through the hoop of committing passing trends to hardware. I suspect thst Microsoft will implement brilliantly on a bad idea, but that it still won't gain much traction, as Co-pil…

[deleted]

Re: Introducing Copilot+ PCs

#85

Post by croes »

milkshakes wrote: post #65
croes wrote: post #36

Earlier quoted context omitted.

Getting a warrant isn't that hard either.

getting a warrant requires probable cause, and a judge to approve it. maybe that's not hard, maybe it is, but i'd personally prefer more steps being required to access what data i have.

In Germany it's the same in theory, in reality even bogus causes are approved by the judges who neither the time and knowledge to really check the inquiry.

Re: Introducing Copilot+ PCs

#87

Post by drevil-v2 »

Being the designated "tech" person for my extended family and friends circle, I don't think I could recommend this to any of them because of the privacy nightmare.

At least with Apple you have a single vendor who is vertically integrated and makes a huge song and dance about data privacy. Even if you discount their PR and marketing spin, IMHO you are still miles ahead of the likes of Microsoft + (pick one) HP, Asus, Lenovo and the rest of them.

There is no way I would trust any of them not to take advantage of the data gold mine.

Re: Introducing Copilot+ PCs

#88

Post by andersa »

DeathArrow wrote: post #82
andersa wrote: post #74

Earlier quoted context omitted.

No one cares for power saving. Turn it into higher performance at same power usage and people will bite. Of course it has to actually be a real upgrade like the Apple Silicon chips were.

>No one cares for power saving. I bought a MacBook Pro just to have 20 hours battery life, even if I don't like macOS too much.

Hm true, it does matter a great deal on laptops.

Re: Introducing Copilot+ PCs

#89

Post by bitwize »

milkshakes wrote: post #65
croes wrote: post #36

Earlier quoted context omitted.

Getting a warrant isn't that hard either.

getting a warrant requires probable cause, and a judge to approve it. maybe that's not hard, maybe it is, but i'd personally prefer more steps being required to access what data i have.

Indeed. It's the difference between already having substantive reason to believe you committed a crime, and surveilling you willy-nilly looking for a crime to pin on you. The less arbitrary access law enforcement has to your data, the more difficult the latter becomes.
Post reply on HN