Live data from Hacker News

Zero-Copy GPU Inference from WebAssembly on Apple Silicon

abacusnoir.com

21–30 of 59 posts

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#21
post #3

> on Apple Silicon, a WebAssembly module's linear memory can be shared directly with the GPU: no copies, no serialization, no intermediate buffers enhance > no copies, no serialization, no intermediate buffers would it kill people to write their own stuff why are we doing this. out of all the things people immediately cede to AI they cede their human ability to communicate and convey/share ideas. this timeline is bon…

I don't know, to me your sentiment sounds a lot like how back in the day they used to say "you can't just use a calculator all the time, use your brain and show the work on pen and paper". humans have been using tools to communicate since pre-history. language itself is one tool of communication invented to supersede body-language and grunting and noises. the thought and idea is theirs, it was communicated. Would it…

[dead]

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#22
post #8

Earlier quoted context omitted.

I think the days of on-site interviews with whiteboard tests may be drawing to a close faster than you suspect

I also think we will never go back to good old days.

It'll put the "everything old becomes new again" idea to the test.

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#24
post #3

> on Apple Silicon, a WebAssembly module's linear memory can be shared directly with the GPU: no copies, no serialization, no intermediate buffers enhance > no copies, no serialization, no intermediate buffers would it kill people to write their own stuff why are we doing this. out of all the things people immediately cede to AI they cede their human ability to communicate and convey/share ideas. this timeline is bon…

I’ve become overly sensitive to it as well because it’s such a reliable indicator that there are other problems in the work. I’ve wasted so much time looking at interesting repos this year before discovering that one of the main claims was a hallucination, or that when I got to the specific part of the codebase it just had a big note from the LLM that’s it’s a placeholder until it can figure out how to do the request…

> The people who have AI write their articles don’t care if it works or if it’s correct.

I'd build on this: The people who have AI write their articles very likely don't know how their thing works or is correct. High chance they'll stumble when they are expected to speak about whatever it is they are presenting with some authority and demonstration of knowledge. Human to human, not being able to do that = obliterates trust. Places it somewhere near the realm of misinformation, which everyone unilaterally has no interest in consuming.

Good luck to people who want to fluff expertise and present as more-capable for job prospects, the world is shit and I know there's more people who need income than there are jobs that provide for our basic human needs, but this level of AI crutching is just going to bode poorly for those who think this is going to get them where they need to go.

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#25
post #3

> on Apple Silicon, a WebAssembly module's linear memory can be shared directly with the GPU: no copies, no serialization, no intermediate buffers enhance > no copies, no serialization, no intermediate buffers would it kill people to write their own stuff why are we doing this. out of all the things people immediately cede to AI they cede their human ability to communicate and convey/share ideas. this timeline is bon…

I don't know, to me your sentiment sounds a lot like how back in the day they used to say "you can't just use a calculator all the time, use your brain and show the work on pen and paper". humans have been using tools to communicate since pre-history. language itself is one tool of communication invented to supersede body-language and grunting and noises. the thought and idea is theirs, it was communicated. Would it…

Calculators have never been the medium in which we communicate our human experience and knowledge transfer. Calculators aren't part of the social fabric or culture. Very 2d extrapolation that somehow resulted in an alleged parallel. Language is woven deeply into civilization and our histories & been a part of our species literal survival against the most unforgiving odds/environments. Using what is effectively a ghost writer nukes trust. You cannot ascertain anything about the person behind the blog if it's clear they used AI to write it. And without that there's no way to infer expertise, rule out hallucinations, falsehoods presented as matter of fact, and the whole broad set of things LLM's get wrong because of their limitations as a technology. I have literally nothing to go off of that would prove this person knows what they are talking about. Why would anyone want to consume that?

Would it kill anyone at all to add a preamble that is forthcoming about using AI to write something? A chance to say these are my ideas and I've used claude to help me state it eloquently because etc ? Not doing that, presenting as more capable/knowing than one probably is, is what destroys trust immediately the moment it's sniffed out that AI was used to write something.

It's irresponsible, a self-nerf, and it's annoying. Venn diagram there is basically a circle. We're all familiar with how vibe coding appears to weaken your ability to write code, like skipping the gym and expecting good muscle density. All I'm saying is people shouldn't be skipping the gym for literally communicating with each other because there's gonna be a lot of times in life where you're not gonna be able to whip out chat jippity to continue a real conversation with another person. Ceding that turf means you're willingly trading your ability to deal with real life scenarios with other human beings for short term gain. It's funny how the universe tends to find balance. Yeah, being well read and expressing ideas well is a skill, it takes work.

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#27
> Apple Silicon changes the physics. The CPU and GPU share the same physical memory (Apple's Unified Memory Architecture) ... no bus!

Beware the reality distortion field: This is of course how it's worked on most x86 machines for a long time. And also on most Macs when they were using Intel chips.

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#28
post #27

> Apple Silicon changes the physics. The CPU and GPU share the same physical memory (Apple's Unified Memory Architecture) ... no bus! Beware the reality distortion field: This is of course how it's worked on most x86 machines for a long time. And also on most Macs when they were using Intel chips.

Why did all my x86 onboard iGPU reserve a fixed amount of RAM on boot, inaccessible to the OS? Why do dGPU bring their own VRAM and how to directly manipulate it from the CPU without copying?

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#29
post #27

> Apple Silicon changes the physics. The CPU and GPU share the same physical memory (Apple's Unified Memory Architecture) ... no bus! Beware the reality distortion field: This is of course how it's worked on most x86 machines for a long time. And also on most Macs when they were using Intel chips.

Why did all my x86 onboard iGPU reserve a fixed amount of RAM on boot, inaccessible to the OS? Why do dGPU bring their own VRAM and how to directly manipulate it from the CPU without copying?

To the first question: blame Windows I guess. But even on older chips, GPU code could access memory allocated on the CPU side so this didn't cap the amount of data your GPGPU code could crunch.

Re: Zero-Copy GPU Inference from WebAssembly on Apple Silicon

#30
post #26

I'm pretty sure this is just "yes (parts of), memory control in WASM works"[1]. The whole Apple Silicon thing is (in this case) just added details that don't actually matter. [1] https://github.com/WebAssembly/memory-control/blob/main/prop...

Apple Silicon uses unified memory where the CPU and GPU use the exact same memory and no copies from RAM to VRAM are needed. The article opens with mentioning just that and indeed it is the whole point of the article.
Post reply on HN