51–59 of 59 posts
Earlier quoted context omitted.
To the first question: blame Windows I guess. But even on older chips, GPU code could access memory allocated on the CPU side so this didn't cap the amount of data your GPGPU code could crunch.
I remember this was mostly a BIOS setting how much memory to allocate for iGPU - and once set in the BIOS, that memory was not accessible to the underlying OS (besides GPU I/O).
Earlier quoted context omitted.
Calculators have never been the medium in which we communicate our human experience and knowledge transfer. Calculators aren't part of the social fabric or culture. Very 2d extrapolation that somehow resulted in an alleged parallel. Language is woven deeply into civilization and our histories & been a part of our species literal survival against the most unforgiving odds/environments. Using what is effectively a ghos…
It's not that deep man, it's just a blog post about some software library. There's no civilisational communication going on here, relax. This whole thing will become irrelevant in a few decades before the end of our lifespans. It's just never that deep. Why does it matter if it's their thought or not. If you currently care about GPU inference from webassembly on apple silicon, you can use this article. That's really…
I'm pretty sure this is just "yes (parts of), memory control in WASM works"[1]. The whole Apple Silicon thing is (in this case) just added details that don't actually matter. [1] https://github.com/WebAssembly/memory-control/blob/main/prop...
Apple Silicon uses unified memory where the CPU and GPU use the exact same memory and no copies from RAM to VRAM are needed. The article opens with mentioning just that and indeed it is the whole point of the article.
That's the same no matter the physical memory system architecture.
> Apple Silicon changes the physics. The CPU and GPU share the same physical memory (Apple's Unified Memory Architecture) ... no bus! Beware the reality distortion field: This is of course how it's worked on most x86 machines for a long time. And also on most Macs when they were using Intel chips.
What is different then is the combination of
1. UMA memory (and yes, iGPU had this, pre-M1) 2. enough bandwidth / GPU throughput for local inference 3. straightforward `makeBuffer(bytesNoCopy:)` path
So, the novelty isn't the shared memory itself, but the whole chain lining up to make the Wasm linear memory -> Metal-buffer approach practical + performant enough.
(and not saying there's some Apple Silicon magic here either ... it'd work anywhere there was UMA and no-copy host-pointer path)
I'm curious what this offers over just building the host side code to be native?
The value would be in actor processes, where you can delegate inference without paying the 'copy tax' for crossing the sandbox boundary.
So, less "inference engine" and more "Tmux for AI agents"
Think pausing, moving, resuming, swapping model backend.
I scoped the post to memory architecture, since it was the least obvious part ... will follow up with one about the actor model aspect.
Doesn't work on web browsers, only with one headless runtime, one one CPU architecture. What's even the point of using webassembly here?
loading third party agents in a sandbox with full custom model support. right now you need to either run that code directly (super dangerous) use a vm/container (slow and complicated) or a interpreter like lua (language bound, slow and weak security). wasm is perfect for this, its almost native speed, built for security and language neutral. onnx and coreml are secure but they can only do the actual model not all the…
It's less about browsers, and more about server/edge/local-agent runtimes.
Wasm lets you have
- sandboxing (untrusted actor code)
- clean snapshot/restore
- portability of actor across machines
If you don’t need those properties, then yes ... native is obviously the better choice
I'm curious what this offers over just building the host side code to be native?
Yes, simply for local inference -- not much, native is the obvious choice. The value would be in actor processes, where you can delegate inference without paying the 'copy tax' for crossing the sandbox boundary. So, less "inference engine" and more "Tmux for AI agents" Think pausing, moving, resuming, swapping model backend. I scoped the post to memory architecture, since it was the least obvious part ... will follow…
Earlier quoted context omitted.
I don't know, to me your sentiment sounds a lot like how back in the day they used to say "you can't just use a calculator all the time, use your brain and show the work on pen and paper". humans have been using tools to communicate since pre-history. language itself is one tool of communication invented to supersede body-language and grunting and noises. the thought and idea is theirs, it was communicated. Would it…
> language itself is one tool of communication invented to supersede body-language and grunting and noises That's a pretty utilitarian view of language. How would it feel if everyone spoke and wrote like a PR representative? This is what an article written by an LLM is starting to sound like. I'm even willing to argue that the way in which you convey your ideas is as important as the idea itself. Like we could all be…
I don't think all this crusading has any place in a technical discussion.
Earlier quoted context omitted.
I don't know, to me your sentiment sounds a lot like how back in the day they used to say "you can't just use a calculator all the time, use your brain and show the work on pen and paper". humans have been using tools to communicate since pre-history. language itself is one tool of communication invented to supersede body-language and grunting and noises. the thought and idea is theirs, it was communicated. Would it…
> the thought and idea is theirs, it was communicated Are they? I don't know how much they used AI, the entire article could be written from a one sentence prompt and so I'd argue that the thoughts and ideas are not their own. This isn't like using a spell checker, it's like using a ghost writer.
I think efforts are better spent boiling oceans, or getting angry at the sun.