411–418 of 418 posts
If file search indexes and such will be replaced by neural networks, this will be small but measurable improvement in battery life, speed and memory usage.
Earlier quoted context omitted.
Yeah they should make a laptop where you can choose what display you want to use, and which keyboard and mouse for that matter. It could be made cheaper by ditching the screen and keyboard, and heck I wouldn’t even mind if it were a bit bigger or heavier since it’ll just sit on or under my desk. That sort of laptop would be amazing.
They have computers that are built into keyboards now. Maybe that will do the trick. https://www.youtube.com/watch?v=J4yl2twJswM
Now that I think of it, I’d also like laptops that come in standardized sizes so I could stack them on shelves and mostly just interact with them through ssh/terminal. I could imagine those laptops being super popular for compute/storage
I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.
What we want as developers: To be able to implement functionality that utilizes a model for tasks like OCR, visual input and analysis, search or re-ranking etc, without having to implement an LLM API and pay for it. Instead we'd like to offer the functionality to users, possibly at no cost, and use their edge computing capacity to achieve it, by calling local protocols and models. What we want as users: To have advan…
The typical consumer doesn't care about any checkbox feature. They just care if they can play the games they care about and word/email/netflix. That being said, netflix would be an impossible app without gfx acceleration APIs that are enabled by specific CPU and/or GPU instruction sets. The typical consumer doesn't care about those CPU/GPU instruction sets. At least they don't care to know about them. However they wo…
Most regular people use AI to get restaurant recommendations or cheat on homework. I think you massively overestimate just how many people actually care about the more advanced features of AI services.
Earlier quoted context omitted.
> For certain workloads, NPU is faster than CPU by quite a bit WTF is an NPU ? What kind of instructions does it support ? Can it add 3 and 5 ? Can it compute matrices ?
Neural Processing Unit. A retarded GPU that does matrix multiplication. https://en.wikipedia.org/wiki/Neural_processing_unit In theory, its your math coprocessor for your 386.
A math coprocessor, AFAIK, can do much more than matrix multiplications. From my POV, having a separate chip doing a single math operation is (so 1970) lame.
Earlier quoted context omitted.
I've yet to see any convincing benchmarks showing that NPUs are more efficient than normal GPUs (that don't ignore the possibility of downclocking the GPU to make it run slower but more efficient)
NPUs are more energy efficient. There is no doubt that a systolic array uses less watts per computation than a tensor operation on a GPU, for these kinds of natural fit applications. Are they more performant? Hell no. But if you're going to do the calculation, and if you don't care about latency or throughput (e.g. batched processing of vector encodings), why not use the NPU? Especially on mobile/edge consumer device…
Maybe because they sleep all the time. To be able to use an NPU you need at least a compiler which generates code for this particular NPU and a CPU scheduler which can dispatch instructions to this NPU.
Earlier quoted context omitted.
They have computers that are built into keyboards now. Maybe that will do the trick. https://www.youtube.com/watch?v=J4yl2twJswM
No, my ideal laptop doesn’t have a keyboard. I’m imagining like a laptop where everything other than the cpu/gpu/ram/basic storage/nic are addressed by peripherals and I can just have one that sits in my office and I plug stuff into it. Now that I think of it, I’d also like laptops that come in standardized sizes so I could stack them on shelves and mostly just interact with them through ssh/terminal. I could imagine…
Earlier quoted context omitted.
Not that far away, you can run a useful model on flagship phones today, something around GPT 3.5's level. So we're probably only a few years out from today's SOTA models on our phones.
> you can run a useful model on flagship phones today How?
Just download the app and it has a few built-in model options. The best of those is probably Gemma-3 1B Q4 but on my Pixel 10 Pro I find the best performing model it can reasonably run is Qwen3 8B Q4_K_M.
You can download and run any GGUF compatible model with that app.