Viewing profile — om8

Why int4? There are a lot of superior 4 bit formats like nf4 from bitsandbytes.

> So how can I regain my ability to focus? Getting diagnosed with ADHD and taking meds did the trick for me. Not sure how relevant this is, since inability to focus was always an i…

This project needs webgpu -- I did it on cpu about a year ago. My demo uses 2 bit quantization to run llama3 models on any device with enough ram. https://galqiwi.github.io/aqlm-rs…

> manufacturing companies making the flimsiest, cheapest, plastic crap to save 1/3 of a cent on every mop they produce. Designed to work for the least amount of time before needing…

Oh, yes, you are right about watts. My bad

> volt-amperes are technically the same unit as watts volt-amperes are joules

Cool way to think about GWh/year: 1 GWh/year = (10 ** 9) / 24 / 365.25 / (10 ** 6) MW = 0.11 MW 70 GWh/year = 8 MW 1755 GWh/year = 200 MW 252 GWh/year = 29 MW

cargo/uv/go have lock files though

https://docs.vllm.ai/en/v0.20.0/api/vllm/model_executor/laye... `vllm.model_executor.layers.quantization.turboquant` > The technique implemented here consists of the scalar case of…

These are very different media types with very different goals.

Is there a way to disable it? Sometimes I value agent not having knowledge that it needs to cut corners

https://steveklabnik.com/writing/does-unsafe-undermine-rusts...

... Yet.

FYI you can run just `uvx pdm`

Also got it, found this thread by googling "ycombiinator"

Also got it, found this thread by googling "ycombiinator"

Have a similar project. Also written in rust, runs in a browser using web assembly In-browser demo: https://galqiwi.github.io/aqlm-rs Source code: https://github.com/galqiwi/demo-a…

https://web.archive.org/web/20250914222638/http://ewaste.fka...

Oh, ok. Thanks, I agree

Why? They acquired books, that’s what they do

Good rationalism includes empiricism though

llama.cpp is a mess and ollama is right to move on from it

Of course it is. GPT-5 is one of the most anticipated things in AI right now. To live up to the hype, it needs to be a reasoning model.

It’s unfortunate that llama.cpp’s code is a mess. It’s impossible to make any meaningful contributions to it.