Viewing profile — lhl

I started a project w/ the assumption that fully tuned for HIP kernels could get closer to roofline and beat llama.cpp's performance (and vs their ROCm engine it does!) but after t…

The one thing that's new/worth pointing out are the https://developer.amd.com/playbooks/ ( https://github.com/amd/playbooks ) - this is AMD's answer to Nvidia's playbooks ( https:/…

You got me curious, so I made a little harness comparison to my model test suite: Model Adapter Suite Score Passed Tasks --------------------------------- ------------ ------------…

https://x.com/thsottiaux/status/2058071172361998482 "A little secret. About 5% of our production traffic is on the Pi harness, about another 5% is on OpenCode. Reminder you can use…

I've been testing Ornith-1.0 35B (my own FP8-block quant) and I like it. It runs at >200 tok/s w/ vLLM on an RTX PRO 6000 (sm120), I've run >140M cached tokens of agentic coding wo…

It's still Python, but I removed torch dependencies (HIP/C++ for hot paths): http://github.com/shisa-ai/hipEngine/ There's a docs/ folder in there that is probably of interest as w…

Took a little longer to clean up than I expected. I'd recommend checking out the ROOFLINE and the LESSONS-LEARNED docs here: https://github.com/shisa-ai/hipEngine/tree/main/docs

When it's in a good state I'll open source it, I am keeping track of what optimizations make the most impact, stuff like this: ### Diagnosing parallelism pathologies (L1) *Grid occ…

I think especially with the ability for SOTA AI to optimize kernels more people should try their hand at making better inference for their specific hardware. I have an older W7900 …

Oh, is this actually out now? If so, great, but I took a quick look and didn't spot any third party review yet. For those interested in this laptop, personally I'd still wait for s…

For those that don't want their data trained on, OpenRouter allows you to have account-wide or per-request routing with either provider.data_collection: "deny" or zdr: true (zero d…

RDNA is a whole different (and much poorer supported) animal than CDNA. As someone with extensive experience in both, if you're asking the question, then, no. (If you're just looki…

BTW, a number of corrections. The TurboQuant paper was submitted to Arxiv back in April 2025: https://arxiv.org/abs/2504.19874 Current "TurboQuant" implementations are about 3.8X-4…

As some other people mentioned, using both/multiple is the way to go if it's within your means. I've been working on a wide range of relatively projects and I find that the latest …

Like others have mentioned, I think the premise of looking at the most popular few projects (pypi.org currently lists 771,120 projects) on pypi as any sort of proxy for AI coding i…

Funy that you mention multi-monitor since it's one of the reasons I eventually moved to Wayland. The only way to support different DPI monitors in X was to do janky scaling or even…

Yes, I read it and specifically pointed it out (that's why there are 3 hours of interactive logs). There are 4 other runs pushed now so you can see what actual clean room runs for …

I am not a theoretical CS or math expert by any means, but I have been wrangling coding agents for a while and reading the paper and the problems Stapper had with dealing w/ Claude…

I was a bit interested to do a replication and see if better harness could avoid some of the problems they ran w/ context management, poor instruction following, etc and it looks l…

Anecdotally, I will say that for my toughest jobs GPT-5+ High in `codex` has been the best tool I've used - CUDA->HIP porting, finding bugs in torch, websockets, etc, it's able to …

We do live in an age of frontier LLMs... For fun, I'll just use Kimi K2 (on Kagi Assistant). > Can you explain what this means and its significance? Assume that I'm a layperson wit…

Looks great, thanks for the suggestion!

I'd previously been giving Hyprland a try, but after lots of customization work, there were still a bunch of things I wasn't happy with and ended back on GNOME as a "just let me ge…

In Linux, you can set it as high as you want, although you should probably have a swap drive and still be prepared for you system to die if you set it to 128GiB. Here's how you'd s…

RDNA3 CUs do not have FP8 support and its INT8 runs at the same speed as FP16 so Strix Halo's max theoretical is basically 60 TFLOPS no matter how you slice it (well it has double …