Viewing profile — robertkarl

[dead]

https://arxiv.org/abs/2606.00206 In this paper they nerf an LLMs ability to emit waffling thinking tokens like "wait", "but", "alternatively", and the models (they're old, small mo…

You can trade off latency / accuracy / cost for any ML task. And with the local models.... the cost is free. Having a local Qwen check another Qwen's work increases the accuracy qu…

This looks sick. I was going to download it but for $10 I am more willing to attempt asking Claude to implement something like it, than to purchase. I would be more willing to purc…

it's also a capable local inference stack!

I can't get excited about these benchmarks they're leading with. I've looked at the Terminal-Bench questions and I just think they're irrelevant. And SWE-Bench has serious flaws, e…

I wrote this blog post about killing a startup idea fast. AI tools help, but talking to humans about workflows and constraints is where it's at.

Ironically, parts of this read as if Sam prompted it with "Write AI bad, but in 16th grade language." What is homogeneously portentous cack? > The language of angels does a surpris…

I emailed dang to politely ask to make the link point to the Verge article since I can't update it.

My bad. I had trouble finding the original source when I googled for it and grabbed a link. I was originally shown a screenshot of a x.com post.

Cancellation effective June 30. This was a _pilot_ launched in December that accidentally consumed their 2026 yearly target spend on AI! I expect the r/LocalLLaMA guys to be going …

https://archive.ph/WfCta

How do you test? I made this comment elsewhere... but I don't see a good benchmark that covers "how good is this thing at actually driving coding with tool use locally"?

I'm interested in how you evaluate quantized models against each other; haven't found a benchmark I love for that. I love this example about 27B debugging. I've seen similar succes…

One thing you can do is offload from Claude to a dumb local model for summarizing. Local LLM sub-agents.

I am trying to figure this out too... what I am seeing is that the local models like Qwen 3.5 family that fit on hardware like yours handle ambiguity poorly. But are capable of emi…

PocketOS's website says "Service Disruption: We're currently experiencing a major outage caused by an infrastructure incident at one of our service providers. We are actively worki…

For what it's worth: here's my experience in the first 10 minutes of using Qwen locally to write some code. https://github.com/robertkarl/local-qwen-first-10-minutes it includes so…

That also was really opaque to me RE: API access. I initially thought at $200/month I could get whatever I needed. I eventually set up a OpenAI API with a few bucks to try what I w…