Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

Post by ttoinou » Wed, Jun 17, 2026, 3:49 PM UTC

girvo wrote: post #568

ttoinou wrote: post #497

Earlier quoted context omitted.

I tried a bunch of stuff with step 3.5 and step 3.7 maybe not as much as you. Could you tell me what parameters and launched you’re using ? Antirez ds4 flash q2-q4 works almost out of the box for me

To be fair: if you're happy with ds4 then IMO stick with it! Step 3.7 is notably better than 3.5 1. Use the official StepFun GGUF, IQ4_XS - theirs is better tuned in my experience than the other quants 2. Temp 1.0 top_p 0.95 sampling parameters for reasoning/agentic coding 3. It's really quite important that you don't quantise the KV cache: it made a surprising amount of difference to the looping and over thinking I…

Post by blurbleblurble » Wed, Jun 17, 2026, 5:10 PM UTC

horsawlarway wrote: post #65

blurbleblurble wrote: post #33

My experience is that it's not the models themselves that are limiting right now, it's the clunky alternative harnesses with weird missing features making for bad ergonomics around stuff like queue management, interruption, subagents, goals, etc.

Pi is decent. I've used the cli agents for claude, cursor, and pi, plus several custom harnesses I've written myself from time to time as experiments (and I guess technically gastown, if we're calling that a harness). Pi is... just fine. It does what I need it to, has a decent selection of tooling out of the box, integrates nicely with other tools, and generally gets out of my way enough that I don't think about it m…

Post by supjeff » Wed, Jun 17, 2026, 6:03 PM UTC

Post by cpburns2009 » Wed, Jun 17, 2026, 7:03 PM UTC

ryandrake wrote: post #16

Always a bit disappointed in the details in these kinds of threads. When you do get answers, they're never specific enough to try out on your own. It'll be something like "I use Qwen 3.5 and get great results!" OK but what quantization are you using? What llama parameters? What context size? What GPU are you running it on, and how much VRAM does it have? Are you hosting it on a separate box, or running it locally on…

    # server-2.env
    LLAMA_ARGS="--host 0.0.0.0 --port 8082 --sleep-idle-seconds 1800"
    LLAMA_ARG_MODELS_MAX=1
    LLAMA_ARG_MODELS_PRESET=/usr/local/etc/llama.cpp/models-2.ini

    # models-2.ini
    [*]
    device = CUDA0
    fit = on
    cache-type-k = q8_0
    cache-type-v = q8_0

    [Qwen3.6 27B It Q6]
    model = /mnt/data1/llm/huggingface/hub/models--unsloth--Qwen3.6-27B-GGUF/snapshots/82d411acf4a06cfb8d9b073a5211bf410bfc29bf/Qwen3.6-27B-UD-Q6_K_XL.gguf
    alias = qwen3.6-27b-it:q6
    ctx-size = 200000
    temp = 0.6
    top-p = 0.8
    top-k = 20
    min-p = 0.0
    presence-penalty = 1.5
    repeat-penalty = 1.0
    chat-template-kwargs = {"enable_thinking":false}

Post by amritanshuamar » Thu, Jun 18, 2026, 2:38 AM UTC