Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

Post by throwawayffffas » Tue, Jun 16, 2026, 1:53 PM UTC

rootlocus wrote: post #100

horsawlarway wrote: post #49

For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with dual RTX3090s in it (I was going to build a new gaming machine anyways, and the llama release had just dropped so I tacked…

2x RTX3090 are around $4400. Without any electricity costs or other parts, that's 3.6 years of $100/m claude.

Post by jrm4 » Tue, Jun 16, 2026, 2:01 PM UTC

robertlagrant wrote: post #462

jrm4 wrote: post #331

Earlier quoted context omitted.

Having, e.g. seen Microsoft maintain a monopoly for well over a decade, there's nothing in my experience that suggests that "quality always beats hype" is remotely true. It's entirely possible Claude is just winning the hype game.

Microsoft have not maintained a monopoly on search, mobile, or maps, and they seem to mostly maintain their large market segments based on familiarity, not hype.

Post by euroderf » Tue, Jun 16, 2026, 2:02 PM UTC

alimbada wrote: post #500

euroderf wrote: post #203

Is anyone managing to do this on a Mac with a measly 8GB ? Asking for a friend.

You could try running the smaller QAT Gemma 4 models but I doubt they'll be very good for software engineering work.

Post by rhdunn » Tue, Jun 16, 2026, 2:12 PM UTC

gwerbin wrote: post #498

girvo wrote: post #293

Earlier quoted context omitted.

Right. Tokens/s decode isn't the most important thing to me: wall clock time for task completion is. And tracking all of that, on my GB10-based Asus box, Step 3.7 Flash at IQ4_XS beats Qwen 3.6 27B despite the latter having MTP, on all of my actual coding task evaluations in real codebases. Qwen seems better at one-shotting things based on vague prompts to an acceptable degree, but thats literally not what I use thes…

Do you think the choice of quantization matters that much for other models? I've seen a lot of discussion about different quantization and FP formats but I feel totally unequipped to make an informed decision about what to try. What's your evaluation setup like? It sounds like maybe the best thing to do is have a realistic evaluation that resembles your actual intended workload and workflow, and then just try everyth…

    def count_repeats(text: str, length: int) -> int:
        n = len(text)
        pattern = text[n - length : n]
        count = 1 # Include the end of the string as matching the substring.

        text = text[: -length]
        while text.endswith(pattern):
            text = text[: -length]
            count = count + 1

        return count


    def repeats(output: str, context: dict[str, any]) -> bool|float|dict[str, any]:
        threshold = context.get('config', {}).get('threshold', 3)
        count = 0
        length = 0

        for n in range(1, (len(output) // 2) + 1):
            n_count = count_repeats(output, n)
            if n_count > count:
                count = n_count
                length = n

        if count >= threshold:
            return { 'pass': True, 'score': 1.0, 'reason': f'Output repeats {count} times with length {length}.' }
        else:
            return { 'pass': False, 'score': 0.0, 'reason': f'Output doesn\'t repeat {threshold} or more times.' }


    def no_repeats(output: str, context) -> dict[str, any]:
        result = repeats(output, context)
        result['pass'] = not result['pass']
        result['score'] = 1.0 - result['score']
        return result

    defaultTest:
      assert:
        - # ----- The output doesn't repeat/get stuck in a loop.
          type: python
          value: file://asserts/repeat.py:no_repeats

Post by tpm » Tue, Jun 16, 2026, 2:36 PM UTC

_zoltan_ wrote: post #418

tpm wrote: post #392

Earlier quoted context omitted.

It's ok if you can send your code and data to the provider. Some of us can't.

We're discussing home use. You can. You just don't want to. Huge difference.