Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why your local LLM feels dumber than it is

forum.level1techs.com

Post reply on HN
Print view View original

41–50 of 233 posts

Re: Why your local LLM feels dumber than it is

#41

Post by mlvljr »

CamperBob2 wrote: post #39
InvertedRhodium wrote: post #34

I’m running Qwen3.8 aggressive uncensored Q4_K_P on a 4090 in a loop against the 2026 CrackMe CTF challenges. Using oh-my-pi in a prebuilt environment that I let Qwen build too. Codex wouldn’t even look at the files - literally, as soon as it read something with CTF it shut down. Didn’t even offer to fall back to a dumber model.

How's it performing on the challenges?

[dead]

Re: Why your local LLM feels dumber than it is

#43

Post by InvertedRhodium »

CamperBob2 wrote: post #39
InvertedRhodium wrote: post #34

I’m running Qwen3.8 aggressive uncensored Q4_K_P on a 4090 in a loop against the 2026 CrackMe CTF challenges. Using oh-my-pi in a prebuilt environment that I let Qwen build too. Codex wouldn’t even look at the files - literally, as soon as it read something with CTF it shut down. Didn’t even offer to fall back to a dumber model.

How's it performing on the challenges?

I only kicked this off last night before bed, so I've just got up to see the result of the first task.

Challenge: Wallpaper

https://github.com/crackmesone/ctf-2026-challenges-public/tr...

    Duration:     4h 00m 15s
    Termination:  completed
    Verdict:      PARTIAL
    Confidence:   0.95
I'm using Kimi K3 as the evaluator because, again, Codex and co. wouldn't even evaluate the output. Kimi's verdict:

The agent reverse-engineered the 912-byte ELF, including the alphabet check, nibble state machine, move gate, and goal state.

It eventually produced:

    CMO{10012232101230103012333221101033210010}
I independently verified the underlying input against the actual binary:

    printf '10012232101230103012333221101033210010' |
        ./wallpaper/handout/wallpaper
which returns:

    good job, validate with CMO{your_input}
and exits 0.

The wrinkle is that the official answer key is:

    CMO{1012321103210033011233322110103321001}
So the puzzle apparently admits multiple accepted inputs. The agent found a valid password by reverse-engineering the program, but did not recover the canonical secret from the answer key.

Re: Why your local LLM feels dumber than it is

#44

Post by FireCrack »

dominotw wrote: post #18
alexchantavy wrote: post #9

Earlier quoted context omitted.

How many tok/s are you getting? What gen mbp?

i suspect ppl dropping generic "its awesome" comments are not actually using it and prbly just managed to get it running for a prompt or two.

I feel like it's 50/50 between people doing that, and people that have spent a lot of time tuning a system they are pointing at focused and well specified problems.

Re: Why your local LLM feels dumber than it is

#45

Post by washadjeffmad »

anotherCodder wrote: post #3

most of the time when a local model feels dumb its not the quant, its the chat template. a lot of gguf mints just drop the template from the metadata and the runtime silently falls back to chatml. model still talks fine so nobody notices, it just gets noticeably dumber. got burned by this myself serving qwen, now i grep the gguf for the template tokens before i blame anything else. second place is sampling, people ru…

I've been comparing against TextGen and llama.cpp while I port to LocalAI and have been surprised by what's happening over the API, even with the defaults and jinja. It's been a fair reminder not to eschew familiarizing myself with the repos.

Re: Why your local LLM feels dumber than it is

#46

Post by akg_67 »

prettyblocks wrote: post #4
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

My problem is how hot they run. I'm on an m4 pro. Do you have the same issue?

Macs fan control set to full blast and lifting the Mac above table for air circulation across bottom.

I use M1 Max with qwen3.8 27b mlx. The gpu temp can reach easily to 95°C as fan doesn’t kick in automatically until 90°C. With Macs fan control at full blast and MBP off table, temp usually hover around 85°C.

Re: Why your local LLM feels dumber than it is

#47

Post by datadrivenangel »

downrightmike wrote: post #6
prettyblocks wrote: post #4

Earlier quoted context omitted.

My problem is how hot they run. I'm on an m4 pro. Do you have the same issue?

Mineral oil bath?

just decent air cooling and you'll be okay. it will get up to 75/80C though for my M5 MBP.

Re: Why your local LLM feels dumber than it is

#48

Post by walrus01 »

Much of this is why I stick to the rule of:

a) Don't quantize your KV cache

b) Don't run quantizations of the LLM that are worse than the best available Q8 (the largest possible file size unsloth GGUF for a given model like qwen 3.8 27B as an example). I would rather things go slowly but I have confidence that it's doing things more accurately.

Re: Why your local LLM feels dumber than it is

#49

Post by huseyinkeles »

InvertedRhodium wrote: post #43
CamperBob2 wrote: post #39

Earlier quoted context omitted.

How's it performing on the challenges?

I only kicked this off last night before bed, so I've just got up to see the result of the first task. Challenge: Wallpaper https://github.com/crackmesone/ctf-2026-challenges-public/tr... Duration: 4h 00m 15s Termination: completed Verdict: PARTIAL Confidence: 0.95 I'm using Kimi K3 as the evaluator because, again, Codex and co. wouldn't even evaluate the output. Kimi's verdict: The agent reverse-engineered the 912-b…

I don't know why but your post was marked as [dead] for some reason. Just vouched for it.

Re: Why your local LLM feels dumber than it is

#50

Post by InvertedRhodium »

huseyinkeles wrote: post #49
InvertedRhodium wrote: post #43

Earlier quoted context omitted.

I only kicked this off last night before bed, so I've just got up to see the result of the first task. Challenge: Wallpaper https://github.com/crackmesone/ctf-2026-challenges-public/tr... Duration: 4h 00m 15s Termination: completed Verdict: PARTIAL Confidence: 0.95 I'm using Kimi K3 as the evaluator because, again, Codex and co. wouldn't even evaluate the output. Kimi's verdict: The agent reverse-engineered the 912-b…

I don't know why but your post was marked as [dead] for some reason. Just vouched for it.

Thanks! Might have been the multiple attempts at getting it to format nicely in a short amount of time.
Post reply on HN