Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why your local LLM feels dumber than it is

forum.level1techs.com

Post reply on HN
Print view View original

81–90 of 233 posts

Re: Why your local LLM feels dumber than it is

#81

Post by WD-42 »

Forgeties79 wrote: post #65
akg_67 wrote: post #46

Earlier quoted context omitted.

Macs fan control set to full blast and lifting the Mac above table for air circulation across bottom. I use M1 Max with qwen3.8 27b mlx. The gpu temp can reach easily to 95°C as fan doesn’t kick in automatically until 90°C. With Macs fan control at full blast and MBP off table, temp usually hover around 85°C.

Yikes isn’t 95 well above the danger zone for most components or have things changed? I remember crypto mining in the early 2010’s and making sure to stay under 85C on my GPU.

Stuff runs so hot these days. I think one of the parameters pushed to keep Moores law from completely flattening was heat. My desktop regularly hits 95 just compiling code.

Re: Why your local LLM feels dumber than it is

#82

Post by dmitrygr »

LeifCarrotson wrote: post #69
coldtea wrote: post #61

Earlier quoted context omitted.

> It’s hot and also LOUD and runs the battery down quick. Who'd run this on battery? Or do you mean kills the battery faster even when used while charging?

There was a lovely window of a few years when processors were fast enough and low-power enough that real development work could trivially happen on a Macbook Air in a lounge. That window is waning as more and more memory and graphics processing power is being used locally. I suppose the future is most likely going to involve farming out AI requests to your desktop machine, your company's compute farm, or a cloud endp…

No idea what you are talking about. My battery lasts longer than ever while running vim and make and GCC. It’s amazing. Not sure why your windows are closing.

Re: Why your local LLM feels dumber than it is

#85

Post by w-ll »

velcrovan wrote: post #80
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.

its all still somewhat of a dice roll

Re: Why your local LLM feels dumber than it is

#86

Post by NamlchakKhandro »

velcrovan wrote: post #80
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.

[flagged]

Re: Why your local LLM feels dumber than it is

#88

Post by treesknees »

InvertedRhodium wrote: post #34

I’m running Qwen3.8 aggressive uncensored Q4_K_P on a 4090 in a loop against the 2026 CrackMe CTF challenges. Using oh-my-pi in a prebuilt environment that I let Qwen build too. Codex wouldn’t even look at the files - literally, as soon as it read something with CTF it shut down. Didn’t even offer to fall back to a dumber model.

This is similar to my use case as well. I’ll use a cloud model to identify security issues in a codebase and then use a report/spec output as input to the local model to build tests or verify the issue is real.

Re: Why your local LLM feels dumber than it is

#89

Post by nick_ »

rurban wrote: post #87
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

We were trying running a local gpt-oss 80GB model on a H100, and honestly I was surprised how dumb it was.

gpt-oss is about a year older than qwen 3.8 27b

Re: Why your local LLM feels dumber than it is

#90

Post by anon373839 »

rurban wrote: post #87
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

We were trying running a local gpt-oss 80GB model on a H100, and honestly I was surprised how dumb it was.

GPT-OSS 20B didn’t really merit the fanfare even when it was released; it’s definitely not competitive now. Even the 120B version has been well eclipsed by smaller LLMs at this point. The last version of Qwen 27B/35B was better, and now the new one is even better than that!
Post reply on HN