Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why your local LLM feels dumber than it is

forum.level1techs.com

Post reply on HN
Print view View original

91–100 of 233 posts

Re: Why your local LLM feels dumber than it is

#91

Post by pmontra »

dmitrygr wrote: post #82
LeifCarrotson wrote: post #69

Earlier quoted context omitted.

There was a lovely window of a few years when processors were fast enough and low-power enough that real development work could trivially happen on a Macbook Air in a lounge. That window is waning as more and more memory and graphics processing power is being used locally. I suppose the future is most likely going to involve farming out AI requests to your desktop machine, your company's compute farm, or a cloud endp…

No idea what you are talking about. My battery lasts longer than ever while running vim and make and GCC. It’s amazing. Not sure why your windows are closing.

Because the local LLM, which you are not running, is running for much longer than gcc and is eating the battery. Different choices, different outcomes.

Re: Why your local LLM feels dumber than it is

#93

Post by adastra22 »

pmontra wrote: post #91
dmitrygr wrote: post #82

Earlier quoted context omitted.

No idea what you are talking about. My battery lasts longer than ever while running vim and make and GCC. It’s amazing. Not sure why your windows are closing.

Because the local LLM, which you are not running, is running for much longer than gcc and is eating the battery. Different choices, different outcomes.

So don’t run a local LLM?

Re: Why your local LLM feels dumber than it is

#94

Post by cellularmitosis »

velcrovan wrote: post #80
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.

We don’t know what quantization level was used for the weights or the kv cache for you or for parent poster, so this is probably an apples to oranges comparison.

Re: Why your local LLM feels dumber than it is

#96

Post by spacebacon »

cellularmitosis wrote: post #94
velcrovan wrote: post #80

Earlier quoted context omitted.

That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.

We don’t know what quantization level was used for the weights or the kv cache for you or for parent poster, so this is probably an apples to oranges comparison.

[dead]

Re: Why your local LLM feels dumber than it is

#97

Post by rahimnathwani »

riddlemethat wrote: post #70
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

I got the qwen 3.8 abliterated model running on my MacBook Pro M5 48GB and it's pretty nice having a local model that can do a lot of experimentation without rails.

orcarouter or obliteratus?

Re: Why your local LLM feels dumber than it is

#98

Post by s1gsegv »

velcrovan wrote: post #80
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.

They made a kind of strange decision with Qwen3.8 27B, the template defaults the reasoning_effort to xhigh. I found if you set it to medium it doesn’t just sit there churning forever.

Re: Why your local LLM feels dumber than it is

#100

Post by [deleted] »

velcrovan wrote: post #80
jonplackett wrote: post #2

I just got qwen 3.8 27b mlx running on my Macbook Pro and honestly I’m pretty blown away by how not-dumb it is.

That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.

[deleted]
Post reply on HN