Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

271–280 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#271

Post by vardhanw »

entrope wrote: post #219
niek_pas wrote: post #205

Earlier quoted context omitted.

What does ‘latent’ mean in this context?

"Latent" means it (the space) is there all the time but usually not exposed or apparent. Latent space is a high-dimensional space that is not legible to humans, so normally reasoning gets constrained to progress in tokens instead so that it can be evaluated. I think https://arxiv.org/abs/2412.06769 is a good description of the premise of reasoning in latent space, although https://arxiv.org/abs/2604.15726 argues it's…

So, does "reasoning in latent space" mean that it does/will not use tokens (e.g. using CoT etc.)?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#272

Post by qeternity »

janalsncm wrote: post #159
fweimer wrote: post #135

Earlier quoted context omitted.

Centralized inference can easily increase batch size, leading to huge efficiency gains in the usual scenario where most users have just one or very few session. Using local resources efficiently requires some way to increase the batch size. I'm not sure if we are there yet.

I think the point is that if people are able to run inference on their laptops batch size efficiency won’t matter. And before that, businesses will be able to get decent results with dedicated inference hardware.

Jevons paradox: large purpose-fit data centers increase efficiency such that you can use AI in more places, and use more tokens for those tasks.

The future is not a single chat bot session of bs=1. The future is many agents performing many tasks in parallel for a single user. Large GPU clusters will always have the edge in efficiency.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#273

Post by tylerKorhonen »

madduci wrote: post #87
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

Tried yesterday on my own laptop (a UltraCore 7 255H without dedicated GPU,with 32 GB RAM), it wasn't even starting thinking, even on a small context window (65k)

> it wasn't even starting thinking

Probably stuck in prompt processing which is compute bound especially for iGPUs.

You've mentioned 3.5 - but it's actually the same model the only differences are training and implicit MTP support (affects prompt processing - can be disabled)

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#274

Post by qeternity »

mattmaroon wrote: post #127
ipdashc wrote: post #126

Earlier quoted context omitted.

Genuine question, what are you spending that on? The $20/month ChatGPT/Codex subscription has largely been enough for me as an IT worker.

There. I have the business plan with two seats and I use them both and blow through it pretty fast. I think it’s because much of what I have it do involves using a browser. For instance I have it pull various permits from cities and there’s no API for that.

Computer use will blow through tokens because it's doing image capture for everything.

You may have better and more reproducible results using browser controls that aren't image based, or writing tools that completely sidestep browser use.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#275

Post by coldtea »

shafyy wrote: post #200
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

Can it? I feel like I instantly recognize if I am chatting with an LLM or a human

"Feel" is doing a lot of work here.

You can recognize 70% of those (true positive rate) and still have a false negative rate of 30%, while thinking you got 100% of the AI ones!

The problem is that you'd be oblivious to those you don't recognize.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#276

Post by Havoc »

ionwake wrote: post #207

forget the paperclip problem, I worry one day the basilisk starts a genetic breeding program for cycling pelicans in an effort to assess itself. The pelican problem.

Well if things must go sideways it may as well be in a creative way

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#277

Post by apples_oranges »

syhol wrote: post #221
apples_oranges wrote: post #203

Idea: Qwen should change its name to OpenQwen - this would probably 10x their usage. :D

The word "open" has lost all meaning

Yes but investors and users seem to react well to it. Opencode for example gets mentioned more often than Cline even though it’s arguably inferior, openclaw is another example. Just name your stuff openX to win..

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#278

Post by StilesCrisis »

zahlman wrote: post #210
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

> Not if an LLM over chat can fool most people they're talking to a human (which it can) I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing. I really can't understand how this gap persists; but then, there seem to have been at least some people who couldn't sniff out ELIZA, back in the day, too.

Many, many people became friends/got romantically entangled with GPT-4o, to the point where OpenAI struggled to replace it due to user backlash.

Most users aren't very critical of the output. They just want a sycophantic ear, and 4o was perfect for that task. It's not _good_ but there is high demand for it.

Post reply on HN