Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't trust large context windows

garrit.xyz

Post reply on HN
Print view View original

131–140 of 211 posts

Re: Don't trust large context windows

#131

Post by api »

dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

Programming has already become this way. Opinions about different languages and architectures are taste, or sometimes even just vibes. Few try to actually ask “can I quantify whether microservices or monoliths are better in terms of either maintainability or scaling?”

A lot of this is a result of systems having long ago exceeded the complexity threshold of things people can hold in their heads. There are too many layers, subsystems, languages, APIs, all glued together. Attempts at radical simplification fail because each of those layers and subsystems has features or behaviors someone needs, and a lot of it isn’t even documented.

AI takes this to the extreme. I’ve already learned that certain models have “personalities.” Some are more likely to go with you on magical journeys into hallucination while others are more critical. Some are better at detail while others seem better at abstraction but fall over on detail. Some are better instruction followers. All their quirks are complex and the systems themselves are impossible to understand.

Computer systems are becoming organic, biological.

Re: Don't trust large context windows

#132

Post by vitally3643 »

cactusplant7374 wrote: post #125
dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

> But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. It will always be this way going forward. Everyone thinks differently about problems. In the past we had experts and only they could do the work at a high level. But now we have ma…

Your argument is that superstition is the way of the future and technical rigor no longer applies.

Because that's what OP is talking about. Superstition presented as factual advice instead of the technically rigorous and scientific fact that already exists.

You're being downvoted because you don't understand this fact, or indeed understand what you're saying at all.

I'll spell it out for you: technically and scientifically rigorous facts do actually exist, even in regards to LLMs. We can, in fact, obtain scientific and objective facts about how LLMs perform. It can be rigorously proven that certain context habits affect certain tasks positively or negatively. Your argument is that none of this matters more than superstition. And you're surprised that arguing to a room full of engineers and scientists that science is dead and superstition is the one true way forward gives you negative response.

Re: Don't trust large context windows

#133

Post by dofm »

api wrote: post #131
dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

Programming has already become this way. Opinions about different languages and architectures are taste, or sometimes even just vibes. Few try to actually ask “can I quantify whether microservices or monoliths are better in terms of either maintainability or scaling?” A lot of this is a result of systems having long ago exceeded the complexity threshold of things people can hold in their heads. There are too many lay…

"Feeping creaturism" has always been a problem, for sure.

But those technologies are layers, and there are reliable things that sometimes bubble across the boundaries — type hints, better code patterns to trigger compiler optimisation, interesting tricks with key column selection — and someone with expertise from that layer below can explain why, and their advice will always work in situations that are sufficiently similar.

You are right about AI personalities. Obvious even with the open weights models. Gemma and Qwen write code and documentation like people from different cultures. Because I guess they are a bit like that.

Re: Don't trust large context windows

#134

Post by tonyarkles »

dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

I feel your frustration for sure and agree to a large extent. Any attempts I’ve made to try to formalize any LLM-based workflows has resulted in me being again dismayed that no one seems to have any real idea of how or why certain things work or don’t work. So I just go back to /plan and “write this down in a markdown document for posterity before we iterate on the implementation”, hoping that maybe next month there might be something a little more rigorous with some kind of rational backing.

> Have you tried cleaning your context with dawn dish soap

I don’t do the glue stick thing at all because I don’t need to, but Dawn really seems to do a good job at getting my Bambu build plate working again. I didn’t seek it out specifically, I already had some for doing dishes. IPA hadn’t worked so I tried Dawn and it has gotten me back having prints stick multiple times now. Not quite up to N=30 yet.

Re: Don't trust large context windows

#135

Post by dofm »

CommieBobDole wrote: post #128
dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

This has always been a thing with IT advice, though - the more complex a system and the outcome, the harder it is to clearly define "better" or "worse". Add in the fact that LLMs are intensely and emphatically non-deterministic and LLM guidance basically becomes gardening advice. Heck, even the 'benchmarks' are mostly somebody's attempt to crystallize their vibes with varying amounts of success.

Gardening advice. Better analogy.

Re: Don't trust large context windows

#136

Post by carterschonwald »

context window size isnt quite the issue though, its that the attention mass kinda spreads out too much and everything kinda converges to a sortah global average region full of what we know to be slop! theres some really cool ways at the harness or model layer to mitigate this. just isnt really prioritized by the labs often.

Re: Don't trust large context windows

#137

Post by mountainriver »

SwellJoe wrote: post #47

Opus in recent versions is fine beyond 100k, but I usually do try to keep it under 200k. But, this is also why so-called "memory" systems are usually a mistake that make the models dumber. They don't have memory, they only have context, and every irrelevant fact you shove into the context is less context for the problem. Less distractions, better results. The way to have the agent remember things is to have it docume…

“Memory” systems are a way for developers to feel like they are contributing to AI

Re: Don't trust large context windows

#138

Post by cactusplant7374 »

vitally3643 wrote: post #132
cactusplant7374 wrote: post #125

Earlier quoted context omitted.

> But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. It will always be this way going forward. Everyone thinks differently about problems. In the past we had experts and only they could do the work at a high level. But now we have ma…

Your argument is that superstition is the way of the future and technical rigor no longer applies. Because that's what OP is talking about. Superstition presented as factual advice instead of the technically rigorous and scientific fact that already exists . You're being downvoted because you don't understand this fact, or indeed understand what you're saying at all. I'll spell it out for you: technically and scienti…

There aren't any good facts that exist regarding LLMs. It's a black box. Also, do not presume to know what I understand or don't understand from one comment.

> I'll spell it out for you

You are a rude and crude individual. I am not interested in discussing anything further with you.

Re: Don't trust large context windows

#139

Post by Revisional_Sin »

cactusplant7374 wrote: post #138
vitally3643 wrote: post #132

Earlier quoted context omitted.

Your argument is that superstition is the way of the future and technical rigor no longer applies. Because that's what OP is talking about. Superstition presented as factual advice instead of the technically rigorous and scientific fact that already exists . You're being downvoted because you don't understand this fact, or indeed understand what you're saying at all. I'll spell it out for you: technically and scienti…

There aren't any good facts that exist regarding LLMs. It's a black box. Also, do not presume to know what I understand or don't understand from one comment. > I'll spell it out for you You are a rude and crude individual. I am not interested in discussing anything further with you.

It's a black box, but you can run tests to quantify the behaviour and establish, for example, that a certain model is X% more likely to give a certain behaviour.

Re: Don't trust large context windows

#140

Post by dindunuf »

dofm wrote: post #123

I guess I am mostly enjoying learning the fundamentals of AI stuff, even though I disagree with the direction it is going. But I am struggling to put into words how alarming I find the comments on threads like this — all sorts of good-natured anecdotes about how XYZ works for them that are more like the suggestions in pet care or cookery threads on Facebook. (Or worse still, like any Facebook 3D printing group: anyon…

first of all, LLM-assisted coding is less than 3 years old. 3 years ago all we had was GPT-4 with 8192 token context, which wasn't enough for most things.

and second of all...

>Any shared sense of rigour is just completely torpedoed by the LLM world, particularly the cloud LLM world it seems, and we are reduced to cargo culting. Nobody is any more right or wrong than anyone else.

what "sense of rigour"? it's way too soon to put those rose-tinted glasses on.

Post reply on HN