Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Are Complicated Now

ianbarber.blog

Post reply on HN
Print view View original

51–60 of 86 posts

Re: LLMs Are Complicated Now

#51

Post by sowbug »

lproven wrote: post #5
christopherwxyz wrote: post #4

Earlier quoted context omitted.

It’s written by AI.

[[citation needed]] I am a professional writer and have been for over 30 years. (I do not use any form of LLM ever.) This means I read a lot . This also means that I have 30+ years of experience of readers not understanding what I wrote, or not getting further than the title, or not getting the main message, or inverting it in their heads, or inserting their own message and then complaining when I diverge, and an end…

I want Scrabble rules for HN AI challenges. If someone finds an AI-generated comment, the commenter has violated HN guidelines and the comment should be deleted. But if the accusation is wrong, there should be a penalty for the often massive disruption the accuser has caused to the discussion.

(As of now, that four-word low-effort comment has generated over a thousand words in response, none of which improve this article's discussion.)

Re: LLMs Are Complicated Now

#52

Post by zsyllepsis »

ffsm8 wrote: post #44
truvem wrote: post #41

[flagged]

If you think statefull LLMs would be easier to handle then stateless... Then I think you haven't done a lot of software engineering

Maybe a charitable reading of the parent comment, but my interpretation of it was that while the _models_ are stateless, modern deployments of these models for inference rely on state.

For example, tiered pricing for cached context relies on state, even if the models don’t.

Re: LLMs Are Complicated Now

#53

Post by KerrAvon »

segmondy wrote: post #20
charcircuit wrote: post #3

Why didn't this author compare Llama 3 with GLM 5.2 (released 1 week ago) which is a more standard attention based LLM? To compare 2 separate families of LLMs and then pointing out that they are different is not a surprising result and detracts from the point the author is trying to make. https://sebastianraschka.com/llm-architecture-gallery/?compa... If you look at it, the diagrams are very similar, but the main dif…

The author is correct, the model architecture is now much more complicated. You can see this if you use llama.cpp and follow the project. The earlier models were always fully implemented. Yet with more contributors, as of today tons of latest models only have partial implementation. DeepSeekv3.2 isn't fully implemented, same with KimiK2.6, GLM5.2+, DeepSeekv4 has no implementation, MiniMaxM3 not supported yet, Hy3-pr…

indeed, there's even a (pretty solid) custom server just for DS4 https://github.com/antirez/ds4

-- works very well on high-RAM Macs

Re: LLMs Are Complicated Now

#55

Post by zahlman »

jordanb wrote: post #29

It's the bitter-lesson to feature-engineering lifecycle. When a technique or technology is new people are making massive gains by just applying it to some use case, or gathering more data for training, or giving it more resources. As time goes on those "bitter lesson" gains start to hit the shallow part of the logistic curve and companies have to start investing more and more effort into engineering for each small, i…

I assume the choice of phrase "bitter lesson" is intentional irony (since the original concept is that you get better results by just scaling up and not trying to be clever with domain-specific knowledge)?

Re: LLMs Are Complicated Now

#56

Post by zahlman »

zsyllepsis wrote: post #52
ffsm8 wrote: post #44

Earlier quoted context omitted.

If you think statefull LLMs would be easier to handle then stateless... Then I think you haven't done a lot of software engineering

Maybe a charitable reading of the parent comment, but my interpretation of it was that while the _models_ are stateless, modern deployments of these models for inference rely on state. For example, tiered pricing for cached context relies on state, even if the models don’t.

For that matter, the agent harness accumulating "chat history" is state.

Re: LLMs Are Complicated Now

#57

Post by vivzkestrel »

- with all due respect, from a ux perspective, could you kindly add a page where i can see just the titles of all your blog posts

- https://ianbarber.blog/blogroll

- https://ianbarber.blog/archive

- https://ianbarber.blog/blog

- https://ianbarber.blog/posts

- none of the above links work

- i really dont want to scroll 200 pages just to see what your blog articles are

Re: LLMs Are Complicated Now

#58

Post by smasher164 »

vivzkestrel wrote: post #57

- with all due respect, from a ux perspective, could you kindly add a page where i can see just the titles of all your blog posts - https://ianbarber.blog/blogroll - https://ianbarber.blog/archive - https://ianbarber.blog/blog - https://ianbarber.blog/posts - none of the above links work - i really dont want to scroll 200 pages just to see what your blog articles are

lol yeah I guess the best move right now is to fetch their /feed and iterate through s

Re: LLMs Are Complicated Now

#59

Post by MarkSweep »

vivzkestrel wrote: post #57

- with all due respect, from a ux perspective, could you kindly add a page where i can see just the titles of all your blog posts - https://ianbarber.blog/blogroll - https://ianbarber.blog/archive - https://ianbarber.blog/blog - https://ianbarber.blog/posts - none of the above links work - i really dont want to scroll 200 pages just to see what your blog articles are

There is https://ianbarber.blog/feed

Re: LLMs Are Complicated Now

#60

Post by seanmcdirmid »

ffsm8 wrote: post #44
truvem wrote: post #41

[flagged]

If you think statefull LLMs would be easier to handle then stateless... Then I think you haven't done a lot of software engineering

Of course you can pass in your own state, but I always wondered about an LLM that has conversation context stay resident in GPU memory somehow.

Or maybe this already effectively covered by context caching and the gains would be minimal (stateless, but if you pass in the same context or the same head context, it’s already in GPU memory and doesn’t need to be loaded?).

Post reply on HN