Live data from Hacker News

Show HN: State of the Art of Coding Models, According to Hacker News Commenters

hnup.date

91–96 of 96 posts

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#91

Earlier quoted context omitted.

It's so interesting to see the wild pendulum swings of LLM sentiment here. If one likes a model then it's capable of one-shotting entire apps. Otherwise it's "only suitable for the most trivial tasks". Never in between.

I have no trivial tasks. Just last week, I was trying to map the weird and wonderful column names emitted by a NetScaler’s detailed REST log into OpenTelemetry semantics. The NetScaler is basically abandoned by its dying vendor. Hence, its new features like sending logs directly to Splunk compatible receivers are basically undocumented. I’m sure there’s like three of us masochists out there stumbling our way through…

(I would drop this somewhere more relevant but apparently replying to a thing from 17 days ago would be necroing here + PMs are hard or something)

>[1] With what prompt!? I like the terse output! Do share...

Not sure if it's complete or completely right, but the matter came up in a recent session, and when asked what gave, Jim 'n' I came up with some supposedly relevant factors, at least one of which is news to me (supposedly, steering LLMs using negative instructions isn't counterproductive anymore (not that I'd been resisting the temptation anyway)):

https://gemini.google.com/share/7af54a6861d7#:~:text=What%20...

With the caveat (or bonus) that it can go (?:too)? far when told to "be blunt" and not to "pull punches":

https://i.vgy.me/WHRZD7.png (from 2024-09) (in this case, in user-config persistent instructions in Kagi's multi-LLM thing)

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#92
I use both Codex & Claude Code. With Codex, their long running agents are still not as good as Claude Code. But I don't complain because they burn very less tokens as compared to Claude Code for the same task. But having said that, if I am provided limitless tokens for each of them, my go to harness will be Claude Code

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#93
post #2

Interpreting these metrics is quite interesting. One thing for sure is that while Claude is currently taking the #1 spot in mentions, it carries a lot of negative sentiment due to API pricing policies and frequent server downtime. On the other hand, the runner-up, GPT-5.5, actually seems to have more positive feedback. Personally, my experience with Codex wasn't as good as with Claude Code (Codex freezes on Windows m…

[flagged]

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#94
post #66

Earlier quoted context omitted.

Didn't expect it to get hammered like that, just added caching for the sheets request. Thanks, my guy ;) Backfilling it further is definitely in the cards, I just want to stabilize the methodology first. If a comment just mentions Opus without being more specific and in the absence of relevant context clues, it gets mapped to Opus Latest. So it's saying more about the model family than a specific version. Tbh I'll pr…

> If a comment just mentions Opus without being more specific and in the absence of relevant context clues, it gets mapped to Opus Latest Consider keeping this data point but instead calling it something like "Opus Unspecified". Let the user decide how to interpret it.

you prob just want to map ALL opuses to "opus-all" or somethign - do we really care on 4.5 vs 4.6 vs 4.7, we just want to see trendline over time

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#95
windows user, not a dev, engineer doing growth. I use Claude, codex, Gemini in my projects. The failure modes I hit far more are 2: an operational one, ie shell confusion. Claude executes commands that do not work or fail in cmd / PowerShell, permissions issues etc. In code (and marketing / growth stuff) i find it many times that i will ask from claude to verify the code it proposes with codex and then come back with the results. Depending on the task i may ask for consensus from the three models - thats fun!
Post reply on HN