Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: State of the Art of Coding Models, According to Hacker News Commenters

hnup.date

Post reply on HN
Print view View original

11–20 of 96 posts

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#11

Post by smeej »

Jabbles wrote: post #3

Please fix your graph so the names of the models are readable

Came here to offer this feedback. If I can't see the name of the model, nothing else in the chart really matters to me. I even tried going to the Google Sheet.

It's way too important a piece of information not to have it visible.

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#12

Post by yunusabd »

ranger_danger wrote: post #7

Just FYI this article seems to define "start of the art" as "popular", as measured by "total mentions and user sentiment", without any bearing on the technical abilities or actual usage of the model.

Calling it sota might be a bit provocative, but what actually is the "state of the art"? We have benchmarks, but those are getting increasingly gamed and don't necessarily reflect the actual performance of a model, see Opus 4.7. So I think it's useful to have real world data from actual users as an additional data point.

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#14

Post by pbgcp2026 »

So, it's a webpage with 3 paragraphs and a simple chart. It has: 1) terrible color scheme – fine, I switch to reader mode 2) shitloads of JS - fine, NoScript works, page breaks 3) Fancy "design" with simple graph but unreadable X axis labels - fine, I can use screen zoom for that ... to see 3x "Claude O..." LOL are we playing guess-me-over game? 4) ... "LxxxLxxx - Learn languages with YouTube!"

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#15

Post by yunusabd »

smeej wrote: post #11
Jabbles wrote: post #3

Please fix your graph so the names of the models are readable

Came here to offer this feedback. If I can't see the name of the model, nothing else in the chart really matters to me. I even tried going to the Google Sheet. It's way too important a piece of information not to have it visible.

Thanks, I replaced it with a custom graph, should be easier to read now.

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#16

Post by marcuskaz »

yunusabd wrote: post #10
marcuskaz wrote: post #6

Earlier quoted context omitted.

Also, the stacked graph only allows you to quickly see total mentions, really hard to compare negative or positive sentiment across models at a glance.

Yep, a toggle to scale all columns to the same height could solve this. I'll look into it when I do the custom graph. Edit: Done

Much better, nice update!

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#18

Post by 2ndorderthought »

Interesting to see the positive sentiment around kimi2.6 qwen3.6 and deepseek relative to the negative. I hope the trend of people appreciating open models continue. They aren't namesakes yet, but it's a higher percentage then I thought it would be. Especially on HN where we are all talking about businesses.

I am upset because now anthropic, openai, meta, etc will continue their smear campaigns here. But I am also happy because it will make HN less useful when they do.

Everything is a give and take I guess. Excited to see where the equilibrium sits

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#19

Post by awesome_dude »

jdw64 wrote: post #2

Interpreting these metrics is quite interesting. One thing for sure is that while Claude is currently taking the #1 spot in mentions, it carries a lot of negative sentiment due to API pricing policies and frequent server downtime. On the other hand, the runner-up, GPT-5.5, actually seems to have more positive feedback. Personally, my experience with Codex wasn't as good as with Claude Code (Codex freezes on Windows m…

> Anyway, one thing for sure is that Gemini is pretty much unusable

Ha! I find that Gemini is quite useful - if only because I am forced to use it (on my personal projects) because it's the only one that has unlimited interaction for "free"

It has its limitations, yes, but so does Claude (which I am leaning on too heavily at work at the moment)

Re: Show HN: State of the Art of Coding Models, According to Hacker News Commenters

#20

Post by gobdovan »

Before harnesses, I'd fix the methodology/claims. A saner methodology would be to see comments that compare two models, say 'gpt5.5>opus4.7' and infer context ('ctx:frontend', for example). For your current methodology, 'opus 4.6 was very smart, opus4.7 is a disappointing upgrade to 4.6' would make normal aspect-based sentiment analysis consider 4.6 is smarter than 4.6. But considering you have <300 mentions total, probably you'd be better off scrapping some other websites as well. I'd also take out completely the SotA claim and downgrade the mentions to measuring something like visibility rather than performance.
Post reply on HN