Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

Post reply on HN
Print view View original

271–280 of 631 posts

Re: The last six months in LLMs in five minutes

#271

Post by hollowturtle »

jmcodes wrote: post #256
hollowturtle wrote: post #228

Earlier quoted context omitted.

Please do not cite Dunning–Kruger effect at random. Who needs to generate a dumb demo of a 97% done crud app? We had code generators for those, everytime I read claims like that and I ask to explain further I then discover it's people who were not productive before generating the so called "MVP level things to completion with ease". If you're trying to solve a HARD problem people REALLY have, it's a novelty that agen…

What would you consider a "hard" problem?

I don't know how to define hard problems.

All I know is that we have a gigantic amount of tech debt we accumulated on the web chasing the next web framework built on top of tons of abstractions with very disappointing native web apis that shouldn't be taken seriously nor the w3c who specified them.

And when an Agent it's capable of gluing together a web app with some crud backend with a very rounded corners UI, that solves nothing for end users, we call them capable. These are not hard problems

Re: The last six months in LLMs in five minutes

#272

Post by hansmayer »

TL;DR:

"Coding agents got really good - here, a bunch of non-releavant slop-pictures of pelicans riding bikes as a key benchmark AND a couple of hardly relevant edge-case demo-projects of mine to prove it right! "

Come on man, where is the AI writing all the code in 6 months? We're close to June and Amodei's latest statement from January does not look like going into fulfilling over the next weeks, does it now?

Re: The last six months in LLMs in five minutes

#273

Post by spacebanana7 »

muvlon wrote: post #165
gnyman wrote: post #99

Earlier quoted context omitted.

If it turns out to be a good change or not is to be seen. The half-full view is that the models are so good at finding vulns that if you plug them into your build-pipeline then the amount of new vulns introduced will go down towards zero. The half-empty view is that we're now producing more junior-level code with less review, so everything will have more vuln, also it's cheaper and easier to find them so prepare for…

There's a major caveat to the half-full view: You'll only stop adding new vulns that your model can find. A threat actor with access to a better model or more money to burn on tokens may yet find more. Some of them have deep pockets, and not nearly every project will get the Glasswing treatment of free Mythos tokens.

There's an interesting economic contest here as well - is it more sustainable for a malware group to spend $500 in tokens looking for an issue in my app? or for me to spend $500 scanning for issues on every deployment?

Systemically this usually favours the offence, as they could scan my app once every 6 months whereas I'd need to do it on weekly releases.

Re: The last six months in LLMs in five minutes

#274

Post by musebox35 »

BOOSTERHIDROGEN wrote: post #117
musebox35 wrote: post #51

Earlier quoted context omitted.

Please see Ben Evans’ podcast on a good take on this. Coding is just one of the task you do in your job, it is not the job or at least it probably is not. You do not get paid to code, you get paid to make a set of decisions that create value to the company. If this is automated then yes sadly your salary is not justified.

Which episode ?

I watched the last one S5:E17 What jobs are AI jobs and I think it gives the right framing to think about this. It is not prescriptive, it does not give a list which is smart. The job title might be the same but the actual role might have different context so the best is to have the right frame to explore your particular situation.

Re: The last six months in LLMs in five minutes

#275

Post by hollowturtle »

AussieWog93 wrote: post #267
hollowturtle wrote: post #228

Earlier quoted context omitted.

Please do not cite Dunning–Kruger effect at random. Who needs to generate a dumb demo of a 97% done crud app? We had code generators for those, everytime I read claims like that and I ask to explain further I then discover it's people who were not productive before generating the so called "MVP level things to completion with ease". If you're trying to solve a HARD problem people REALLY have, it's a novelty that agen…

I don't really see your point. Most problems that people have aren't really super-novel, but just extremely bespoke. To give a specific example, 12 months ago I had a client pay me me to make a Chrome plugin that changed the rows in his Shopify Products page to display Quantity and SKU. These days you'd just one-shot it in Claude.

First of all it just underlines how shitty the web has become, second If that's your work I'd chase a career path where Claude can't one-shot this kind of dumb stuff

Re: The last six months in LLMs in five minutes

#276

Post by wickedsight »

kstenerud wrote: post #219
hollowturtle wrote: post #209

> The coding agents got really good It's since november 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". All I observe they got better at tool call and answering questions about big codebases, especially if the question has a vague pattern to search, and they're superuseful for that! For generating production code even with a lot of steering and baby sitting…

The polarization comes from the very disparate coding experiences and output quality that different people find when using these tools. For example, I've had the opposite experience of yours, generating very high quality work using Claude (such as https://github.com/kstenerud/yoloai ). Just in dealing with all the bugs and idiosyncrasies in the technologies I'm using, the agent has been a godsend in discovering and c…

> The polarization comes from the very disparate coding experiences and output quality that different people find when using these tools.

Not just when using tools, also when using humans. The frame of reference of what is considered 'production code' differs immensely between organizations, teams and people. The code I get from LLM's is usually much better than what I get from my peers. Maybe not one shot, but after some steering it gets there.

It also isn't lazy. When generating test cases for relatively simple pieces of code, it usually tests pretty much every path and doesn't stop right at the 80% code coverage quality gate.

I can imagine if you're at the level of Linus or something, you might conclude differently, but most people aren't there at all.

Re: The last six months in LLMs in five minutes

#277

Post by hexasquid »

jmcodes wrote: post #256
hollowturtle wrote: post #228

Earlier quoted context omitted.

Please do not cite Dunning–Kruger effect at random. Who needs to generate a dumb demo of a 97% done crud app? We had code generators for those, everytime I read claims like that and I ask to explain further I then discover it's people who were not productive before generating the so called "MVP level things to completion with ease". If you're trying to solve a HARD problem people REALLY have, it's a novelty that agen…

What would you consider a "hard" problem?

Extrapolating the final position of the goalposts

Re: The last six months in LLMs in five minutes

#278

Post by walthamstow »

Philip-J-Fry wrote: post #248
kstenerud wrote: post #219

Earlier quoted context omitted.

The polarization comes from the very disparate coding experiences and output quality that different people find when using these tools. For example, I've had the opposite experience of yours, generating very high quality work using Claude (such as https://github.com/kstenerud/yoloai ). Just in dealing with all the bugs and idiosyncrasies in the technologies I'm using, the agent has been a godsend in discovering and c…

I don't want to offend (it's AI coded anyway :)) but that does not scream "high quality" to me. The headline gif on that repo just paints a terrible picture. It can't draw a box correctly, there's random underscores all over the screen. The UI itself is just incredibly incoherent. I don't even know what I'm looking at. Like, no it doesn't seem like very high quality work... It just seems like a vibe coded tool. Edit:…

Take it up with Anthropic. It's actually their billion-dollar TUI product you're commenting on.

The problem with being such a naysayer is that you're entirely disconnected from what's going on. You haven't tried an agent like Claude Code and experienced it for yourself, so you don't recognise what it looks like when it's in front of you.

Re: The last six months in LLMs in five minutes

#279

Post by grey-area »

AussieWog93 wrote: post #268
grey-area wrote: post #173

Earlier quoted context omitted.

That’s really impressive, and slightly worrying for creatives involved in film, animation or modelling.

I wouldn't be that concerned that animation is going anywhere. Both outputs look really off, especially around the feet.

Yes sure if you look closely it’s slop, but a huge number of companies and advertisers just don’t care (and they feel the same about their social media content, blogs and yes code) - they will attempt to cut corners where they can to the detriment of true artists.

But yes, for anyone who does this for a living there will be obvious deficiencies, esp when you try to do something truly novel, intentional and interesting and don’t quite want what it produces.

But in this area they have made quite a lot of progress.

Re: The last six months in LLMs in five minutes

#280

Post by AussieWog93 »

hollowturtle wrote: post #275
AussieWog93 wrote: post #267

Earlier quoted context omitted.

I don't really see your point. Most problems that people have aren't really super-novel, but just extremely bespoke. To give a specific example, 12 months ago I had a client pay me me to make a Chrome plugin that changed the rows in his Shopify Products page to display Quantity and SKU. These days you'd just one-shot it in Claude.

First of all it just underlines how shitty the web has become, second If that's your work I'd chase a career path where Claude can't one-shot this kind of dumb stuff

It's not my work. I'm not even a full time dev any more.

But the client's problem was solved, and they're happy.

This is a genuinely useful thing. You don't need to shit all over it.

Post reply on HN