Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

241–250 of 631 posts

Re: The last six months in LLMs in five minutes

#241

Earlier quoted context omitted.

Even more worrying are the implications for fakenews, propaganda, fraud, deception and mental health.

Maybe short term yes. But longer term people will finally put their guard up against deception that’s been around for decades.

At that point their ground truth is completely skewed (already for some folk), everything is relative. Some of them will probably die off in self-induced Darwin award winning ways, but sadly certain skewed world views may persist.

Re: The last six months in LLMs in five minutes

#242
It definitely seems like the point of no return has been passed.

The size of the codebase doesn't matter anymore. In fact, I am finding that the larger the codebase the better the performance. Starting from scratch with vague ambition is not the same as solving a specific stack trace over a mountain of decade-old code. The later performs better and is also more exciting for the business. It would seem more callers = more constraints to verify against.

For the last 3 months I've felt like I've been dropping gps guided bombs from orbit. No one can tell the difference between AI authored and my hand written code, other than via the implication of the radically increased daily work volume. There's definitely AI in there, but it's like a homogeneous cybernetic blend of my work and the computer's. I own all of it, can explain all of it immediately, but I only wrote maybe 10% of it by hand.

The development team should be mostly "solved" by now with regard to the AI transformation. If you are still at Home Depot picking out your proverbial hammer, it's time to start heading for the self checkout. The rest of the business is where the real money and headlines will be made at this point. AI writing code is ancient news now. Custom harnesses that business people can use to automate workflows will print a lot more money. Bringing some bacon to the rest of the business may also help to preserve your career path in these uncertain times.

Remember what Jobs said about the customer. A lot of times, people don’t know what they want until you show it to them. Most people wouldn't have believed the iPhone was even remotely possible until the moment it was publicly revealed and made available for purchase. I am finding the same effect in the business with AI. What it can actually do when well engineered and applied to the domain will usually outperform the expectations of its users by a wide margin. All these fears about alignment, hallucinations, cost, ethics, the environment, my ego/career, etc., seem to melt away like some kind of luxurious chocolate once the performance becomes clear to the executive staff. I was able to convince the board with an unsolicited, 5 minute demo I didn't even personally deliver. I've never seen these people sign contracts so quickly.

Re: The last six months in LLMs in five minutes

#243

Earlier quoted context omitted.

I've "vibed" some non-trivial stuff lately using a combination of Codex with 5.5 and Claude Code with Opus 4.7. Key has been to spend a fair amount of time on initial overall design document, which is split into tangible and limited phases. I go back and forth between them on this document until we're all happy. For each phase an implementation plan is made. At the end, a summary document of what was delivered and wh…

Do you use anything to orcheatrate multiple agent pitted against each other (coder, reviewer, tester, etc)?

Currently just manual. I'm not pushing the frontier here, just getting my feet wet.

While both Claude Code and Codex are capable harnesses, I definitely think there's a lot more to be gained from the harnesses. Quite a few of the times I needed to nudge the steering wheel it was things that a separate agent with the right prompt could have picked up on.

Re: The last six months in LLMs in five minutes

#244
post #50

Earlier quoted context omitted.

Because the tool will happily give you a "solution" that kinda works for a few inputs. It will happily correct itself when you give it more incorrect tests. It will almost never converge on the general solution that will pass tests you haven't given it yet. This is why AI is sooo good at Javascript and related slop. A solution that "kinda works" is good enough 9 times out of 10 and if some tests fail well ... YOLO an…

To be fair, take away a human's paren highlighting and see how well they do.

[deleted]

Re: The last six months in LLMs in five minutes

#245
post #155

Earlier quoted context omitted.

I've "vibed" some non-trivial stuff lately using a combination of Codex with 5.5 and Claude Code with Opus 4.7. Key has been to spend a fair amount of time on initial overall design document, which is split into tangible and limited phases. I go back and forth between them on this document until we're all happy. For each phase an implementation plan is made. At the end, a summary document of what was delivered and wh…

That’s not vibing, but waterfall development.

It's vibing in the sense that I'm not really writing code, and I'm leaving a lot of decision to the models. I let them drive a lot of the design document details, I just made sure it contained the salient points. Implementation plans I just skimmed. Didn't write any code, just did some checks here and there.

But yes, I did think that it sorta felt like being a team lead for some eager programmers.

Re: The last six months in LLMs in five minutes

#246

Earlier quoted context omitted.

The polarization comes from the very disparate coding experiences and output quality that different people find when using these tools. For example, I've had the opposite experience of yours, generating very high quality work using Claude (such as https://github.com/kstenerud/yoloai ). Just in dealing with all the bugs and idiosyncrasies in the technologies I'm using, the agent has been a godsend in discovering and c…

Don't want to be rough, but I'd like to read experiences about novelty ideas that solve people real problems in the real world, your project it's just about selling new shovels. As I commented on another thread > If you're trying to solve a HARD problem people REALLY have, it's a novelty that agents can't help with, otherwise if it gets 97% there MAYBE it's just a signal that your idea isn't that novel!

Claude wrote me a little python script to help me sort and rank all the AI videos I've generated. It also extracted the metadata and organized it into a CSV. I sent it some hex dumps of the header and it got it first try. The header structure of webms generated by comfy are pretty novel.

Re: The last six months in LLMs in five minutes

#247

Earlier quoted context omitted.

I've "vibed" some non-trivial stuff lately using a combination of Codex with 5.5 and Claude Code with Opus 4.7. Key has been to spend a fair amount of time on initial overall design document, which is split into tangible and limited phases. I go back and forth between them on this document until we're all happy. For each phase an implementation plan is made. At the end, a summary document of what was delivered and wh…

None of it is non-trivial tho. You might think so, but it’s not.

It wasn't trivial in that I used a lot of my programming and domain knowledge, both when iterating on the design document and skimming implementation plans.

I didn't use it often, but when it was needed it was needed.

Re: The last six months in LLMs in five minutes

#248

> The coding agents got really good It's since november 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". All I observe they got better at tool call and answering questions about big codebases, especially if the question has a vague pattern to search, and they're superuseful for that! For generating production code even with a lot of steering and baby sitting…

The polarization comes from the very disparate coding experiences and output quality that different people find when using these tools. For example, I've had the opposite experience of yours, generating very high quality work using Claude (such as https://github.com/kstenerud/yoloai ). Just in dealing with all the bugs and idiosyncrasies in the technologies I'm using, the agent has been a godsend in discovering and c…

I don't want to offend (it's AI coded anyway :)) but that does not scream "high quality" to me. The headline gif on that repo just paints a terrible picture. It can't draw a box correctly, there's random underscores all over the screen. The UI itself is just incredibly incoherent. I don't even know what I'm looking at.

Like, no it doesn't seem like very high quality work... It just seems like a vibe coded tool.

Edit: yes it's wrapping Claude. It's BREAKING the TUI. Not sure what people aren't getting here...

Re: The last six months in LLMs in five minutes

#249

> and there’s zero chance any AI lab would train a model for such a ridiculous task. I'm not sure that's true anymore considering how popular Simon's blog is

It's practically a benchmark now. Some friends have been specifically training models to count the R's in "strawberry"

Re: The last six months in LLMs in five minutes

#250

Earlier quoted context omitted.

Don't want to be rough, but I'd like to read experiences about novelty ideas that solve people real problems in the real world, your project it's just about selling new shovels. As I commented on another thread > If you're trying to solve a HARD problem people REALLY have, it's a novelty that agents can't help with, otherwise if it gets 97% there MAYBE it's just a signal that your idea isn't that novel!

This is a pretty wild take. What percentage of human engineers are creating novel solutions for hard problems, you think? I work in R&D and even my work is 90% doing things that other people already solved. If you are really doing cutting edge SOTA work that has never been done by another human in some form or another, then kudos to you and I want your job.

> What percentage of human engineers are creating novel solutions for hard problems, you think?

IMO Every engineer should try spending his time in a company that tries to solve new problems.

Otherwise we will be stuck, as we are now, with big tech paying you mountains of money for doing nothing, incentivizing you to embark on useless activities for letting other managers have a career, fear layoffs and when that happen complaining about it because "it's a year i'm looking for a new job" pretending same compensation and environment. Web development jobs are particularly affected by that.

In the game industry, for example, if you don't do something interesting your game won't sell a copy.

Let me stress this out again, if LLMs get you 97% there, maybe you should try another idea.

Post reply on HN