Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

Post reply on HN
Print view View original

91–100 of 631 posts

Re: The last six months in LLMs in five minutes

#91

Post by beng-nl »

angled wrote: post #70
LZ_Khan wrote: post #57

I'm curious how the 6 months have looked from a non-programmer's perspective. What kind of co-working tools and similar optimizations have people from other fields experienced?

In business: using coworking tools to review and propose filing of emails; manage my files and folders; on a daily basis scour the intranet for interesting and relevant content. Personal: my wife tutors in her native language to non-native primary and high school kids. They are all using these tools now generate fresh content for practice based on school lesson plans. The kids are improving much more quickly now than…

As someone who works somewhere where the intranet is a bit of a jungle: what tool do you use to scour the intranet?

Thanks!

Re: The last six months in LLMs in five minutes

#92

Post by magicalhippo »

Insanity wrote: post #10

I wonder how much the 'inflection point' is a thing vs marketing. I'm sure the models got somewhat better, but even now when I'm trying to 'vibe code' a game with the latest models (combination of Codex w/ gpt5.5 and gpt5.3-codex), they really do struggle. They definitely get something barebones up and running, but it's far from a fully fledged application.

I've "vibed" some non-trivial stuff lately using a combination of Codex with 5.5 and Claude Code with Opus 4.7.

Key has been to spend a fair amount of time on initial overall design document, which is split into tangible and limited phases. I go back and forth between them on this document until we're all happy.

For each phase an implementation plan is made. At the end, a summary document of what was delivered and what was discovered. This becomes input to next phase.

I do check the documents, and what they're doing. I also check the tests, some more thorough. And some spot checks on the code to see if I like the structure.

I have mainly used Claude for coding and Codex for design and code review after phases. I ask both to check test coverage after phases.

Managed to implement some tools and libraries without writing a single line of code this way, which have been very beneficial to us.

Since it's so async I can work on other stuff while they plod along.

I think it's not universal though. But stuff that can be tested easily and which you have a firm grasp of what you want to achieve, but not necessarily exactly how, that I've been impressed with.

Re: The last six months in LLMs in five minutes

#94

Post by MikeNotThePope »

viccis wrote: post #44
kvakkefly wrote: post #30

Earlier quoted context omitted.

I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.

How do you justify your salary given that you're just using a tool that any of us could use for $20 an hour in your role?

You can build things quickly with AI, but you can’t delegate your responsibilities to AI. Once the AI starts struggling, you’ll need to takeover and figure it out.

Re: The last six months in LLMs in five minutes

#95

Post by trojans1290 »

vanuatu wrote: post #82
vishal_new wrote: post #49

what are your thoughts on Software engineer replacement. My team has already seen big reductions. Q/A team is gone. Software Engineer reduced by a third. Scared for the future

I think there will be larger markets, more companies, more jobs than before due to AI, but also a very painful transition period AI reduces the cost of producing software (and other intellectual tasks), which greatly improves the viability for more and more ambitious projects. As far as we know the amount of problems software (and humanity) can solve is unbounded It feels like the market has shifted in SWE yet again…

What are these skills?

Re: The last six months in LLMs in five minutes

#96

Post by tptacek »

thierrydamiba wrote: post #62
tptacek wrote: post #41

If you're a vulnerability researcher or a security person generally, there's a big inflection point from Spring of this year.

Can you be more specific?

Broadly, I'm talking about the shift from building elaborate vulnerability research harnesses towards using the frontier models and their RL-optimized harnesses to build simpler vulnerability discovery pipelines. And then: the ensuing carnage.

Re: The last six months in LLMs in five minutes

#97

Post by 2001zhaozhao »

bunzee wrote: post #93

Spot on. Building our tool, we found AI is magic at scraping competitor data, but terrible at market validation. The 'why' is strictly human.

So, the best way to use LLMs is to wait for your competitors to do market validation and then scrape their data.

Hmmm......

Re: The last six months in LLMs in five minutes

#98

Post by jrowen »

There's something fitting about the mystical nature of LLMs and scrolling through a bunch of goofy pelicans on bicycles representing report cards for the bleeding edge of technology.

How are these even graded? Qwen3.6-35B-A3B gets high marks for a pelican with a gaping hole in its bill?

edit: Just noticed its feet are disconnected from its legs as well (but right on the pedals!). Pardon my French but that's Chinese af.

Re: The last six months in LLMs in five minutes

#99

Post by gnyman »

tptacek wrote: post #41

If you're a vulnerability researcher or a security person generally, there's a big inflection point from Spring of this year.

If it turns out to be a good change or not is to be seen.

The half-full view is that the models are so good at finding vulns that if you plug them into your build-pipeline then the amount of new vulns introduced will go down towards zero.

The half-empty view is that we're now producing more junior-level code with less review, so everything will have more vuln, also it's cheaper and easier to find them so prepare for chaos.

Short term there is sure to be chaos either way as the models are clearly good enough to find all the old bugs, and not everyone has the resources or will to try to stay ahead of the curve like Mozilla is trying to do with their Mythos access https://blog.mozilla.org/en/firefox/ai-security-zero-day-vul...

Re: The last six months in LLMs in five minutes

#100

Post by kenloef »

simonw wrote: post #80
vishal_new wrote: post #49

what are your thoughts on Software engineer replacement. My team has already seen big reductions. Q/A team is gone. Software Engineer reduced by a third. Scared for the future

Ditching the QA team when the single highest challenge is verifying that vibe-coded systems do what they're meant to is extraordinarily short-sighted. Personally, the more time I spend working with coding agents the least worried I am for my career. Getting the best results out of them is really hard . They amplify existing skills and experience, so the more experience you have the better.

I believe that many of those saying that they "never write code anymore" or are experiencing "10x productivity," are heavily underestimating (or outright misrepresenting) how much they are guiding the model, and ignoring everything else that goes into shipping fit for purpose software. I frequently see zero measurements or factual arguments supplied to support such claims. I also see many people say that they are "vibe coding," when they are almost certainly reviewing, editing, or otherwise steering the output.

I wonder why there is such a mad dash to trump up the capabilities of coding agents. And why such loose terminology and lack of rigor? I thought programmers were supposed to be rational people (har har!)

Post reply on HN