Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

Post reply on HN
Print view View original

81–90 of 631 posts

Re: The last six months in LLMs in five minutes

#81

Post by baq »

minimaxir wrote: post #15
Insanity wrote: post #10

I wonder how much the 'inflection point' is a thing vs marketing. I'm sure the models got somewhat better, but even now when I'm trying to 'vibe code' a game with the latest models (combination of Codex w/ gpt5.5 and gpt5.3-codex), they really do struggle. They definitely get something barebones up and running, but it's far from a fully fledged application.

Opus 4.5 in November 2025 was legitimately, unironically an inflection point and is the sole reason for the current hysteria. GPT 5.5 is a significant improvement over GPT 5.4 but I wouldn't call it an inflection.

5.2 and the first codex model were step function changes in capability

Re: The last six months in LLMs in five minutes

#82

Post by vanuatu »

vishal_new wrote: post #49

what are your thoughts on Software engineer replacement. My team has already seen big reductions. Q/A team is gone. Software Engineer reduced by a third. Scared for the future

I think there will be larger markets, more companies, more jobs than before due to AI, but also a very painful transition period

AI reduces the cost of producing software (and other intellectual tasks), which greatly improves the viability for more and more ambitious projects. As far as we know the amount of problems software (and humanity) can solve is unbounded

It feels like the market has shifted in SWE yet again to heavily prioritize a new set of skills, of which those in the top quartile are desired more than ever

Re: The last six months in LLMs in five minutes

#83

Post by vanuatu »

ex-aws-dude wrote: post #31

Is the RLVR the key breakthrough for the uplift or is there more to it? Does that suggest the uplift was only for things that are easily verifiable like code?

Yes, with good RLVR at scale you can greatly improve performance especially on benchmarks

The hope was that good RLVR on relatively contrived datasets (like benchmarks) would be generalized to good software taste, which has somewhat succeeded but also the models fail in horrible ways still

And the hope beyond that is that good skills in fundamental problem solving tasks (coding, math) would generalize to tasks beyond math and code, which did happen but less so

Re: The last six months in LLMs in five minutes

#84

Post by tetha »

thierrydamiba wrote: post #62
tptacek wrote: post #41

If you're a vulnerability researcher or a security person generally, there's a big inflection point from Spring of this year.

Can you be more specific?

Three deterministic Linux LPEs in a week, an LPE in BSD in execve (of all things...), nginx vulnerabilities, one or two new gnarly supply chain attacks. Linus noting that the linux-security mailing list is getting flooded with duplicated, AI-driven reports of varying quality. There are pretty crazy keycloak vulnerabilities getting discovered.

We're most likely entering a year or two or rapid vulnerability discovery, patching, as well as reducing and minimalizing system footprints just to survive the onslaught of strange vulnerabilities from e.g. ancient and widely unused kernel modules.

Re: The last six months in LLMs in five minutes

#85

Post by wilg »

viccis wrote: post #44
kvakkefly wrote: post #30

Earlier quoted context omitted.

I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.

How do you justify your salary given that you're just using a tool that any of us could use for $20 an hour in your role?

They don't need to justify it!

Re: The last six months in LLMs in five minutes

#86

Post by krzyk »

simonw wrote: post #76
thierrydamiba wrote: post #62

Earlier quoted context omitted.

Can you be more specific?

The Claude Mythos / Project Glasswing thing is real: https://www.anthropic.com/glasswing I met a few people at PyCon this week who have been part of Glasswing (they're just starting to be allowed to talk about it) and it really does drive down the cost of finding vulnerabilities. I've been collecting notes on that here: https://simonwillison.net/tags/ai-security-research/

People in my company sounded underwhelmed by it. It usually was founding issues by not understanding deployment (or not being fed that info).

Re: The last six months in LLMs in five minutes

#87

Post by wilg »

dmpk2k wrote: post #48
throwaway2027 wrote: post #8

December 2025 was the breakthrough for me. January Claude was euphoric, ChatGPT was up there. February Gemini cooked for a second there. March amazing. April the big bad nerf. May GPT 5.5 is just pure bliss altough 2x limits temporarily, not sure about Claude it's sort of okay still not as good as it felt before, slowly increasing limits with more compute and rebuilding good will.

I find your emotional language truly quite fascinating. I've heard people talk like that about drugs.

Similarly, I've heard people talk like that about things that are not drugs.

Re: The last six months in LLMs in five minutes

#88

Post by krzyk »

kvakkefly wrote: post #30
Insanity wrote: post #10

I wonder how much the 'inflection point' is a thing vs marketing. I'm sure the models got somewhat better, but even now when I'm trying to 'vibe code' a game with the latest models (combination of Codex w/ gpt5.5 and gpt5.3-codex), they really do struggle. They definitely get something barebones up and running, but it's far from a fully fledged application.

I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.

It is sad. I like programming, if I couldn't do it and had to write text (which I do hate, I'm not a writer) it would be make quite a sad world.

Re: The last six months in LLMs in five minutes

#89

Post by gib444 »

Starting from zero today, how would someone quickly get upto speed with the latest and greatest AI tooling on an extremely limited budget?

Is the only choice to pay for the "max" plans?

Or just read so much about it that you bs your way through an interview and then use the company's resources?

Simon, I'm curious too how much you invest each month researching all the latest and great AI tech?

Re: The last six months in LLMs in five minutes

#90

Post by vanuatu »

LZ_Khan wrote: post #57

I'm curious how the 6 months have looked from a non-programmer's perspective. What kind of co-working tools and similar optimizations have people from other fields experienced?

I work at a company that deploys AI to enterprises

The average office worker is amazed at Copilot (not in the IDE - but the app bundled with Windows), and they mostly copy paste material into their enterprise provided ChatGPT / Gemini, and get tips from Facebook / Instagram on their top 5 best prompts for work productivity

Showing them agents that automate work at scale is a very magical experience

Post reply on HN