If you're a vulnerability researcher or a security person generally, there's a big inflection point from Spring of this year.
The last six months in LLMs in five minutes
41–50 of 631 posts
Re: The last six months in LLMs in five minutes
#42> and there’s zero chance any AI lab would train a model for such a ridiculous task. I'm not sure that's true anymore considering how popular Simon's blog is
> So maybe the AI labs have been paying attention after all! > I think this mainly demonstrates that the pelican on the bicycle has firmly exceeded its limits as a useful benchmark. As acknowledged in the article.
Gemini 3.1 basically takes it home on that benchmark, anyway, it's done.
Re: The last six months in LLMs in five minutes
#43Haven’t noticed much significant progress in LLMs myself in 6 months (significant as in new or vastly improved capabilities or understanding, not new releases, there are plenty of those).
I feel like if anything people started to realise the significant limitations of LLMs when you try to use them as ‘agents’ which was the big direction LLM companies tried to push recently.
Best use of LLMs so far IMO is finding vulnerabilities (with human help) and pattern matching in other domains. For generating code and prose they are still mediocre and somewhat unreliable and for use as personal assistant agents I wouldn’t trust them.
So what’s happening with openclaw, the biggest experiment in agentic, vibe coded by the agents themselves? The thing that was so hot a few months ago.
https://github.com/openclaw/openclaw/pulse?period=daily
279 commits to main from 77 authors in the last 24 hours.
Why is there so much churn and how could you trust it with your data? This is changes in ONE day!
If these are useful changes, surely it’d be superhuman by now given months of this pace.
What are people using this for?
Re: The last six months in LLMs in five minutes
#44I wonder how much the 'inflection point' is a thing vs marketing. I'm sure the models got somewhat better, but even now when I'm trying to 'vibe code' a game with the latest models (combination of Codex w/ gpt5.5 and gpt5.3-codex), they really do struggle. They definitely get something barebones up and running, but it's far from a fully fledged application.
I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.
How do you justify your salary given that you're just using a tool that any of us could use for $20 an hour in your role?
Re: The last six months in LLMs in five minutes
#45Somewhere right now some human artist is being tasked with drawing illustrations of pelicans riding bicycles to be used as training data at a big AI lab.
The quality of the Gemini pelican was such a step change in one iteration, while the other benchmarks remained quite flat, that I think you are right. Although whether they targeted Pelicans in particular or just svg, I can't say.
Re: The last six months in LLMs in five minutes
#46Earlier quoted context omitted.
I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.
How do you justify your salary given that you're just using a tool that any of us could use for $20 an hour in your role?
Someone competent using them is today a requirement and for awhile will make the marginal utility of skilled workers greater than that of unskilled. The justification is that they are much more productive than they were before.
Re: The last six months in LLMs in five minutes
#47Is the RLVR the key breakthrough for the uplift or is there more to it? Does that suggest the uplift was only for things that are easily verifiable like code?
I would say that most improvements are in easily verifiable things like code or math. Atleast that's where all the amazing results seem to be coming from.
Other domains I am not sure but I've heard from people like Cal Newport that the rate of increase outside of code and math are not as equally impressive
Re: The last six months in LLMs in five minutes
#48December 2025 was the breakthrough for me. January Claude was euphoric, ChatGPT was up there. February Gemini cooked for a second there. March amazing. April the big bad nerf. May GPT 5.5 is just pure bliss altough 2x limits temporarily, not sure about Claude it's sort of okay still not as good as it felt before, slowly increasing limits with more compute and rebuilding good will.
I find your emotional language truly quite fascinating. I've heard people talk like that about drugs.
Re: The last six months in LLMs in five minutes
#49what are your thoughts on Software engineer replacement. My team has already seen big reductions. Q/A team is gone. Software Engineer reduced by a third.
Scared for the future
Re: The last six months in LLMs in five minutes
#50Earlier quoted context omitted.
I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.
How do you justify your salary given that you're just using a tool that any of us could use for $20 an hour in your role?
Because the tool will happily give you a "solution" that kinda works for a few inputs. It will happily correct itself when you give it more incorrect tests.
It will almost never converge on the general solution that will pass tests you haven't given it yet.
This is why AI is sooo good at Javascript and related slop. A solution that "kinda works" is good enough 9 times out of 10 and if some tests fail well ... YOLO and the web page will probably render anyway.
Contrast that to using Scheme or Lisp where AI will have trouble simply keeping the parentheses balanced.