Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

Post reply on HN
Print view View original

251–260 of 631 posts

Re: The last six months in LLMs in five minutes

#251

Post by magicalhippo »

dawnerd wrote: post #176
WesolyKubeczek wrote: post #163

Earlier quoted context omitted.

> Key has been to spend a fair amount of time on initial overall design document, which is split into tangible and limited phases. > For each phase an implementation plan is made. At the end, a summary document of what was delivered and what was discovered. > I do check the documents, and what they're doing. I also check the tests, some more thorough. Sounds like programming, but with extra steps.

Also the least fun part of development. Maybe I’m the weird one but I like to just jump right in, planning every last detail before writing code is boring.

For me, the fun in programming is sometimes to actually write code, solving a problem in a specific way or try some new approach. Other times the fun is to create something that works, and the code is more a means to an end.

The first case I'll probably still do by hand, like handmade vases despite factory made are cheap and readily available.

For the second case I think these newfangled tools have made it even more fun, since writing lots of boiler plate, repetitive event handles and whatnot is not my idea of fun.

Re: The last six months in LLMs in five minutes

#252

Post by musebox35 »

Timwi wrote: post #103
musebox35 wrote: post #51

Earlier quoted context omitted.

Please see Ben Evans’ podcast on a good take on this. Coding is just one of the task you do in your job, it is not the job or at least it probably is not. You do not get paid to code, you get paid to make a set of decisions that create value to the company. If this is automated then yes sadly your salary is not justified.

> Coding is just one of the task[s] you do in your job But it's by far the most fun part and the only reason to take such a job...

I totally agree. I loved coding because of its closed feedback loop. Since last November, I also delegated it mostly to agents. Now I concentrate more on the design part, which is not the same. However, you move with the times and hope something else will become exciting. I do not know a more worthwhile and satisfying way than computing to spend my work hours.

Re: The last six months in LLMs in five minutes

#253

Post by viking123 »

tayo42 wrote: post #32

The claw thing really came and went fast lol

I mean yeah? It was marketing campaign to boost the model providers and give Steinberger a cozy job at OpenAI. Hook, line and sinker.

Wake me up when we have an agent with constant learning and changing weights that I can have personally, not some LLM that can always fall prone to jailbreak and context injection attacks.

You think most of this stuff here is organic? Oh boy..

Re: The last six months in LLMs in five minutes

#255

Post by OvervCW »

dmpk2k wrote: post #48
throwaway2027 wrote: post #8

December 2025 was the breakthrough for me. January Claude was euphoric, ChatGPT was up there. February Gemini cooked for a second there. March amazing. April the big bad nerf. May GPT 5.5 is just pure bliss altough 2x limits temporarily, not sure about Claude it's sort of okay still not as good as it felt before, slowly increasing limits with more compute and rebuilding good will.

I find your emotional language truly quite fascinating. I've heard people talk like that about drugs.

I actually thought it was a joke comment, but I'm worried now that it's not the case.

Re: The last six months in LLMs in five minutes

#256

Post by jmcodes »

hollowturtle wrote: post #228
treme wrote: post #220

Earlier quoted context omitted.

you are experiencing reverse Dunning–Kruger effect. For someone that just dabbled in coding prior, it went from AI building 80%, and struggling through to finish the 20% when trying to build an app/website. now it's like 97% and struggling with last 3%. Yes it'll look rough around the edges when evaulated by a senior dev, but being able to build MVP level things to completion with ease helps you stay engaged and moti…

Please do not cite Dunning–Kruger effect at random. Who needs to generate a dumb demo of a 97% done crud app? We had code generators for those, everytime I read claims like that and I ask to explain further I then discover it's people who were not productive before generating the so called "MVP level things to completion with ease". If you're trying to solve a HARD problem people REALLY have, it's a novelty that agen…

What would you consider a "hard" problem?

Re: The last six months in LLMs in five minutes

#257

Post by jiggawatts »

hollowturtle wrote: post #235
kstenerud wrote: post #219

Earlier quoted context omitted.

The polarization comes from the very disparate coding experiences and output quality that different people find when using these tools. For example, I've had the opposite experience of yours, generating very high quality work using Claude (such as https://github.com/kstenerud/yoloai ). Just in dealing with all the bugs and idiosyncrasies in the technologies I'm using, the agent has been a godsend in discovering and c…

Don't want to be rough, but I'd like to read experiences about novelty ideas that solve people real problems in the real world, your project it's just about selling new shovels. As I commented on another thread > If you're trying to solve a HARD problem people REALLY have, it's a novelty that agents can't help with, otherwise if it gets 97% there MAYBE it's just a signal that your idea isn't that novel!

As a random example of a "hard" problem solved by AI that I couldn't have realistically done myself, despite having decades of wide industry experience:

Reverse engineering a proprietary protocol from a binary executable.

I heard about people finding security vulnerabilities in compiled code with the combination of Claude Mythos wired up to a disassembler like NSA's Ghidra. Someone here mentioned that GPT 5.5 "extra high" is just as capable, I had a problem to solve, spare token quota for the week, so... I gave it a go.

My problem was that I'm working with a product that uses a legacy 1990s style network appliance output log format that is proprietary, undocumented, and has no publicly available decoders other than an app by the same vendor, and that app has fundamental limitations. (I.e.: it's nothing like Splunk or Elastic.)

Codex with a Ghidra MCP bridge figured it all out: the framing, bit and byte packing, endian order, field names, data types, etc. It made me a neat little protocol parser in a modern language that I can use to spit out something sane like NDJSON or OTLP protobufs.

There is no way I could have reverse engineered this myself from compiled C++ code and/or packet captures! The format isn't self-describing and is incredibly dense (similar to NetFlow). In a hex viewer it looks like line noise!

Re: The last six months in LLMs in five minutes

#258

Post by kstenerud »

hollowturtle wrote: post #235
kstenerud wrote: post #219

Earlier quoted context omitted.

The polarization comes from the very disparate coding experiences and output quality that different people find when using these tools. For example, I've had the opposite experience of yours, generating very high quality work using Claude (such as https://github.com/kstenerud/yoloai ). Just in dealing with all the bugs and idiosyncrasies in the technologies I'm using, the agent has been a godsend in discovering and c…

Don't want to be rough, but I'd like to read experiences about novelty ideas that solve people real problems in the real world, your project it's just about selling new shovels. As I commented on another thread > If you're trying to solve a HARD problem people REALLY have, it's a novelty that agents can't help with, otherwise if it gets 97% there MAYBE it's just a signal that your idea isn't that novel!

The comment was directed at:

> For generating production code even with a lot of steering and baby sitting? Absolutely not, not quite there not even close in my experience.

As I said, this is an example of using AI successfully to produce a high quality product (one that I use every day).

But to your point: I am solving hard problems that people really have. You just don't see those because I haven't mentioned them publicly yet. And they won't be released or talked about until they're ready.

Re: The last six months in LLMs in five minutes

#259

Post by opto »

bradley13 wrote: post #186
opto wrote: post #116

Earlier quoted context omitted.

I am an instructor who helps deliver an apprenticeship. My new boss has been in our industry for about 20 years and is one of the most respected people in our company. They've just joined us to teach and are off doing a two week course. On the first day she was told to let AI write all of her lesson plans, and then feed the lesson plans to AI to make her slides... Hopefully she rejects all this out of hand, but if sh…

I've been a teacher (most of the time a college professor) for...a long time. Nowadays, when preparing a new course, I definitely work with AI: "Here's what I want, and who my audience is - give me a course outline". That gives me a starting point. Of course, I modify it. Maybe I bounce back and forth to the AI for further refinements and suggestions, but ultimately I have to be happy with the result. When prepping t…

> AI is a tool. Use it appropriately

Yes, but no room is made for people who see no use for it. There is a forced-consensus that this technology is useful, which I have to combat against at work.

We teach in a very different environment, but your use sounds typical of my colleagues. "I ask it for suggestions and pick one", but nobody seems to wonder about what is lost when we shrink the horizon of what we will teach to the most likely outputs from a chatbot, one of which we will use.

Maybe this makes more sense in other fields. I have to prepare people to work in the shipping industry, in extremely dangerous roles where they will be operating heavy machinery, steering ships, driving cranes etc. The fact is that AI knows next to nothing about this field because an AI cannot experience handling a ship in rough weather, has never secured a boat to a ship's side with the rain and wind in its face.

Yet, when people are brought in to instruct our trainees, they are told to "tell AI what you want and pick one of the suggestions", in the best case, or just give over everything to the AI in the worst case. And nobody seems to be able to explain why this is a better way of working than sitting with a pen and paper, brainstorming some ideas for a lesson based on your real experiences, and then delivering it. The only justification I'm ever given is your one, "I pick from a list so I am really still in control", "it's quicker and I don't have to think as hard or as long", "it's better at making slides or writing good-sounding (to management and auditors) lesson plans". No-one ever seems to justify it by saying it is genuinely a better experience for the trainees.

Re: The last six months in LLMs in five minutes

#260

Post by AussieWog93 »

kvakkefly wrote: post #30
Insanity wrote: post #10

I wonder how much the 'inflection point' is a thing vs marketing. I'm sure the models got somewhat better, but even now when I'm trying to 'vibe code' a game with the latest models (combination of Codex w/ gpt5.5 and gpt5.3-codex), they really do struggle. They definitely get something barebones up and running, but it's far from a fully fledged application.

I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.

Exact same experience here. Prior to Opus 4.5 I'd sometimes use AI for some frontend webdev stuff (I am a C/C++/Python programmer; my HTML/CSS/JS knowledge is probably on par with a first-year uni student) and I'd have to manually edit things and retry, tell it not to attempt a paradigm that had failed before or cycle between models in Cursor just to try and get one that could make a simple widget that worked properly.

Now, I'm using Claude or Codex (GPT-5.5) for frontend and backend and it just gets it right first time more often than not. I've been making use of things like LSPs, Context7 and CLAUDE.md (global and per-repo) and it just stops doing the dumb LLM things that I hate.

Post reply on HN