Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

301–310 of 615 posts

Re: Things we learned about LLMs in 2024

#301
post #241

Earlier quoted context omitted.

It's amazing this is still an opinion in 2025. I now ask devs how they use AI as part of their workflows when I interview. It's a standard skill I expect my guys to have.

Just curious, but what AI related skills do you expect them to have?

The ability to recognize and join a hype train, I presume. It’s one way to appear proactively leading-edge to marginally-informed product managers, marketers, execs and press.

Re: Things we learned about LLMs in 2024

#302
post #264

Earlier quoted context omitted.

Claude Sonnet 3.5 can write whole React applications with proper contextual clues and some minor iterations. Google has never coded for you. I’ve written two large applications and about a dozen smaller ones using Claude as an assistant. I’m a terrible front-end developer and almost none of that work was possible without Claude. The API and AWS deployment were sped up tremendously. I’ve created unit tests and I’ve re…

I've never really used Claude for writing code, becuase I'm not really bottlenecked by that problem. I have used it quite a bit for asking questions about what code to write and it's almost always wrong (usually in subtle ways that would trick someone with little experience). Maybe it was overtrained on react sources, but for me it's pretty useless. The big annoyance for me is it just makes up APIs that don't exist.…

They can't think at all. The task must be strict macroexpansion of original input(doesn't mean that always works).

I'm suspecting LLM works for a lot of front end and app coding just because code in those fields are insanely overbloated and value proposition is almost disconnected from logic. There must be metric tons of typing in those fields, and in those areas LLMs must be useful. They certainly handle paper test questions well.

Re: Things we learned about LLMs in 2024

#303
post #267

Earlier quoted context omitted.

To the un-sticking point: it's also great at letting people ask questions without being perceived as dumb Tragically - admitting ignorance, even with the desire to learn, often has negative social reprocussions

Asking "stupid" questions without fear of judgement is legit one of my favorite personal applications of LLMs.

That is one of the great strengths of LLMs for school education as well. Students often refrain from asking questions in class out of embarrassment at showing their ignorance or hesitation at interrupting the flow of the class. When used well, LLMs offer a good way for motivated learners to fill in the gaps in their understanding.

The pervasive problem of low student motivation won't be solved by LLMs, though. Human teachers will, I think, still be needed.

Re: Things we learned about LLMs in 2024

#304
post #111

Earlier quoted context omitted.

I agree, but I think my biggest issue with LLMs (and a lot of GenAI) is that they act as a massive accelerator for the WORST (and unfortunately most common) type of human - the lazy one. The signal-to-noise ratio just goes completely out of control. https://journal.everypixel.com/ai-image-statistics

Isn't it expected that most, if not all, of the content will be produced by AI/AGI in the near future? It won't matter much, if you're lazy or not. It leads to the question, what we'll do instead? People may want to be productive, but we're observing in real-time how world is going shit for workers and that's basically fact for many reasons. One reason is that it's cheaper to use AI, even if the result is poor. It do…

Why would this be expected?

Re: Things we learned about LLMs in 2024

#305
post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

To get the most out of them you have to provide context. Treat these models like some kind of eager beaver junior engineer who wants to jump in and write code without asking questions. Force it to ask questions (eg: “do not write code yet, please restate my requirements to make sure we are in alignment. Are there any extra bits of context or information that would help? I will tell you when to write code”) If your mo…

You are right, but doing all that is incredibly cumbersome, at least to some people, which is why they don’t like working with LLMs.

Re: Things we learned about LLMs in 2024

#307

Earlier quoted context omitted.

I swear these goalposts keep getting moved, I remember being told that GPT3.5 is a useless toy but the paid GPT4 is lifechanging, and now that GPT4 is free I'm told that it's a useless toy but paid o1 or paid Sonnet are lifechanging. Looking forward to o1 and Sonnet becoming useless toys, unlike the lifechanging o3.

You will also be dismayed to hear that a 2011 iPhone is no longer state-of-the-art, and indeed can't run most modern apps.

It would be quite useful if that were the only phone available.

Re: Things we learned about LLMs in 2024

#308
post #305

Earlier quoted context omitted.

To get the most out of them you have to provide context. Treat these models like some kind of eager beaver junior engineer who wants to jump in and write code without asking questions. Force it to ask questions (eg: “do not write code yet, please restate my requirements to make sure we are in alignment. Are there any extra bits of context or information that would help? I will tell you when to write code”) If your mo…

You are right, but doing all that is incredibly cumbersome, at least to some people, which is why they don’t like working with LLMs.

That was one of the themes of my article: LLMs are power-user tools, mis-sold as "easy to use". To get great results out of them you need to invest a whole lot of under-documented and under-appreciated effort. https://simonwillison.net/2024/Dec/31/llms-in-2024/#llms-som...

Re: Things we learned about LLMs in 2024

#309
post #235

Earlier quoted context omitted.

I'm surprised at the description that it's "useless" as a programming / design partner. Even if it doesn't make "elegant" code (whatever that means), it's the difference between an app existing at all, or not. I built and shipped a Swift app to the App Store, currently generating $10,200 in MRR, exclusively using LLMs. I wouldn't describe myself as a programmer, and didn't plan to ever build an app, mostly because in…

The context here is super-important - the commenter is the author of Redis. So, a super-experienced and productive low-level programmer. It’s not surprising that Staff-plus experts find LLMs much less useful. Though I’d be interested if this was an opinion on “help me write this gnarly C algorithm” or “help me to be productive in ” as I find a big productivity increase from the latter.

antirez is clearly going to be “Staff-plus” for almost any definition.

Can you clarify what you mean?

Re: Things we learned about LLMs in 2024

#310
post #202

Earlier quoted context omitted.

> Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. It's not as helpful as Google was ten years ago. It's more helpful than Google today, because Google search has slowly been corrupted by garbage SEO and other LLM spam, including their own suggestions.

Claude Sonnet 3.5 can write whole React applications with proper contextual clues and some minor iterations. Google has never coded for you. I’ve written two large applications and about a dozen smaller ones using Claude as an assistant. I’m a terrible front-end developer and almost none of that work was possible without Claude. The API and AWS deployment were sped up tremendously. I’ve created unit tests and I’ve re…

Imagine not needing Claude to do any of that.
Post reply on HN