Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

Post reply on HN
Print view View original

291–300 of 615 posts

Re: Things we learned about LLMs in 2024

#291

Post by theptip »

mvkel wrote: post #235
antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

I'm surprised at the description that it's "useless" as a programming / design partner. Even if it doesn't make "elegant" code (whatever that means), it's the difference between an app existing at all, or not. I built and shipped a Swift app to the App Store, currently generating $10,200 in MRR, exclusively using LLMs. I wouldn't describe myself as a programmer, and didn't plan to ever build an app, mostly because in…

The context here is super-important - the commenter is the author of Redis. So, a super-experienced and productive low-level programmer. It’s not surprising that Staff-plus experts find LLMs much less useful.

Though I’d be interested if this was an opinion on “help me write this gnarly C algorithm” or “help me to be productive in ” as I find a big productivity increase from the latter.

Re: Things we learned about LLMs in 2024

#292

Post by sureglymop »

pton_xd wrote: post #132
dxbydt wrote: post #92

Earlier quoted context omitted.

> best LLMs are able to accelerate you https://www2.math.upenn.edu/~ghrist/preprints/LAEF.pdf - this math textbook was written in just 55 days! Paraphrasing the acknowledgements - ...Begun November 4, 2024, published December 28, 2024. ...assisted by Claude 3.5 sonnet, trained on my previous books... ...puzzles co-created by the author and Claude ...GPT-4o and -o1 were useful in latex configurations...doing proof-rea…

"The story of linear algebra begins with systems of equations, each line describing a constraint or boundary traced upon abstract space. These simplest mathematical models of limitation — each equation binding variables in measured proportion — conjoin to shape the realm of possible solutions. When several such constraints act in concert, their collaboration yields three possible fates: no solution survives their col…

I agree. Not what I would expect from a math book or script.

Re: Things we learned about LLMs in 2024

#293

Post by wrs »

jpc0 wrote: post #213
pton_xd wrote: post #132

Earlier quoted context omitted.

"The story of linear algebra begins with systems of equations, each line describing a constraint or boundary traced upon abstract space. These simplest mathematical models of limitation — each equation binding variables in measured proportion — conjoin to shape the realm of possible solutions. When several such constraints act in concert, their collaboration yields three possible fates: no solution survives their col…

Even putting it here is annoying to me... Those are a lot of words saying nothing that I just spend time reading. I'm agreeing with you.

It's rather purple prose, but it's entirely meaningful. Maybe it doesn't seem to mean anything until after you know some linear algebra, though...

Re: Things we learned about LLMs in 2024

#294

Post by theptip »

dartos wrote: post #278
swalsh wrote: post #241

Earlier quoted context omitted.

It's amazing this is still an opinion in 2025. I now ask devs how they use AI as part of their workflows when I interview. It's a standard skill I expect my guys to have.

I feel bad for your team. Let people work how they want. I wouldn’t not hire someone on the basis of them not using a language server. The creator of the Odin language famously doesn’t use one. He’s says that he, specifically, is faster without one.

No, it’s reasonable. If your team uses Git then it’s a valid question to establish if someone has only worked with Perforce.

They didn’t say how heavily they weight the question.

(All that said I expect that, soon, experience with the appropriate LLM tooling will be as important as having experience with the language your system is implemented in.)

Re: Things we learned about LLMs in 2024

#295

Post by JambalayaJimbo »

theptip wrote: post #291
mvkel wrote: post #235

Earlier quoted context omitted.

I'm surprised at the description that it's "useless" as a programming / design partner. Even if it doesn't make "elegant" code (whatever that means), it's the difference between an app existing at all, or not. I built and shipped a Swift app to the App Store, currently generating $10,200 in MRR, exclusively using LLMs. I wouldn't describe myself as a programmer, and didn't plan to ever build an app, mostly because in…

The context here is super-important - the commenter is the author of Redis. So, a super-experienced and productive low-level programmer. It’s not surprising that Staff-plus experts find LLMs much less useful. Though I’d be interested if this was an opinion on “help me write this gnarly C algorithm” or “help me to be productive in ” as I find a big productivity increase from the latter.

Why would the author of Redis describe himself as “not a programmer”? That’s a little odd.

Re: Things we learned about LLMs in 2024

#296

Post by [deleted] »

bdangubic wrote: post #239
duped wrote: post #202

Earlier quoted context omitted.

> Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. It's not as helpful as Google was ten years ago. It's more helpful than Google today, because Google search has slowly been corrupted by garbage SEO and other LLM spam, including their own suggestions.

comparing google to claude 3.5 is like comparing tesla s plaid with a horse

[deleted]

Re: Things we learned about LLMs in 2024

#298

Post by harrisi »

JambalayaJimbo wrote: post #295
theptip wrote: post #291

Earlier quoted context omitted.

The context here is super-important - the commenter is the author of Redis. So, a super-experienced and productive low-level programmer. It’s not surprising that Staff-plus experts find LLMs much less useful. Though I’d be interested if this was an opinion on “help me write this gnarly C algorithm” or “help me to be productive in ” as I find a big productivity increase from the latter.

Why would the author of Redis describe himself as “not a programmer”? That’s a little odd.

They didn't.

EDIT: antirez is the creator of redis, not mvkel.

Re: Things we learned about LLMs in 2024

#300

Post by sramam »

swalsh wrote: post #241
dartos wrote: post #198

> There’s a flipside to this too: a lot of better informed people have sworn off LLMs entirely because they can’t see how anyone could benefit from a tool with so many flaws. The key skill in getting the most out of LLMs is learning to work with tech that is both inherently unreliable and incredibly powerful at the same time. This is a decidedly non-obvious skill to acquire! I wish the author qualified this more. How…

It's amazing this is still an opinion in 2025. I now ask devs how they use AI as part of their workflows when I interview. It's a standard skill I expect my guys to have.

I concur that asking devs how they use AI is a great idea.

Recently, I shared a code base with a junior dev and she was surprised with the speed and sophistication of the code. The LLM did 80+% of the "coding".

What was telling was as she was grokking the code (for helping the ~20%), she was surprised at the quality of the code - her use of the LLM did not yield code of similar quality.

I find that the more domain awareness one brings to the table, the better the output is. Basically the clearer one's vision of the end-state, the better the output.

One other positive side-effect of using "LLMs as a junior-dev" for me has been that my ambitions are greater. I want it all - better code, more sophisticated capabilities even for relatively not-important projects, documentation, tests, debug-ability. And once the basic structure is in place, many a time it is trivial to get the rest.

It's never 100%, but even with 80+%, I am faster than ever before, deliver better quality code, and can switch domains multiple times a week and never feel drained.

Sharing best AI hacks within a team will have the same effect as code-reviews do in ensuring consistency. Perhaps an "LLM chat review", especially when something particularly novel was accomplished!

Post reply on HN