Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

Post reply on HN
Print view View original

411–420 of 615 posts

Re: Things we learned about LLMs in 2024

#411

Post by 8n4vidtmkvmk »

Ruxbin wrote: post #356
mvkel wrote: post #235

Earlier quoted context omitted.

I'm surprised at the description that it's "useless" as a programming / design partner. Even if it doesn't make "elegant" code (whatever that means), it's the difference between an app existing at all, or not. I built and shipped a Swift app to the App Store, currently generating $10,200 in MRR, exclusively using LLMs. I wouldn't describe myself as a programmer, and didn't plan to ever build an app, mostly because in…

May you expand how you did this? I'm seeing a number of apps that claim to do just this and there are number that are becoming super popular. Not just the development of the code but the entire the thing from the code, infra, auth, cc payments, etc.

For CC payments, just use Stripe. The docs are great!

Re: Things we learned about LLMs in 2024

#412

Post by mvdtnz »

dartos wrote: post #198

> There’s a flipside to this too: a lot of better informed people have sworn off LLMs entirely because they can’t see how anyone could benefit from a tool with so many flaws. The key skill in getting the most out of LLMs is learning to work with tech that is both inherently unreliable and incredibly powerful at the same time. This is a decidedly non-obvious skill to acquire! I wish the author qualified this more. How…

It's really important to go and read the code that the author of this article actually produces with LLMs. He posted on hacker news a few months ago, a post called something like "everything I've made with ChatGPT in the month of September" or something. He's producing little toy applications that don't even begin to resemble real production code. He thinks these "tools" are useful because they help him write pointless slop.

Re: Things we learned about LLMs in 2024

#413

Post by fragmede »

jiggawatts wrote: post #270
FooBarWidget wrote: post #257

Earlier quoted context omitted.

Why do people have such narrow views on what makes LLMs useful? I use them for basically everything. My son throwing an irrational tantrum at the amusement park and I can't figure out why he's like that (he won't tell me or he doesn't know himself either) or what I should do? I feed Claude all the facts of what happened that day and ask for advice. Even if I don't agree with the advice, at the very least the analysis…

At the risk of sounding impolite or critical of your personal choices: this, right here, is the problem! You don’t understand how medicine works, at any level. Yet you turn to a machine for advice, and take it at face value . I say these things confidently, because I do understand medicine well enough to not to seek my own answers. Recently I went to a doctor for a serious condition and every notion I had was wrong.…

To be fair though, humanity doesn't know how some medicines work at a fundamental level either. The method of action for Tylenol, lithium, and metformin, among others isn't fully understood.

Re: Things we learned about LLMs in 2024

#414

Post by raincole »

jsheard wrote: post #151
antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

I swear these goalposts keep getting moved, I remember being told that GPT3.5 is a useless toy but the paid GPT4 is lifechanging, and now that GPT4 is free I'm told that it's a useless toy but paid o1 or paid Sonnet are lifechanging. Looking forward to o1 and Sonnet becoming useless toys, unlike the lifechanging o3.

Except GPT4 isn't free.

The GP is claiming GPT4o is bad but Sonnet is good. GPT4o is about only 20% cheaper than Sonnet.

Re: Things we learned about LLMs in 2024

#415

Post by askl56 »

simonw wrote: post #404
salawat wrote: post #402

Earlier quoted context omitted.

Why are you pasting huge chunks of potentially crown jewels code into a 3rd party service where prompts are going to most likely be turned into training/surveillance material?

A lot of vendors promise not to train on input to their models. I choose to believe those promises.

A scorpion, not knowing how to swim, asked a frog to carry it across the river. “Do I look like a fool?” said the frog. “You’d sting me if I let you on my back!”

“Be logical,” said the scorpion. “If I stung you I’d certainly drown myself.”

“That’s true,” the frog acknowledged. “Climb aboard, then!” But no sooner than they were halfway across the river, the scorpion stung the frog, and they both began to thrash and drown. “Why on earth did you do that?” the frog said morosely. “Now we’re both going to die.”

“I can’t help it,” said the scorpion. “It’s my nature.”

Re: Things we learned about LLMs in 2024

#416

Post by draw_down »

webmaven wrote: post #324
draw_down wrote: post #27

I agree the criticism is poor; it’s often very lazy. There are currently a lot of dog-brain “wrap a LLM around it” products, which are worthy of scorn. Much of the lazy criticism is pointing at such products and therefore writing off the whole endeavor. But that doesn’t necessarily reflect the potential of the underlying technology, which is developing rapidly. Websites were goofy and pointless until Amazon came arou…

This is HN. The canonical example for that is pg's Viaweb.

[dead]

Re: Things we learned about LLMs in 2024

#417

Post by dartos »

BeetleB wrote: post #391
dartos wrote: post #327

Earlier quoted context omitted.

> Don't use LLMs where accuracy is paramount. Then why do people keep pushing it for code related tasks? Accuracy and precision is paramount with code. It needs to express exactly what needs to be done and how.

> Then why do people keep pushing it for code related tasks? They don't. You are likely experiencing selection bias. My guess is you work in SW, and so it makes sense that you're the target of those campaigns. The bulk of ChatGPT subscribers are not doing SW, and no one is bugging them to use it for code related tasks.

I mean people in the software field absolutely push for LLMs to write code…

Obviously people not in the software field wouldn’t care…

Re: Things we learned about LLMs in 2024

#418

Post by saltcured »

fragmede wrote: post #382
qsort wrote: post #201

Earlier quoted context omitted.

I believe it's more frustration directed at the mismatch between marketing and reality, combined with the general well deserved growing hatred for SV culture, and, more broadly, software engineers. The sentiment would be completely different if the entire industry marketed themselves like the helpful tools they are rather than the second coming of Christ they aren't. This distinction is hard to make on "fast food" fo…

> there isn't that much of an upside in being an early adopter. Other than, y'know, using the new tools. As a programmer heavy forum, we focus a lot on LLMs' (lack of) correctness. There's more than a little bit of annoyance when things are wrong, like being asked to grab the red blanket and then getting into an argument over it being orange instead of what was important, someone needed the blanket because they were…

Yikes.

Clearly said, yet the general sentiment awakens in me a feeling more gothic horror than bright futurism. I am stuck with wonder and worry at the question of how rapidly this stuff will infiltrate into the global tech supply chain, and the eventual consequences of misguided trust.

To my eye, too much current AI and related tech are just exaggerated versions of magic 8-balls, Ouija boards, horoscopes, or Weizenbaum's ELIZA. The fundamental problem is people personifying these toys and letting their guard down. Human instincts take over and people effectively social engineer themselves, putting trust in plausible fictions.

It's not just LLMs though. It's been a long time coming, the way modern tech platforms have been exaggerating their capability with smoke and mirrors UX tricks, where a gleaming facade promises more reality and truth than it actually delivers. Individual users and user populations are left to soak up the errors and omissions and convince themselves everything is working as it should.

Someday, maybe, anthropologists will look back on us and recognize something like cargo cults. When we kept going through the motions of Search and Retrieval even though real information was no longer coming in for a landing.

Re: Things we learned about LLMs in 2024

#419

Post by zahlman »

antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

>They are also great at doing boring tasks for which you can provide perfect guidance (but that still would take you hours)

All the tasks I can think of dealing with on my own computer that would take hours, a) are actually pretty interesting to me and b) would equally well take hours to "provide perfect guidance". The drudge work of programming that I notice comes in blocks of seconds at a time, and the mental context switch to using an LLM would be costlier.

Re: Things we learned about LLMs in 2024

#420

Post by yogrish »

mvkel wrote: post #235
antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

I'm surprised at the description that it's "useless" as a programming / design partner. Even if it doesn't make "elegant" code (whatever that means), it's the difference between an app existing at all, or not. I built and shipped a Swift app to the App Store, currently generating $10,200 in MRR, exclusively using LLMs. I wouldn't describe myself as a programmer, and didn't plan to ever build an app, mostly because in…

May I know what is the name of app that is built using LLM? 10k MRR is highly successful app.
Post reply on HN