Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

Post reply on HN
Print view View original

201–210 of 615 posts

Re: Things we learned about LLMs in 2024

#201

Post by qsort »

antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

I believe it's more frustration directed at the mismatch between marketing and reality, combined with the general well deserved growing hatred for SV culture, and, more broadly, software engineers. The sentiment would be completely different if the entire industry marketed themselves like the helpful tools they are rather than the second coming of Christ they aren't. This distinction is hard to make on "fast food" forums like this one.

If you aren't a coder, it's hard to find much utility in "Google, but it burns a tree whenever you make an API call, and everything it tells you might be wrong". I for one have never used it for anything else. It just hasn't ever come up.

It's great at cheating on homework, kids love GPTs. It's great at cheating in general, in interviews for instance. Or at ruining Christmas, after this year's LLM debacle it's unclear if we'll have another edition of Advent of Code. None of this is the technology's fault, of course, you could say the same about the Internet, phones or what have you, but it's hardly a point in favor either.

And if you are a coder, models like Claude actually do help you, but you have to monitor their output and thoroughly test whatever comes out of them, a far cry from the promises of complete automation and insane productivity gains.

If you are only a consumer of this technology, like the vast majority of us here, there isn't that much of an upside in being an early adopter. I'll sit and wait, slowly integrating new technology in my workflow if and when it makes sense to do so.

Happy new year, I guess.

Re: Things we learned about LLMs in 2024

#202

Post by duped »

antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

> Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful.

It's not as helpful as Google was ten years ago. It's more helpful than Google today, because Google search has slowly been corrupted by garbage SEO and other LLM spam, including their own suggestions.

Re: Things we learned about LLMs in 2024

#203

Post by thinkingemote »

Animats wrote: post #10

> Some of those GPT-4 models run on my laptop That's an indication that most business-sized models won't need some giant data center. This is going to be a cheap technology most of the time. OpenAI is thus way overvalued.

Most of the laptops that the models can run on today are in the high end of dedicated bare metal servers. Most shared VM servers are way below these laptops. Most people buying a new laptop today won't be able to run them, most devs getting a website up with a server won't be able to run them.

This means that the definitions of "laptop" and "server" are dependent on use. We should instead talk about RAM, GPU and CPU speed which is more useful and informative but less engaging than "my laptop".

Re: Things we learned about LLMs in 2024

#204

Post by nyarlathotep_ »

torginus wrote: post #177
datadrivenangel wrote: post #33

Earlier quoted context omitted.

It's justified if AGI is possible. If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That is of course, assuming AGI is possible and exponential, and that marketshare goes to a single entity instead of a set of entities. Lots of big assumptions. Seems like we're heading towards a slow-lackluster singularity though.

I was thinking about how the economy has been actively makes less sense and gets divorced more and more from reality year after year, AI or not. It's the simple fact that the ability of assets to generate wealth has far outstripped the abiliy of individuals to earn money by working. Somehow real estate has become so expensive everywhere that owning a shitty apartment is impossible for the vast majority. When the worl…

> And the weird thing is it didn't use to be nearly this bad even a decade ago, and we can only guess how bad it will get in a decade, AGI or not.

The last 5 years have reflected a substantial decline in QOL in the states; you don't even have to to look back that far.

The coronacircus money-printing really accelerated the decline.

Re: Things we learned about LLMs in 2024

#205

Post by datadrivenangel »

viccis wrote: post #95

I didn't realize "agent" designs were that ambiguously defined. Every AI engineer I've talked to uses it to mean a design that combines several separate LLM prompts (or even models) to solve problems in multiple stages.

If the investors ask, those same AI engineers will probably allow the answer to be much more ambiguous.

Re: Things we learned about LLMs in 2024

#206

Post by alexashka »

I wonder what the author of this post thinks of human generated slop.

For example if someone just takes random information about a topic, organizes it in chronological order and adds empty opinions and preferences to it and does that for years on end - what do you call that?

Re: Things we learned about LLMs in 2024

#207

Post by jpc0 »

aetherson wrote: post #169
jsheard wrote: post #151

Earlier quoted context omitted.

I swear these goalposts keep getting moved, I remember being told that GPT3.5 is a useless toy but the paid GPT4 is lifechanging, and now that GPT4 is free I'm told that it's a useless toy but paid o1 or paid Sonnet are lifechanging. Looking forward to o1 and Sonnet becoming useless toys, unlike the lifechanging o3.

You will also be dismayed to hear that a 2011 iPhone is no longer state-of-the-art, and indeed can't run most modern apps.

GPT4 is a 13 year old technology? Compared to o1 and Sonnet 3.5?

If someone told me an iPhone 4 is terrible but an iPhone 5 would definitely serve my needs, then when I get an iPhone 5 they say the same of the 6 you really want me to believe them a second time? Then a third time? Then a 4th? In the mean time my time and money is wasted?

Re: Things we learned about LLMs in 2024

#208

Post by timewizard »

duped wrote: post #202
antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

> Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. It's not as helpful as Google was ten years ago. It's more helpful than Google today, because Google search has slowly been corrupted by garbage SEO and other LLM spam, including their own suggestions.

[dead]

Re: Things we learned about LLMs in 2024

#209

Post by dash2 »

Look, when are these models going to not just talk to me, but do stuff for me? If they're so clever, why can't I tell one to buy chocolates and send them to my wife? Meanwhile, they can allegedly solve frontier maths problems. What's the holdup to models that go online and perform simple tasks?

Re: Things we learned about LLMs in 2024

#210

Post by scubbo »

aetherson wrote: post #169
jsheard wrote: post #151

Earlier quoted context omitted.

I swear these goalposts keep getting moved, I remember being told that GPT3.5 is a useless toy but the paid GPT4 is lifechanging, and now that GPT4 is free I'm told that it's a useless toy but paid o1 or paid Sonnet are lifechanging. Looking forward to o1 and Sonnet becoming useless toys, unlike the lifechanging o3.

You will also be dismayed to hear that a 2011 iPhone is no longer state-of-the-art, and indeed can't run most modern apps.

Holy false-equivalency, Batman! The definitions of "useless toy / lifechanging tool" are _not_ changing over time (or, at least, not over the timescale being explored here), whereas the expectations and requirements of processing power of a phone are.
Post reply on HN