Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

Post reply on HN
Print view View original

501–510 of 615 posts

Re: Things we learned about LLMs in 2024

#501

Post by BlueTemplar »

FooBarWidget wrote: post #450
tokai wrote: post #320

Earlier quoted context omitted.

This reads like satire to me. Scarry that it isn't.

I'm guessing that mindset is what cause some people to find this scary. I see a new tool and opportunities. Like all tools, it has drawbacks and caveats, but when wielded properly, it can give me more choice. I suspect some others focus too much on flaws and don't bother looking for opportunities. They are expecting a holy grail: if it's not perfect then it's useless. It's like people who proclaim that Linux as a who…

Grandma has a reason to care about you.

At the opposite, my trust of Russian / Chinese / USian platforms is low enough that I consider it my duty to publicly shame people that still use them in 2025.

(With some caveats of course, for instance HN is not a yet negative to the world. Yet.)

There's also the question of stickiness of habits : your grandmas are for life, human professionals you might have a shallow enough relationship with that switching them might be relatively easy, while it might be very hard to stop smoking or to stop using Github once you started smoking / create an account.

Re: Things we learned about LLMs in 2024

#502

Post by scrollaway »

Jensson wrote: post #476
beoberha wrote: post #449

Earlier quoted context omitted.

LLMs being able to detect bugs in my own code is absolutely mind blowing to me. These things are “just” predicting the next token, but somehow are able to take in code that has never been written before and somehow understand it and find what’s wrong with it. I think I’m more amazed by them because I know how they work. They shouldn’t be able to do this, but the fact that they can is absolutely jaw dropping science f…

Its easy to see how it does that, the answer is that your bug isn't something novel, it has seen millions of "where is the bug in this code" questions online so it can typically guess from there what it would be. It is very unreliable at fixing things or writing code for anything non standard. Knowing this you can easily construct queries that trips them up by noticing what it is in your code they notice, so you cons…

Both of your claims are way off the mark (I run an AI lab).

The LLMs are good at finding bugs in code not because they’ve been trained on questions that ask for existing bugs, but because they have built a world model in order to complete text more accurately. In this model, programming exists and has rules and the world model has learned that.

Which means that anything nonstandard … will be supported. It is trivial to showcase this: just base64 encode your prompts and see how the LLMs respond. It’s a good test because base64 is easy for LLMs to understand but still severely degrades the quality of reasoning and answers.

Re: Things we learned about LLMs in 2024

#503

Post by BlueTemplar »

dxbydt wrote: post #381
wrs wrote: post #293

Earlier quoted context omitted.

It's rather purple prose, but it's entirely meaningful. Maybe it doesn't seem to mean anything until after you know some linear algebra, though...

its been a long time, but when i was taught this material, i was told there are only 3 cases - x+y=1, x+y=2 clearly has no solution since two numbers can’t simultaneously add to both one and two. x+y=1,2x+2y=2 clearly has infinitely many solutions. There’s only one equation here after canceling the 2, so you can plug in x’s and y’s all day long, no end to it. x+y=1, 2x+y=1 clearly has exactly one solution (0,1) after…

Also, isn't this a great example of "when you have a hammer, everything looks like a nail" ?

To explain this I would first and foremost use a picture, where the 3 cases : parallel, identical, intersection can be intuitively seen (using our visual system, rather than our language system), with merely a glance.

Re: Things we learned about LLMs in 2024

#504

Post by aleph_minus_one »

antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

> About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful.

I see much deeper problems. Just to give two examples:

- I asked various AIs concerning explanations of proofs of some deep (established) mathematical theorems: the explanations were to my understanding very hallucinated, and thus worse than "obviously wrong". I also asked for literature references for some deep mathematical theory frameworks: bascially all of the references were again hallucinated.

- I asked lots of AIs on https://lmarena.ai/ to write a suitably long text about some political topic that is quite controversial in my country (but does have lots proponents even in a very radical formulation, even though most people would not use such a radical formulation in public). All of the LLMs that I checked refused or tried to indoctrinate me that this thesis is wrong. I did not ask the LLM to lecture me, but I gave it a concrete task! Society is deeply divided, so if the LLM only spreads propaganda of its political teaching, it will be useless for many tasks for a very significant share of the society.

Re: Things we learned about LLMs in 2024

#505

Post by Jensson »

scrollaway wrote: post #502
Jensson wrote: post #476

Earlier quoted context omitted.

Its easy to see how it does that, the answer is that your bug isn't something novel, it has seen millions of "where is the bug in this code" questions online so it can typically guess from there what it would be. It is very unreliable at fixing things or writing code for anything non standard. Knowing this you can easily construct queries that trips them up by noticing what it is in your code they notice, so you cons…

Both of your claims are way off the mark (I run an AI lab). The LLMs are good at finding bugs in code not because they’ve been trained on questions that ask for existing bugs, but because they have built a world model in order to complete text more accurately. In this model, programming exists and has rules and the world model has learned that. Which means that anything nonstandard … will be supported. It is trivial…

> just base64 encode your prompts and see how the LLMs respond

This is done via translations, LLM are good at translations, being able to translate doesn't mean you understand the subject.

And no I am not wrong here, I've tested this before, for example if you ask if a CPU model is faster than a GPU model it will say the GPU model is faster, even if the CPU is much more modern and faster overall since it learned that GPU names are faster than CPU names it didn't really understood what faster meant there. Exactly what the LLM gets wrong depends on the LLM of course, and the larger it is the more fine grained these things are but in general it doesn't really have much that can be called understanding.

If you don't understand how to break the LLM like this then you don't really understand what the LLM is capable of, so it is something everyone who uses LLM should know.

Re: Things we learned about LLMs in 2024

#507

Post by monero-xmr »

adwn wrote: post #486
monero-xmr wrote: post #445

Earlier quoted context omitted.

The environmental arguments are hilarious to me as a diehard crypto guy. The ultimate answer to “waste” of electricity arguments is that energy is a free market and people pay the price if it’s useful for them. As long as the activity isn’t illegal then training LLMs or mining bitcoins, it doesn’t matter. I pay for the electricity I use.

One argument against that line of thinking is that energy production has negative externalities. If you use a lot of electricity, its price goes up, which incentivizes more electricity production, which generates more negative externalities. It will also raise the costs for other consumers of electricity. Now that alone is not yet an argument against crypto currencies, and one person's frivolous squandering of resour…

I greatly despise video games. Why is that not a waste of energy? If you are entertained by something, even if it serves no human purpose other than entertainment, is that not a valid use of electricity?

Re: Things we learned about LLMs in 2024

#508

Post by danielbln »

abhijeetpbodas wrote: post #488
antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

> ask very precise questions explaining the background IME, being forced to write about something or verbally explaining/enumerating things in detail _by itself_ leads to a lot of clarity in the writer's thoughts, irrespective of if there's an LLM answering back. People have been doing rubber-duck-debugging since long. The metaphorical duck (LLMs in our context), if explained to well, has now started answering back w…

One thing LLMs have been incredibly strong even since gpt-3.5 is being the most advanced non-human rubber duck, and while they can do plenty more, that alone provides (me at least) with tremendous utility.

Re: Things we learned about LLMs in 2024

#509

Post by danielbln »

dartos wrote: post #327
BeetleB wrote: post #242

Earlier quoted context omitted.

I think most tech folks struggle with it because they treat LLMs as computer programs, and their experience is that SW should be extremely reliable - imagine using a calculator that was wrong 5% of the time - no one would accept that! Instead, think of an LLM as the equivalent of giving a human a menial task. You know that they're not 100% reliable, and so you give them only tasks that you can quickly verify and corr…

> Don't use LLMs where accuracy is paramount. Then why do people keep pushing it for code related tasks? Accuracy and precision is paramount with code. It needs to express exactly what needs to be done and how.

Because there are other ways to validate the output, types being one of them, tests another. Or simply running the code. It's easy enough to validate the output given the right approach that code generated by an LLM (usually as the result of a conversation/discussion about what should be accomplished) is a net positive.

If you zero-prompt and copy-paste the first result into your codebase, yeah, the accuracy problem will rear its ugly head real quick.

Re: Things we learned about LLMs in 2024

#510

Post by PurestGuava »

archagon wrote: post #465
mvkel wrote: post #235

Earlier quoted context omitted.

I'm surprised at the description that it's "useless" as a programming / design partner. Even if it doesn't make "elegant" code (whatever that means), it's the difference between an app existing at all, or not. I built and shipped a Swift app to the App Store, currently generating $10,200 in MRR, exclusively using LLMs. I wouldn't describe myself as a programmer, and didn't plan to ever build an app, mostly because in…

Off topic, but I'm a bit confused. Your iOS apps as listed on your website are CarPrep and Brocly, neither of which appear to have notable review activity or buzz in the media. If the app you're referring to is one of these, the more interesting question (to me) is: how on Earth are you generating $10,200 MRR from it? Or is there another app that I'm missing? (In my experience as an app developer, getting any tractio…

> (In my experience as an app developer, getting any traction and/or money from your app can be much more difficult than actually building it.)

This. The app I built has maybe 50 downloads despite me trying quite hard to promote it. It's very difficult work, even with the app being completely free of charge (save for a donation button).

Post reply on HN