Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

Post reply on HN
Print view View original

611–615 of 615 posts

Re: Things we learned about LLMs in 2024

#611

Post by CRConrad »

simonw wrote: post #50
henning wrote: post #37

Spookily good at writing code? LLMs frequently hallucinate broken nonsense shit when I use them. Recognize what they do well (generate simple code in popular languages) while acknowledging where they are weak (non-trivial algorithms, any novel code situation the LLM hasn't seen before, less popular languages).

Did you try learning HOW to get good code out of them? As with all things LLM there's a whole lot of undocumented and under appreciated depth to getting decent results. Code hallucinations are also the least damaging type of hallucinations, because you get fact checking for free: if you run the code and get an error you know there's a problem. A lot of the time I find pasting that error message back into the LLM gets…

> Did you try learning HOW to get good code out of them?

Isn't that a bit "You're holding it wrong"? I mean, why isn't that the default; did anyone really think one would mainly want bad results out of them?

Re: Things we learned about LLMs in 2024

#612

Post by ben_w »

lucubratory wrote: post #332
ToucanLoucan wrote: post #77

Earlier quoted context omitted.

I mean, I appreciate that distinction and don't disagree. And, if this is going to continue being a trend, I think we need more stringent restrictions on what sorts of resources are permitted to be consumed in the power plants that are constructed to meet the needs of hyperscaler data centers. Because whether we're using tons of compute to provide value or not doesn't change that we are using tons of compute and tons…

I think basically everyone should support a carbon tax. It's a really obvious solution that is both environmentally friendly and should be acceptable to free market fanatics because it is explicitly and only taxing a negative externality on the public - it's hard to imagine a more justified tax. Combined with the increased cost effectiveness of renewables & batteries, & the new build-out of nuclear, it could plausibl…

I'd be fine with a carbon tax, if only we could get every nation to do it near-simultaneously (within a few years of each other at most) — I don't think it's sufficient for any one nation to say they'll do that for local production plus an equivalent import tariff to compensate for what other nations are doing (we also want to lower emissions of everyone else's internal markets, but also there's a lot of people who will fraudulently claim they're eco-friendly when they're not, and that's harder to catch when there is an international border in the way) — but "the perfect is the enemy of the good", and this may still be a step in the right direction even if my concerns are proportionate to the actual risks (which they may not be).

I think the rapidly decreasing costs of renewables and storage are likely to make the transition happen before the political will to get a carbon tax, but if you recon you can push the right buttons, I encourage you to try it :)

Re: Things we learned about LLMs in 2024

#613

Post by simonw »

CRConrad wrote: post #611
simonw wrote: post #50

Earlier quoted context omitted.

Did you try learning HOW to get good code out of them? As with all things LLM there's a whole lot of undocumented and under appreciated depth to getting decent results. Code hallucinations are also the least damaging type of hallucinations, because you get fact checking for free: if you run the code and get an error you know there's a problem. A lot of the time I find pasting that error message back into the LLM gets…

> Did you try learning HOW to get good code out of them? Isn't that a bit "You're holding it wrong"? I mean, why isn't that the default; did anyone really think one would mainly want bad results out of them?

When I say that these things are deceptively difficult to use I don't intend that as a ringing endorsement of the technology.

Re: Things we learned about LLMs in 2024

#614

Post by CRConrad »

fragmede wrote: post #608
CRConrad wrote: post #607

Earlier quoted context omitted.

> the "lazy is bad" crowd is ludditism in another form Capital --> capitalist, capitalism. Commune --> communist, communism. Ned Ludd --> Luddite, Luddism. Not "capitalistism" or "communistism", so not "ludditism" either.

Going by your other examples, shouldn't it be Luddist and not Luddite? English is famously inconsistent and is a difficult language to learn, owing to it's linguistic heritage, as it's a bunch of exceptions to rules. Like i before e, except after c, but also in a bunch of other words, so every English student just needs to remember those.

Yeah, maybe not the best examples. There are other "-ite" words, but I can't recall off the top of my head any such that also have "-ism" forms. It is, as always with languages, "just the way it is": I've always seen it rendered as "Luddite", never "-ist". (Maybe because it's named for a person, not a thing or principle? Tried to hint at that possibility with his full name.)

Yup, English may be the most inconsistent of languages. When I was a kid, we used to blame French for being "just exceptions to rules, exceptions to exceptions, and exceptions to those exceptions!", but with a few decades of perspective... Nope, English is far worse.

Re: Things we learned about LLMs in 2024

#615

Post by phantompeace »

hatefulmoron wrote: post #576
phantompeace wrote: post #372

Earlier quoted context omitted.

The quality of the summary is only as good as the effort you put into writing your workflow. If you’re simply one shotting the paper into a message and saying “plz summarise this and I’ll reward you with $1m” then of course it’s gonna be shit. But if you semantically chunked along sections and do some RAG Q&A summaries before combining into a well formatted schema then it’s probably going to be better than the first…

> I’m using the summaries as a juicier abstract. I’m not taking them as gospel. I'm not sure of the value of this. Papers already have abstracts, rewording them using LLMs is just playing with your food. If you're seeing use out of it that's awesome though.

You do have a point you know. I have actually been thinking about this recently and have decided to try and focus more on extracting value out of abstracts instead of summarising papers, and relying on embeddings of the paper in case the answer needs more context.
Post reply on HN