Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

Post reply on HN
Print view View original

141–150 of 615 posts

Re: Things we learned about LLMs in 2024

#141

Post by lolinder »

simonw wrote: post #112
lolinder wrote: post #73

Earlier quoted context omitted.

> Code hallucinations are also the least damaging type of hallucinations, because you get fact checking for free: if you run the code and get an error you know there's a problem. This is great when the error is a thrown exception, but less great when the error is a subtle logic bug that only strikes in some subset of cases. For trivial code that only you will ever run this is probably not a big deal—you'll just fix i…

Yeah, the other skill you need to develop to make the most of AI-assisted programming is really good manual QA.

Have you found that to be a good trade-off for large-scale projects?

Where I'm at right now with LLMs is that I find them to be very helpful for greenfield personal projects. Eliminating the blank canvas problem is huge for my productivity on side projects, and they excel at getting projects scaffolded and off the ground.

But as one of the lead engineers working on a million+ line, 10+ year-old codebase, I've yet to see any substantial benefit come from myself or anyone else using LLMs to generate code. For every story where someone found time saved, we have a near miss where flawed code almost made it in or (more commonly) someone eventually deciding it was a waste of time to try because the model just wasn't getting it.

Getting better at manual QA would help, but given the number of times where we just give up in the end I'm not sure that would be worth the trade-off over just discouraging the use of LLMs altogether.

Have you found these things to actually work on large, old codebases given the right context? Or has your success likewise been mostly on small things?

Re: Things we learned about LLMs in 2024

#142

Post by simonw »

viccis wrote: post #95

I didn't realize "agent" designs were that ambiguously defined. Every AI engineer I've talked to uses it to mean a design that combines several separate LLM prompts (or even models) to solve problems in multiple stages.

I'll add that one to the list. Surprisingly it doesn't closely match most of the 211 definitions I've collected already!

The closest in that collection is "A division of responsibilities between LLMs that results in some sort of flow?" - https://lite.datasette.io/?json=https://gist.github.com/simo...

Re: Things we learned about LLMs in 2024

#143

Post by simonw »

uludag wrote: post #98
agentultra wrote: post #2

Don’t forget that 2024 was also a record year for new methane power plant projects. Some 200 new projects in the US alone and I’d wager most of them are funded directly by big tech for AI data centres. https://www.bnnbloomberg.ca/investing/2024/09/16/ai-boom-is-... This is definitely extending the runway of O&G at a crisis point in the climate disaster when we’re supposed to be reducing and shutting down these power…

But according to the author, apparently bringing this up isn't helpful criticism. I'm curious what peoples thoughts are of what the future of LLMs would be like if we severely overshoot our carbon goals. How bad would thinks have to get for people to stop caring about this technology?

It's helpful criticism as part of the conversation. What frustrates me is when people go "LLMs are burning the planet!" and leave it at that.

Re: Things we learned about LLMs in 2024

#144

Post by lolinder »

polishdude20 wrote: post #128
lolinder wrote: post #73

Earlier quoted context omitted.

> Code hallucinations are also the least damaging type of hallucinations, because you get fact checking for free: if you run the code and get an error you know there's a problem. This is great when the error is a thrown exception, but less great when the error is a subtle logic bug that only strikes in some subset of cases. For trivial code that only you will ever run this is probably not a big deal—you'll just fix i…

When they spit out these subtle bugs, are you promoting the LLM to watch our for that particular bug? I wonder if it just needs a vir more guidance in more explicit terms

At a certain point it becomes more work to prompt the LLM with each and every edge case than it is to just write the dang code.

I work out what the edge cases are by writing and rewriting the code. It's in the process of shaping it that I see where things might go wrong. If an LLM can't do that on its own it isn't of much value for anything complicated.

Re: Things we learned about LLMs in 2024

#145

Post by cactusfrog »

nntwozz wrote: post #65

I think John Gruber summed it up nicely: https://daringfireball.net/2024/12/openai_unimaginable OpenAI’s board now stating “We once again need to raise more capital than we’d imagined” less than three months after raising another $6.6 billion at a valuation of $157 billion sounds alarmingly like a Ponzi scheme — an argument akin to “Trust us, we can maintain our lead, and all it will take is a never-ending stream of…

Every waste of money is not a Ponzi scheme.

Re: Things we learned about LLMs in 2024

#146

Post by behnamoh »

vikramkr wrote: post #76
Alex-Programs wrote: post #60

Earlier quoted context omitted.

Why would they be able to output a Google doc? It's a proprietary format. The closest thing would be rich text format to copy paste.

That proprietary format is owned by a company associated with folks who won two nobel prizes for AI related work this year and the employer at the time of the researchers who wrote the attention is all you need paper and also the owner of a search engine with access to like, all the data. Doesn't seem unreasonable lol

[flagged]

Re: Things we learned about LLMs in 2024

#147

Post by cruffle_duffle »

antirez wrote: post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

To get the most out of them you have to provide context. Treat these models like some kind of eager beaver junior engineer who wants to jump in and write code without asking questions. Force it to ask questions (eg: “do not write code yet, please restate my requirements to make sure we are in alignment. Are there any extra bits of context or information that would help? I will tell you when to write code”)

If your model / chat app has the ability to always inject some kind of pre-prompt make sure to add something like “please do not jump to writing code. If this was a coding interview and you jumped to writing code without asking questions and clarifying requirements you’d fail”.

At the top of all your source files include a comment with the file name and path. If you have a project on one of these services add an artifact that is the directory tree (“tree —-gitignore” is my goto). This helps “unaided” chats get a sense of what documents they are looking at.

And also, it’s a professional bullshitter so don’t trust it with large scale code changes that rely on some language / library feature you don’t have personal experience with. It can send you down a path where the entire assumption that something was possible turns out to be false.

Does it seek like a lot of work? Yes. Am I actually more productive with the tool than without? Probably. But it sure as shit isn’t “free” in terms of time spent providing context. I think the more I use these models, the more I get a sense of what it is good at and what is going to be a waste of time.

Long story short, prompting is everything. These things aren’t mind readers (and worse they forget everything in each new session)

Re: Things we learned about LLMs in 2024

#148

Post by api »

com2kid wrote: post #104
benreesman wrote: post #32

Earlier quoted context omitted.

Us skeptics believe that valuation prices in some form of regulatory capture or other non-market factor. The non-skeptical interpretation is that it's a threshold function, a flat-out race with an unambiguous finish line. If someone actually hit self-improving AGI first there's an argument that no one would ever catch up.

There are some really good books about wars between cultures that have AGI and it always comes down to math - whoever can get their hands on more compute faster wins.

This is also a strong argument for immigration, particularly high-skill immigration. In the absence of synthetic AGI whoever imports the most human AGI wins.

Re: Things we learned about LLMs in 2024

#149

Post by EMIRELADERO »

GOD_Over_Djinn wrote: post #140
com2kid wrote: post #100

Earlier quoted context omitted.

If we can simulate a full human intelligence at a reasonable speed, we can simulate 100 of them and ask the AGI to figure out how to make itself 10x faster. Rinse and repeat. That is exponential take off. At the point where you have an army of AIs running at 1000x human speed it can just ask it to design the mechanisms for and write the code to make robots that automate any possible physical task.

This sounds like magic, not science.

What do you mean by this? Is there any fundamental property of intelligence, physicality, or the universe, that you think wouldn't let this work?

Re: Things we learned about LLMs in 2024

#150

Post by philipkglass »

datadrivenangel wrote: post #137
philipkglass wrote: post #108

Earlier quoted context omitted.

I believe that AGI cannot be exponential for long because any intelligent agent can only approach nature's limits asymptotically. The first company with AGI will be about as much ahead as, say, the first company with electrical generators [1]. A lot of science fiction about a technological singularity assumes that AGI will discover and apply new physics to develop currently-believed-impossible inventions, but I don't…

I don't recall editing my message, but HN can be wonky sometimes. :) Nothing is truly exponential for long, but the logistic curve could be big enough to do almost anything if you get imaginative. Without new physics, there are still some places where we can do some amazing things with the equivalent of several trillion dollars of applied R&D, which AGI gets you.

I had to edit my message just now because I was actually unsure if you edited. Sorry for any miscommunication.
Post reply on HN