Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

61–70 of 615 posts

Re: Things we learned about LLMs in 2024

#61
post #20

Earlier quoted context omitted.

The last OpenAI valuation I read about was 157 billion. I am struggling to understand what justifies this. To me, it feels like OpenAI is at best few months ahead of competitors in some areas. But even if I am underestimating the advantage, it's few years instead of few months, why does it matter? It's not like AI companies are going to enjoy the first-mover advantage internet giants had over the competition.

It's justified if AGI is possible. If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That is of course, assuming AGI is possible and exponential, and that marketshare goes to a single entity instead of a set of entities. Lots of big assumptions. Seems like we're heading towards a slow-lackluster singularity though.

> If AGI is possible, then the entire human economy stops making sense as far as money goes

I heard people on HN saying this (even without the money condition) and I fail to grasp the reasoning behind it. Suppose in a few years Altman announces a model, say o11, that is supposedly AGI, and in several benchmarks it hits over 90%. I don't believe it's possible with LLMs because of their inherent limitations but let's assume it can solve general tasks in a way similar to an average human.

Now, how come that "the entire human economy stops making sense"? In order to eat, we need farmers, we need construction workers, shops etc. As for white collar workers, you will need a whole range of people to maintain and further develop this AGI. So IMHO the opposite is true: the human economy will work exactly as before but the job market will continue to evolve withe people using AGI in a similar way that they use LLMs now but probably with greater confidence. (Or not.)

Re: Things we learned about LLMs in 2024

#63
post #57
post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

Right, in simpler terms: The measure of LLMs success is how effectively they help you achieve your goal faster.

Exactly, and right now the LLMs acceleration effect is a tool, not "give me the final solution". Even people that can't code, using LLMs to build applications from scratch, still have this tool mindset. This is why they can use them effectively: they don't stop at the first failed solution; they provide hints to the LLM, test the code, try to figure what's the problem (also with the LLM help), and so forth. It's a matter of mindset.

Re: Things we learned about LLMs in 2024

#64
post #20

Earlier quoted context omitted.

The last OpenAI valuation I read about was 157 billion. I am struggling to understand what justifies this. To me, it feels like OpenAI is at best few months ahead of competitors in some areas. But even if I am underestimating the advantage, it's few years instead of few months, why does it matter? It's not like AI companies are going to enjoy the first-mover advantage internet giants had over the competition.

It's justified if AGI is possible. If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That is of course, assuming AGI is possible and exponential, and that marketshare goes to a single entity instead of a set of entities. Lots of big assumptions. Seems like we're heading towards a slow-lackluster singularity though.

If AGI is invented and the inventor tries to keep it secret then everyone in the world will be trying to steal it. And funding to independently create it would become effectively unlimited once it has been proven possible, much like with nuclear weapons.

Re: Things we learned about LLMs in 2024

#65
I think John Gruber summed it up nicely:

https://daringfireball.net/2024/12/openai_unimaginable

OpenAI’s board now stating “We once again need to raise more capital than we’d imagined” less than three months after raising another $6.6 billion at a valuation of $157 billion sounds alarmingly like a Ponzi scheme — an argument akin to “Trust us, we can maintain our lead, and all it will take is a never-ending stream of infinite investment.”

Re: Things we learned about LLMs in 2024

#66
post #23

Earlier quoted context omitted.

It would be really cool if big tech could find a new hyperscaler model that didn't also require offsetting the goals of green energy projects worldwide. Between LLM and crypto you'd swear they're trying to find the most energy-wasteful tech possible.

It seems odd to put crypto and LLMs in the same boat in this regard - I might be wrong but are there any crypto projects that actually provide value? I'm sure there are ones that do folding or something but among the big ones?

Value is a hard term, this link will seem snarky, but: https://www.axios.com/2024/12/25/russia-bitcoin-evade-sancti...

So in a way, it is providing value to someone, whether we like it or not.

Or Drug Cartels. https://www.context.news/digital-rights/how-crypto-helps-lat...

But this is the promise of uncontrollable decentralization providing value, for good or bad?

Re: Things we learned about LLMs in 2024

#68

In spite of all this progress, I can't find LLMs that solve simple tasks like: Here is my resume. Make it look nice (some design hints). They can spit html and css, but not Google doc. On the other hand, Google results are dominated by SEO spam. You can probably find one usable result on page 10. The problem is not technology. It's a business model that can support the humans feeding data into the LLM.

They can spit out LaTeX, and a PDF from that is going to look much nicer than a Google doc (and display the same everywhere). As an added bonus, the recruiter can't randomly rewrite parts of it (at least not so easily).

Re: Things we learned about LLMs in 2024

#69
post #54

About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…

While Claude Sonnet is superior than 4o for most my use cases, there are still occasionally some specific tasks where it performs slightly better.

Re: Things we learned about LLMs in 2024

#70
post #20

Earlier quoted context omitted.

The last OpenAI valuation I read about was 157 billion. I am struggling to understand what justifies this. To me, it feels like OpenAI is at best few months ahead of competitors in some areas. But even if I am underestimating the advantage, it's few years instead of few months, why does it matter? It's not like AI companies are going to enjoy the first-mover advantage internet giants had over the competition.

It's justified if AGI is possible. If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That is of course, assuming AGI is possible and exponential, and that marketshare goes to a single entity instead of a set of entities. Lots of big assumptions. Seems like we're heading towards a slow-lackluster singularity though.

> If AGI is possible, then the entire human economy stops making sense as far as money goes,

What does this mean in terms of making me coffee or building houses?

Post reply on HN