Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Inevitabilism

tomrenner.com

Post reply on HN
Print view View original

911–920 of 1001 posts

Re: LLM Inevitabilism

#911

Post by scarface_74 »

Centigonal wrote: post #678
dbalatero wrote: post #538

Earlier quoted context omitted.

They might generate 10b ARR, but they lose a lot more than that. Their paid users are a fraction of the free riders. https://www.wheresyoured.at/openai-is-a-systemic-risk-to-the...

This echoes a lot of the rhetoric around "but how will facebook/twitter/etc make money?" back in the mid 2000s. LLMs might shake out differently from the social web, but I don't think that speculating about the flexibility of demand curves is a particularly useful exercise in an industry where the marginal cost of inference capacity is measured in microcents per token. Plus, the question at hand is "will LLMs be rele…

No one ever doubted that Facebook would make money. It was profitable early on, never lost that much money and was definitely profitable by the time it went public.

Twitter has never been consistently profitable.

ChatGPT also has higher marginal costs than any of the software only tech companies did previously.

Re: LLM Inevitabilism

#912

Post by gonzobonzo »

lsy wrote: post #63

I think two things can be true simultaneously: 1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them. 2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expen…

My guess is that LLM's are bridge technology, the equivalent of cassette tapes. A big step forward, allowing things that we couldn't before. But before long they'll be surpassed by much better technology, and future generations will look back on them as primitive.

You have top scientists like LeCun arguing this position. I'd imagine all of these companies are desperately searching for the next big paradigm shift, but no one knows when that will be, and until then they need to squeeze everything they can out of LLMs.

Re: LLM Inevitabilism

#913

Post by jittery41 »

lsy wrote: post #63

I think two things can be true simultaneously: 1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them. 2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expen…

Oh wow I forgot that the microwave oven was once marketed as the total replacement of cooking chores and in futuristic life people can just press a button and have a delicious good meal ( well you can now but microwave meals are often seen as worse than fastfood ).

Re: LLM Inevitabilism

#914

Post by protocolture »

LLMs are going to be around for a long time.

The technology being available, is "inevitable".

The legal, technological and social consequences are not known, but it is inevitable that whatever they are our kids will have to live with them.

Re: LLM Inevitabilism

#915

Post by alexpotato »

lsy wrote: post #63

I think two things can be true simultaneously: 1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them. 2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expen…

To use the Internet as a comparison:

Phase 1 - mid to late 1990s:

- "The Internet is going to change EVERYTHING!!!"

Phase 2 - late 1990s to early 2000s:

- "It's amazing and we are all making SO much money!"

- "Oh no! The bubble burst"

- "Of course everyone could see this coming: who is going to buy 40 lb bags of dogfood or their groceries over the Internet?!?!?"

Phase 3 - mid 2000s to 2020:

- "It is astounding the amount of money being by tech companies"

- "Who could have predicted that social media would change the ENTIRE landscape??"

Re: LLM Inevitabilism

#916

Post by JohnMakin »

famouswaffles wrote: post #901
JohnMakin wrote: post #862

Earlier quoted context omitted.

> That is a such a wild claim. People like the output of LLMs so much that ChatGPT is the fastest growing app ever. And this kind of meaningless factoid was immediately usurped by the Threads app release, which IMO is kind of a pointless app. Maybe let's find a more meaningful metric before saying someone else's claim is wild.

Asking your Instagram Users to hop on to your ready made TikTok Clone is hardly in the same sphere as spinning up that much users from nothing. And while Threads growth and usage stalled, ChatGPT is very much still growing and has *far* more monthly visits than threads. There's really nothing meaningless about ChatGPT being the 5th most visited site on the planet, not even 3 years after release. Threads doesn't make…

I think you just precisely explained why MAU / DAU growth is a meaningless metric in such discussions.

Re: LLM Inevitabilism

#917

Post by ojr »

seunosewa wrote: post #432
ojr wrote: post #420

The company name was changed from Facebook to Meta because Mark thought the metaverse was inevitable, it's ironic that you use a quote from him

The true reason was to have a new untainted brand after the election scandal.

After the scandal… Why did he choose the word “Meta” over the million other possibilities? Its because he thought the metaverse had a great future.

Re: LLM Inevitabilism

#918

Post by ojr »

babelfish wrote: post #785
ojr wrote: post #420

The company name was changed from Facebook to Meta because Mark thought the metaverse was inevitable, it's ironic that you use a quote from him

nope https://www.techemails.com/p/mark-zuckerberg-facebook-strate...

To clarify, not talking about why it was changed but why it was changed to the name “Meta”

Re: LLM Inevitabilism

#919

Post by ogogmad »

api wrote: post #667
lsy wrote: post #63

I think two things can be true simultaneously: 1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them. 2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expen…

My take since day one: (1) Model capabilities will plateau as training data is exhausted. Some additional gains will be possible by better training, better architectures, more compute, longer context windows or "infinite" context architectures, etc., but there are limits here. (2) Training on synthetic data beyond a very limited amount will result in overfitting because there is no new information. To some extent you…

The next step is clearly improved vision and everyday-physics models. These can also solve hallucinations.

Re: LLM Inevitabilism

#920

Post by stickfigure »

kenjackson wrote: post #845
strange_quark wrote: post #828

Earlier quoted context omitted.

> There are many technologies that have seemed inevitable and seen retreats under the lack of commensurate business return (the supersonic jetliner) I think this is a great analogy, not just to the current state of AI, but maybe even computers and the internet in general. Supersonic transports must've seemed amazing, inevitable, and maybe even obvious to anyone alive at the time of their debut. But hiding under that…

I think this is both right and wrong. There was a good book that came out probably 15 years ago about how technology never stops in aggregate, but individual technologies tend to grow quickly and then stall. Airplane jets were one example in the book. The reason why I partially note this as wrong is that even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don…

> even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today

I hate to dogpile on this statement but I can think of two major issues right now:

* Small context windows, and serious degradation when pushing the limits of existing context windows. A human can add large amounts of state to their "context window" every day.

* Realtime learning. My humans get smarter every day, especially in the context of working with a specific codebase.

Maybe the AI companies will figure this out, but they are not "same technique more processor power" kinds of problems.

Post reply on HN