Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new AI winter is coming?

taranis.ie

Post reply on HN
Print view View original

31–40 of 301 posts

Re: A new AI winter is coming?

#31

Post by cmiles8 »

LLMs are an amazing advancement. The tech side of things is very impressive. Credit where credit is due.

Where the current wave all falls apart is on the financials. None of that makes any sense and there’s no obvious path forward.

Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment.

Tech wise I’m bullish. Business wise, AI is setting up to be a big disaster. Those that aimlessly chased the hype are heading for a world of financial pain.

Re: A new AI winter is coming?

#32

Post by bgwalter »

The poster is right. LLMs are Gish Gallop machines that produce convincing sounding output.

People have figured it out by now. Generative "AI" will fail, other forms may continue, though it it would be interesting to hear from experts in other fields how much fraud there is. There are tons of material science "AI" startups, it is hard to believe they all deliver.

Re: A new AI winter is coming?

#33

Post by danielvaughn »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

I think it’s a disingenuous read to assume original commenter means “understanding” in the literal sense. When we talk about LLM “understanding”, we usually mean it from a practical sense. If you give an input to the computer, and it gives you an expected output, then colloquially the computer “understood” your input.

Re: A new AI winter is coming?

#34

Post by pawelduda »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

The end effect certainly gives off "understanding" vibe. Even if method of achieving it is different. The commenter obviously didn't mean the way human brain understands

Re: A new AI winter is coming?

#35

Post by frotaur »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

It astonishes me how people can make categorical judgements on things as hard to define as 'understanding'.

I would say that, except for the observable and testable performance, what else can you say about understanding?

It is a fact that LLMs are getting better at many tasks. From their performance, they seem to have an understanding of say python.

The mechanistic way this understanding arises is different than humans.

How can you say then it is 'not real', without invoking the hard problem of consciousness, at which point, we've hit a completely open question.

Re: A new AI winter is coming?

#36

Post by baseballdork »

HardCodedBias wrote: post #20
teaearlgraycold wrote: post #4

LLMs have failed to live up to the hype, but they haven't failed outright.

Two claims here: 1) LLMs have failed to live up to the hype. Maybe. Depends upon's who's hype. But I think it is fine to say that we don't have AGI today (however that is defined) and that some people hyped that up. 2) LLMs haven't failed outright I think that this is a vast understatement. LLMs have been a wild success. At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is…

> At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is larger.

I have a really hard time believing that stat without any context, is there a source for this?

Re: A new AI winter is coming?

#37

Post by Barathkanna »

Interesting take. His argument is basically that LLMs have hit their architectural ceiling and the industry is running on hype and unsustainable economics. I’m not fully convinced, but the points about rising costs and diminishing returns are worth paying attention to. The gap between what these models can actually do and what they’re marketed as might become a real problem if progress slows.

Re: A new AI winter is coming?

#38

Post by ilikeatari »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

We could use a little more kindness in discussion. I think the commenter has a very solid understanding on how computer works. The “understanding” is somewhat complex but I do agree with you that we are not there yet. I do think that the paradigm shift though is more about the fact that now we can interact with the computer in a new way.

Re: A new AI winter is coming?

#39

Post by neom »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

Birds and planes operate using somewhat different mechanics, but they do both achieve flight.

Re: A new AI winter is coming?

#40

Post by ryanjshaw »

Barathkanna wrote: post #37

Interesting take. His argument is basically that LLMs have hit their architectural ceiling and the industry is running on hype and unsustainable economics. I’m not fully convinced, but the points about rising costs and diminishing returns are worth paying attention to. The gap between what these models can actually do and what they’re marketed as might become a real problem if progress slows.

The existence of an AI hype train doesn’t mean there isn’t a productive AI no-hype train.

Context: I have been writing software for 30 years. I taught myself assembly language and hacked games/apps as a kid, and have been a professional developer for 20 years. I’m not a noob.

I’m currently building a real-time research and alerting side project using a little army of assistant AI developers. Given a choice, I would never go back to how I developed software before this. That isn’t my mind poisoned by hype and marketing.

Post reply on HN