Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new AI winter is coming?

taranis.ie

Post reply on HN
Print view View original

71–80 of 301 posts

Re: A new AI winter is coming?

#71

Post by stanfordkid »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

What do you mean by “understand”? Do you mean conscious?

Understand just means “parse language” and is highly subjective. If I talk to someone African in Chinese they do not understand me but they are still conscious.

If I talk to an LLM in Chinese it will understand me but that doesn’t mean it is conscious.

If I talk about physics to a kindergartner they will not understand but that doesn’t mean they don’t understand anything.

Do you see where I am going?

Re: A new AI winter is coming?

#72

Post by stego-tech »

The winters are the best part, economic harm aside.

Winters are when technology falls out of the vice grip of Capital and into the hands of the everyman.

Winters are when you’ll see folks abandon this AIaaS model for every conceivable use case, and start shifting processing power back to the end user.

Winters ensure only the strongest survive into the next Spring. They’re consequences for hubris (“LLMs will replace all the jobs”) that give space for new things to emerge.

So, yeah, I’m looking forward to another AI winter, because that’s when we finally see what does and does not work. My personal guess is that agents and programming-assistants will be more tightly integrated into some local IDEs instead of pricey software subscriptions, foundational models won’t be trained nearly as often, and some accessibility interfaces will see improvement from the language processing capabilities of LLMs (real-time translation, as an example, or speech-to-action).

That, I’m looking forward to. AI in the hands of the common man, not locked behind subscription paywalls, advertising slop, or VC Capital.

Re: A new AI winter is coming?

#73

Post by gdulli »

cmiles8 wrote: post #31

LLMs are an amazing advancement. The tech side of things is very impressive. Credit where credit is due. Where the current wave all falls apart is on the financials. None of that makes any sense and there’s no obvious path forward. Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Tech wise I’…

> Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment.

We should factor in that messaging that's seamless and undisclosed in conversational LLM output will be a lot more valuable that what we think of as advertising today.

Re: A new AI winter is coming?

#74

Post by zzbzq »

swalsh wrote: post #64
cmiles8 wrote: post #31

LLMs are an amazing advancement. The tech side of things is very impressive. Credit where credit is due. Where the current wave all falls apart is on the financials. None of that makes any sense and there’s no obvious path forward. Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Tech wise I’…

Hard disagree, I'm in the process of deploying several AI solutions in Healthcare. We have a process a nurse usually spends about an hour on, and costs $40-$70 depending on if they are offshore and a few other factors. Our AI can match it at a few dollars often less. A nuse still reviews the output, but its way less time. The economics of those tokens is great. We have another solution that just finds money, $10-$30…

I think they were referring to the costs of training and hosting the models. You're counting the cost of what you're buying, but the people selling it to you are in the red.

Re: A new AI winter is coming?

#75

Post by ipdashc »

HardCodedBias wrote: post #20
teaearlgraycold wrote: post #4

LLMs have failed to live up to the hype, but they haven't failed outright.

Two claims here: 1) LLMs have failed to live up to the hype. Maybe. Depends upon's who's hype. But I think it is fine to say that we don't have AGI today (however that is defined) and that some people hyped that up. 2) LLMs haven't failed outright I think that this is a vast understatement. LLMs have been a wild success. At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is…

> At big tech over 40% of checked in code is LLM generated.

Assuming this is true though, how much of that 40% is boilerplate or simple, low effort code that could have been knocked out in a few minutes previously? It's always been the case that 10% of the code is particularly thorny and takes 80% of the time, or whatever.

Not to discount your overall point, LLMs are definitely a technical success.

Re: A new AI winter is coming?

#76

Post by Nevermark »

I am simply stunned at the negativity.

Yes, there is hype.

But if you actually filter it out, instead of (over) reacting to it in either direction, progress has been phenomenal and the fact there is visible progress in many areas, including LLMs, in the order of months demonstrates no walls.

Visible progress doesn’t mean astounding progress. But any tech that is improving year to year is moving at a good speed.

Huge apparent leaps in recent years seem to have spoiled some people. Or perhaps desensitized them. Or perhaps, created frustration that big leaps don’t happen every week.

I can’t fathom anyone not using models for 1000 things. But we all operate differently, and have different kinds of lives, work and problems. So I take claims that individuals are not getting much from models at face value.

But that some people are not finding the value isn’t an argument that those of us getting value, increasing value isn’t real.

Re: A new AI winter is coming?

#77

Post by [deleted] »

frotaur wrote: post #35
gishh wrote: post #21

Earlier quoted context omitted.

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

It astonishes me how people can make categorical judgements on things as hard to define as 'understanding'. I would say that, except for the observable and testable performance, what else can you say about understanding? It is a fact that LLMs are getting better at many tasks. From their performance, they seem to have an understanding of say python. The mechanistic way this understanding arises is different than huma…

[deleted]

Re: A new AI winter is coming?

#78

Post by cess11 »

dangus wrote: post #57
john61 wrote: post #49

Earlier quoted context omitted.

> The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why. OP says it is because that predicting the next token can be correct or not, but it always looks plausible because that is what it calculates. Therefore it is dangerous and can not be fixed because it is how it works in essence.

I just want to point out a random anecdote. Literally yesterday ChatGPT hallucinated an entire feature of a mod for a video game I am playing including making up a fake console command. It just straight up doesn’t exist, it just seemed like a relatively plausible thing to exist. This is still happening. It never stopped happening. I don’t even see a real slowdown in how often it happens. It sometimes feels like the o…

There is no difference between "hallucination" and "soberness", it's just a database you can't trust.

The response to your query might not be what you needed, similar to interacting with an RDBMS and mistyping a table name and getting data from another table or misremembering which tables exist and getting an error. We would not call such faults "hallucinations", and shouldn't when the database is a pile of eldritch vectors either. If we persist in doing so we'll teach other people to develop dangerous and absurd expectations.

Re: A new AI winter is coming?

#79

Post by AnimalMuppet »

citizenpaul wrote: post #10

>Expect OpenAI to crash, hard, with investors losing their shirts. Lol someone doesn't understand how the power structure system works "the golden rule". There is a saying if you owe the bank 100k you have a problem. If you owe the bank ten million the bank has a problem. OpenAI and the other players have made this bubble so big that there is no way the power system will allow themselves to take the hit. Expect some…

The difference is that OpenAI isn't financed by borrowed money.

Re: A new AI winter is coming?

#80

Post by jansan »

Barathkanna wrote: post #37

Interesting take. His argument is basically that LLMs have hit their architectural ceiling and the industry is running on hype and unsustainable economics. I’m not fully convinced, but the points about rising costs and diminishing returns are worth paying attention to. The gap between what these models can actually do and what they’re marketed as might become a real problem if progress slows.

I think we are not even close to using the potential of current LLMs. Even if capabilities of LLMs would not improve, we will see better performance on the software and hardware side. It is no longer a question of "if", but of "when" there will be a Babelfish like device available. And this is only one obvious application, I am 100% sure that people are still finding useful new applications of AI every day.

However, there is a real risk that AI stocks will crash and pull the entire market down, just like it happened in 2000 with the dotcom bubble. But did we see an internet or dotcom winter after 2000? No, everybody kept using the Internet, Windows, Amazon, Ebay, Facebook and all the other "useless crap". Only the stock market froze over for a few years and previously overhyped companies had a hard time, but given the exaggeration before 2000 this was not really a surprise.

What will happen is that the hype train will stop or slow down, and people will no longer get thousands, millions, billions, or trillions in funding just because they slap "AI" to their otherwise worthless project. Whoever is currently working on such a project should enjoy the time while it lasts - and rest assured that it will not last forever.

Post reply on HN