Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new AI winter is coming?

taranis.ie

Post reply on HN
Print view View original

41–50 of 301 posts

Re: A new AI winter is coming?

#41

Post by Uehreka »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

“You understand how the brain works right? It’s neurons and electrical charges. The brain understands nothing.”

I’m always struck by how confidently people assert stuff like this, as if the fact that we can easily comprehend the low-level structure somehow invalidates the reality of the higher-level structures. As if we know concretely that the human mind is something other than emergent complexity arising from simpler mechanics.

I’m not necessarily saying these machines are “thinking”. I wish I could say for sure that they’re not, but that would be dishonest: I feel like they aren’t thinking, but I have no evidence to back that up, and I haven’t seen non-self-referential evidence from anyone else.

Re: A new AI winter is coming?

#42

Post by Lambdanaut »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

You understand how the brain works right? It's probability distributions mapped to sodium ion channels. The human understands nothing.

Re: A new AI winter is coming?

#43

Post by asah »

chomp wrote: post #14
tarr11 wrote: post #7

This has convinced many non-programmers that they can program, but the results are consistently disastrous, because it still requires genuine expertise to spot the hallucinations. I've been programming for 30+ years and now a people manager. Claude Code has enabled me to code again and I'm several times more productive than I ever was as an IC in the 2000s and 2010s. I suspect this person hasn't really tried the most…

For toy and low effort coding it works fantastic. I can smash out changes and PRs fantastically quick, and they’re mostly correct. However, certain problem domains and tough problems cause it to spin its wheels worse than a junior programmer. Especially if some of the back and forth troubleshooting goes longer than one context compaction. Then it can forget the context of what it’s tried in the past, and goes back to…

That was true six months ago - the latest versions are much better at memory and adherence, and my senior engineer friends are adopting LLMs quickly for all sorts of advanced development.

Re: A new AI winter is coming?

#45

Post by Eddy_Viscosity2 »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

It could very well be that statistics and tokens is how our brains work at the computational level too. Just that our algorithms have slightly better heuristics due to all those millennia of A/B testing of our ancestors.

Re: A new AI winter is coming?

#46

Post by uejfiweun »

"The technology is essentially a failure" is in the headline of this article. I have to disagree with that. For the first time in the history of the UNIVERSE, an entity exists that can converse in human language at the same level that humans can.

But that's me being a sucker. Because in reality this is just a clickbait headline for an article basically saying that the tech won't fully get us to AGI and that the bubble will likely pop and only a few players will remain. Which I completely agree with. It's really not that profound.

Re: A new AI winter is coming?

#47

Post by HPsquared »

neom wrote: post #39
gishh wrote: post #21

Earlier quoted context omitted.

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

Birds and planes operate using somewhat different mechanics, but they do both achieve flight.

Birds and planes are very similar other than the propulsion and landing gear, and construction materials. Maybe bird vs helicopter, or bird vs rocket.

Re: A new AI winter is coming?

#48

Post by LatencyKills »

gishh wrote: post #21
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't…

As someone who was an engineer on the original Copilot team, yes I understand how tech works.

You don’t know how your own mind “understands” something. No one on the planet can even describe how human understanding works.

Yes, LLMs are vast statistical engines but that doesn’t mean something interesting isn’t going on.

At this point I’d argue that humans “hallucinate” and/or provide wrong answers far more often than SOTA LLMs.

I expect to see responses like yours on Reddit, not HN.

Re: A new AI winter is coming?

#49

Post by john61 »

stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why.

OP says it is because that predicting the next token can be correct or not, but it always looks plausible because that is what it calculates. Therefore it is dangerous and can not be fixed because it is how it works in essence.

Re: A new AI winter is coming?

#50

Post by Groxx »

baseballdork wrote: post #36
HardCodedBias wrote: post #20

Earlier quoted context omitted.

Two claims here: 1) LLMs have failed to live up to the hype. Maybe. Depends upon's who's hype. But I think it is fine to say that we don't have AGI today (however that is defined) and that some people hyped that up. 2) LLMs haven't failed outright I think that this is a vast understatement. LLMs have been a wild success. At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is…

> At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is larger. I have a really hard time believing that stat without any context, is there a source for this?

from what I've seen in a several-thousand-eng company: LLMs generally produce vastly more code than is necessary, so they quickly out-pace human coders. they could easily be producing half or more of all of the code even if only 10% of the teams use it. particularly because huge changes often get approved with just a "lgtm", and LLM-coding teams also often use/trust LLMs for reviews.

but they do that while making the codebase substantially worse for the next person or LLM. large code size, inconsistent behavior, duplicates of duplicates of duplicates strewn everywhere with little to no pattern so you might have to fix something a dozen times in a dozen ways for a dozen reasons before it actually works, nothing handles it efficiently.

the only thing that matters in a business is value produced, and I'm far from convinced that they're even break-even if they were free in most cases. they're burning the future with tech debt, on the hopes that it will be able to handle it where humans cannot, which does not seem true at all to me.

Post reply on HN