Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new AI winter is coming?

taranis.ie

Post reply on HN
Print view View original

281–290 of 301 posts

Re: A new AI winter is coming?

#281

Post by geekymartian »

swalsh wrote: post #64
cmiles8 wrote: post #31

LLMs are an amazing advancement. The tech side of things is very impressive. Credit where credit is due. Where the current wave all falls apart is on the financials. None of that makes any sense and there’s no obvious path forward. Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Tech wise I’…

Hard disagree, I'm in the process of deploying several AI solutions in Healthcare. We have a process a nurse usually spends about an hour on, and costs $40-$70 depending on if they are offshore and a few other factors. Our AI can match it at a few dollars often less. A nuse still reviews the output, but its way less time. The economics of those tokens is great. We have another solution that just finds money, $10-$30…

1) welp, I hope this is not my healthcare provider. 2) Do you realize the cost fallacy just extends one level deeper, since those pennies on tokens soon will become hundreds your nurses become cheaper again? 3) "Our AI", come on. What exactly are you using? this is a technical forum.

Re: A new AI winter is coming?

#282

Post by LatencyKills »

judgmentday wrote: post #230
LatencyKills wrote: post #207

Earlier quoted context omitted.

> as someone who is a sociopath completely devoid of ethics Ah yes... the hundred thousand researchers and engineers who work at MS are all evil. Many people who've made truly significant contributions to AI have either worked directly (through MS Research) or indirectly (OpenAI, Anthropic, etc) at MS. ResNet and concepts like Differential Privacy were invented there. What about the researchers at Stanford, Carnegie…

T2 would have been a different movie if Miles Dyson just said to Sarah Connor: "The tech was fantastic to work with, we had an amazing team, and I am proud of what we accomplished."

We had no idea that there would be emergent properties. AlexNet was not designed for language translation and yet it worked.

If we go with your joke, we’d never build or create anything.

Re: A new AI winter is coming?

#283

Post by LatencyKills »

Libidinalecon wrote: post #254
LatencyKills wrote: post #48

Earlier quoted context omitted.

As someone who was an engineer on the original Copilot team, yes I understand how tech works. You don’t know how your own mind “understands” something. No one on the planet can even describe how human understanding works. Yes, LLMs are vast statistical engines but that doesn’t mean something interesting isn’t going on. At this point I’d argue that humans “hallucinate” and/or provide wrong answers far more often than…

Yes because working on co-pilot makes one well educated in philosophy of mind. The standard, meaningless, HN appeal to authority. "I worked at Google, therefore I am an expert on both stringing a baroque lute and the finer points of Lao cooking. " Gemini 3 gives a nice explanation if asked "can you explain how you don't really understand anything"

I’ve worked with many neuroscience researchers during my career. At a minimum, I’m extremely well-read on the subject of cognition.

I am not going to lie or hide my experience. The world is a fucked up place because we no longer respect “authority”. I helped build one of these systems; my opinion is as valid as yours.

Yours is the “standard meaningless” response that adds zero technical insight. Let’s talk about supportive tracing or optimization of KV values during pre-training and how those factors impact the apparent “understanding” of the resulting model.

Re: A new AI winter is coming?

#284

Post by LatencyKills »

claytongulick wrote: post #269
LatencyKills wrote: post #48

Earlier quoted context omitted.

As someone who was an engineer on the original Copilot team, yes I understand how tech works. You don’t know how your own mind “understands” something. No one on the planet can even describe how human understanding works. Yes, LLMs are vast statistical engines but that doesn’t mean something interesting isn’t going on. At this point I’d argue that humans “hallucinate” and/or provide wrong answers far more often than…

My brain is able to determine when it doesn't know something. That does seem to be a bit important for any "intelligent" system.

So, if we find a way to ensure that transformers balk at hallucinating will you then say that they “understand” what they’re saying?

Because that’s what your comment indicates.

Re: A new AI winter is coming?

#285

Post by LatencyKills »

Jensson wrote: post #240
LatencyKills wrote: post #209

Earlier quoted context omitted.

> Where the humans aren't omniscient it fills the blanks with nonsense As do most humans. People lie. People make things up to look smart. People fervently believe things that are easily disproved. Some people are willfully ignorant, anti-science, anti-education, etc. The problem isn't the transformer architecture... it is the humans who advertise capabilities that are not there yet.

> As do most humans. People lie. People make things up to look smart. The main difference is that people know when they lie and make things up to look smart, LLM doesn't.

There are human beings who believe absolutely insane and easily disprovable things. Even in the face of facts they continue to remain willfully ignorant.

Humans can convince themselves of almost anything. So I don’t understand your point.

Re: A new AI winter is coming?

#286

Post by LatencyKills »

gishh wrote: post #222
LatencyKills wrote: post #207

Earlier quoted context omitted.

> as someone who is a sociopath completely devoid of ethics Ah yes... the hundred thousand researchers and engineers who work at MS are all evil. Many people who've made truly significant contributions to AI have either worked directly (through MS Research) or indirectly (OpenAI, Anthropic, etc) at MS. ResNet and concepts like Differential Privacy were invented there. What about the researchers at Stanford, Carnegie…

Discover, not invent

Excuse me? Algorithms are invented. The transformer architecture was invented.

The transformer was not some aspect of nature that was waiting to be discovered. The human mind created it.

Re: A new AI winter is coming?

#287

Post by emp17344 »

Eddy_Viscosity2 wrote: post #277
emp17344 wrote: post #141

Earlier quoted context omitted.

Except we know for a fact that the brain doesn’t work that way. You’re ignoring the entire history of neuroscience.

I didn't know that neuroscience even claimed to a full enough understanding of how people think to conclusively disprove this.

Certainly not a full understanding, but there’s enough understanding to refute vague claims that we are somehow analogous to a text model.

Re: A new AI winter is coming?

#288

Post by code_biologist »

lamontcg wrote: post #167
stanfordkid wrote: post #16

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal model…

> that enables a computer to pretty effectively understand natural language I'd argue that it pretty effectively mimics natural language. I don't think it really understands anything, it is just the best madlibs generator that the world has ever seen. For many tasks, this is accurate 99+% of the time, and the failure cases may not matter. Most humans don't perform any better, and arguably regurgitate words without un…

Break the aspects of language understanding and language generation apart. While I would agree that generative LLMs are understanding-free madlibs for writing text, embedding vector spaces and LLM latent spaces seem are a pretty genuine understanding of natural language. High dimensional vector spaces seem like the best machine representation we currently have for meaning and LLMs are using it effectively.

Re: A new AI winter is coming?

#289

Post by encyclopedism »

lambdaone wrote: post #279
encyclopedism wrote: post #272

Earlier quoted context omitted.

> The fact is: something fundamental has changed that enables a computer to pretty effectively understand natural language. I have commented elsewhere but this bears repeating LLM's do not think, understand, reason, reflect, comprehend and they never shall. If you had enough paper and ink and the patience to go through it, you could take all the training data and manually step through and train the same model. Then o…

You could use a similar argument to argue that "brains do not think, understand, reason, reflect, comprehend and they never shall." After all, there's nothing in there but neurons, synapses and other biological gunk. If you look at it that way. Which you shouldn't.

That argument you posed does not follow. Brains do all of those things, at least I know mine does because those are the most intimate experiences I have. More intimate than even my sense experiences. It's what lead Kant to exclaim 'cogito ergo sum' and Ibn Sina before him to also note that fact.

Moreover how the brain does what it does an open academic question. One of the most difficult, please see the Hard problem of consciousness for example.

I can make certain determined judgments about things, my digital thermometer does not think or understand when it tells me the temperature, something I myself would be unable to determine. My digital LLM does not think for the same reason. Importantly my paper and pen version of that very same LLM would also not think.

Re: A new AI winter is coming?

#290

Post by sph »

swalsh wrote: post #64
cmiles8 wrote: post #31

LLMs are an amazing advancement. The tech side of things is very impressive. Credit where credit is due. Where the current wave all falls apart is on the financials. None of that makes any sense and there’s no obvious path forward. Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Tech wise I’…

Hard disagree, I'm in the process of deploying several AI solutions in Healthcare. We have a process a nurse usually spends about an hour on, and costs $40-$70 depending on if they are offshore and a few other factors. Our AI can match it at a few dollars often less. A nuse still reviews the output, but its way less time. The economics of those tokens is great. We have another solution that just finds money, $10-$30…

I hope someone develops an AI that can do your job at a few dollars often less. That would be great, wouldn’t it? The economics of those tokens is great. It would be a solution that just finds money.

Of course, you can still be on the loop to double check its work, but no worries, you can do it part time.

Post reply on HN