Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I genuinely don't understand why some people are still bullish about LLMs

twitter.com

Post reply on HN
Print view View original

681–690 of 1001 posts

Re: I genuinely don't understand why some people are still bullish about LLMs

#681

Post by DeathArrow »

I don't trust Sabine Hossenfelder on this. Even I as a computer scientist/programmer with just some old experience from my master courses towards AI and ML, I know much more than her about how the things work.

She became more of an influencer than a scientist. And that is nothing wrong with that unless she doesn't try to pose as an authority on subjects she doesn't have a clue. It's OK to have an opinion as an outsider but it's not OK to pretend you are right and that you are an expert on every scientific or technical subject that happens to make you want to make a tweet about.

Re: I genuinely don't understand why some people are still bullish about LLMs

#682

Post by Terretta »

WhyOhWhyQ wrote: post #646
gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

The technology is not just less than superintelligence, for many applications it is less than prior forms of intelligence like traditional search and Stack Exchange, which were easily accessible 3 years ago and are in the process of being displaced by LLMs. I find that outcome unimpressive. And this Tweeter's complaints do not sound like a demand for superintelligence. They sound like a demand for something far more…

"They continue to fabricate links, references, and quotes, like they did from day one." - "I ask them to give me a source for an alleged quote, I click on the link, it returns a 404 error."

Why have these companies not manually engineered out a problem like this by now? Just do a check to make sure links are real. That's pretty unimpressive to me.

There are no fabricated links, references, or quotes, in OpenAI's GPT 4.5 + Deep Research.

It's unfortunate the cost of a Deep Research bespoke white paper is so high. That mode is phenomenal for pre-work domain research. You get an analyst's two week writeup in under 20 minutes, for the low cost of $200/month (though I've seen estimates that white paper cost OpenAI over USD 3000 to produce for you, which explains the monthly limits).

You still need to be a domain expert to make use of this, just as you need to be to make use of an analyst. Both the analyst and Deep Research can generate flawed writeups with similar misunderstandings: mis-synthesizing, misapplication, or missing inclusion of some essential.

Neither analyst nor LLM is a substitute for mastery.

Re: I genuinely don't understand why some people are still bullish about LLMs

#683

Post by tiahura »

adastra22 wrote: post #390
ThrowawayTestr wrote: post #339

Earlier quoted context omitted.

Surely you understand why an LLM that has no knowledge of your niche wouldn't be useful right?

Hallucinating incorrect information is worse than useless. It is actively harmful. I wonder how much this affects our fundraising, for example. No VC understands the science here, so they turn to advisors (which is great!) or to LLMs… which has us starting off on the wrong foot.

Good thing humans never make mistakes.

Re: I genuinely don't understand why some people are still bullish about LLMs

#684

Post by brookst »

Thlom wrote: post #669
WhyOhWhyQ wrote: post #646

Earlier quoted context omitted.

The technology is not just less than superintelligence, for many applications it is less than prior forms of intelligence like traditional search and Stack Exchange, which were easily accessible 3 years ago and are in the process of being displaced by LLMs. I find that outcome unimpressive. And this Tweeter's complaints do not sound like a demand for superintelligence. They sound like a demand for something far more…

A municipality in Norway used LLM to create a report about the school structure in the municipality (how many schools are there, how many should there be, where should they be, how big should they be, pros and cons of different size schools and classes etc etc). Turns out the LLM invented scientific papers to use as references and the whole report is complete and utter garbage based on hallucinations.

And that says… what? The entire LLM technology is worthless for all applications, from all implementations?

A company I worked for spent millions on a customer service solution that never worked. I wouldn’t say that contracted software is useless.

Re: I genuinely don't understand why some people are still bullish about LLMs

#685

Post by brookst »

jibal wrote: post #626
somenameforme wrote: post #554

Earlier quoted context omitted.

It's mostly because of how they were initially marketed. In an effort to drive hype 'we' were promised the world. Remember the "leaks" from Google about an engineer trying to get the word out that they had created a sentient intelligence? In reality Bard, let alone whatever early version he was using, is about as sentient as my left asscheek. OpenAI did similar things by focusing to the point of absurdity on 'safety'…

> Remember the "leaks" from Google about an engineer trying to get the word out that they had created a sentient intelligence? That's not what happened. Google stomped hard on Lemoine, saying clearly that he was wrong about LaMDA being sentient ... and then they fired him for leaking the transcripts. Your whole argument here is based on false information and faulty logic.

Which is pretty ironic in a thread littered with people asserting LLMs are useless because they can hallucinate and create illogical outputs.

Re: I genuinely don't understand why some people are still bullish about LLMs

#687

Post by causality0 »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

Yeah the amount of "piffle work" that LLMs save me is astounding. Sure, I can look up fifty different numbers and copy them into excel. Or I can just tell an LLM "make a chart comparing measurements XYZ across devices ABC" and I'm looking at the info right there.

Re: I genuinely don't understand why some people are still bullish about LLMs

#688

Post by brookst »

hatefulmoron wrote: post #645
jibal wrote: post #626

Earlier quoted context omitted.

> Remember the "leaks" from Google about an engineer trying to get the word out that they had created a sentient intelligence? That's not what happened. Google stomped hard on Lemoine, saying clearly that he was wrong about LaMDA being sentient ... and then they fired him for leaking the transcripts. Your whole argument here is based on false information and faulty logic.

I think you're missing the point. He's pointing out what the atmosphere was/is around LLMs in these discussions, and how that impacts stories like with Lemoine. I mean, you're right that he's silly and Google didn't want to be part of it, but it was (and is?) taken seriously that: LLMs are nascent AGI, companies are pouring money to get there first, we might be a year or two away. Take these as true, it's at least po…

Sure, but does any of that support the claim that LLMs were marketed as superintelligence?

Re: I genuinely don't understand why some people are still bullish about LLMs

#689

Post by trashtester »

inopinatus wrote: post #290
Animats wrote: post #178

Earlier quoted context omitted.

> inbuilt into the models of all LLMs is that they'll fabricate a lot. Still the elephant in the room. We need an AI technology that can output "don't know" when appropriate. How's that coming along?

Unfortunately they are trained first and foremost as plausibility engines. The central dogma is that plausibility will (with continuing progress & scale) converge towards correctness, or "faithfulness" as it's sometimes called in the literature. This remains very far from proven. The null hypothesis that would be necessary to reject, therefore, is a most unfortunate one, viz. that by training for plausibility we are…

If the training data had a lot of humans saying "I don't know", then the LLM's would too.

Humans don't and LLM's are essentially trained to resemble most humans.

Re: I genuinely don't understand why some people are still bullish about LLMs

#690

Post by brookst »

emsign wrote: post #564
somenameforme wrote: post #554

Earlier quoted context omitted.

It's mostly because of how they were initially marketed. In an effort to drive hype 'we' were promised the world. Remember the "leaks" from Google about an engineer trying to get the word out that they had created a sentient intelligence? In reality Bard, let alone whatever early version he was using, is about as sentient as my left asscheek. OpenAI did similar things by focusing to the point of absurdity on 'safety'…

They have their value in analyzing huge amounts of data for example scientific papers or raw observations, but the popular public ones are mostly trained on stolen/pirated texts offthe internet and from social media clouds the companies control. So this means: bullshit in -> bullshit out. I don't need machines for that the regular human bullshitters do this job just fine.

A lot of irony in that comment.
Post reply on HN