Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I genuinely don't understand why some people are still bullish about LLMs

twitter.com

Post reply on HN
Print view View original

291–300 of 1001 posts

Re: I genuinely don't understand why some people are still bullish about LLMs

#291

Post by aetherson »

snitty wrote: post #286
gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

Folks really over index when an LLM is very good for their use case. And most of the folks here are coders, at which they're already good and getting better. For some tasks they're still next to useless, and people who do those tasks understandably don't get the hype. Tell a lab biologist or chemist to use an LLM to help them with their work and they'll get very little useful out of it. Ask an attorney to use it and…

I know at least two attorneys who use LLMs productively.

As with all LLM usage right now, it's a tool and not fit for every purpose. But it has legit uses for some attorney tasks.

Re: I genuinely don't understand why some people are still bullish about LLMs

#292

Post by biophysboy »

maple3142 wrote: post #171

I think many people are just not really good at dealing with "imperfect" tools. Different tools can have different success probability, let's call that probability p here. People typically use tool that have p=100%, or at least very close to it. But LLM is a tool that is far from that, so making use of it takes different approach. Imagine there is an probabilistic oracle that can answer any question with a yes/no wit…

Your point reminded me of Terrence Tao’s point that AI has a “plausibility problem”. When it can’t be accurate, it still disguises itself as accurate.

Its true success rate is by no means 100%, and sometimes is 0%, but it always tries to make you feel confident.

I’ve had to catch myself surrendering too much judgment to it. I worry a high school kid learning to write will have fewer qualms surrendering judgment

Re: I genuinely don't understand why some people are still bullish about LLMs

#293

Post by snitty »

jandrewrogers wrote: post #282

My anecdotal experience is similar. For any important or hard technical questions relevant to anything I do, the LLM results are consistently trash. And if you are an expert in the domain you can’t not notice this. On the other hand, for trivial technical problems with well known solutions, LLMs are great. But those are in many senses the low value problems; you can throw human bodies against that question cheaply. A…

>For any important or hard technical questions relevant to anything I do, the LLM results are consistently trash. And if you are an expert in the domain you can’t not notice this.

This is also my experience. My day job isn't programming, but when I can feed an LLM secretarial work, or simple coding prompts to automate some work, it does great and saves me time.

Most of my day is spent getting into the details on things for which there's no real precedent. Or if there is, it hasn't been widely published on. LLMs are frustrating useless for these problems.

Re: I genuinely don't understand why some people are still bullish about LLMs

#295

Post by strangattractor »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

Back when handwriting recognition was a new thing I was greatly impressed by how good it was. This was primarily because being an engineer I knew how difficult the problem is to solve. %90 recognition seemed really good to me.

When I tried to use the technology that %90 meant 1 out of every 10 things I wrote were incorrect. If it had been a keyboard I would have thrown it in the trash. That is were my Palm ended up.

People expect their technology to do things better not almost as well as a human. Waymo with LIDAR hasn't killed people. Tesla, with camera only, has done so multiple times. I will ride in a Waymo never in a Tesla self driving car.

Re: I genuinely don't understand why some people are still bullish about LLMs

#296

Post by codegangsta »

I think there are multiple conversations happening that are tying to converge on one.

On one hand, LLMs are overhyped and not delivering on promises made by their biggest advocates.

On the other hand, any other type of technology (not so overhyped) would be massively celebrated in significantly improving a subset of niche problems.

It’s worth acknowledging that LLMs do solve a good set of problems well, while also being overhyped as a silver bullet by folks who are generally really excited about its potential.

Reality is that none of us know what the future is, and whether LLMs will have enough breakthroughs to solve more problems then today, but what they do solve today is still very impressive as is.

Re: I genuinely don't understand why some people are still bullish about LLMs

#297

Post by jandrewrogers »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

TBH, they produce trash results for almost any question I might want to ask them. This is consistently the case. I must use them differently than other people.

LLMs produce midwit answers. If you are an expert in your domain, the results are kind of what you would expect for someone who isn’t an expert. That is occasionally useful but if I wanted a mediocre solution in software I’d use the average library. No LLM I have ever used has delivered an expert answer in software. And that is where all the value is.

I worked in AI for a long time, I like the idea. But LLMs are seemingly incapable of replacing anything of value currently.

The elephant in the room is that there is no training data for the valuable skills. If you have to rely on training data to be useful, LLMs will be of limited use.

Re: I genuinely don't understand why some people are still bullish about LLMs

#298

Post by bachmeier »

snitty wrote: post #286
gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

Folks really over index when an LLM is very good for their use case. And most of the folks here are coders, at which they're already good and getting better. For some tasks they're still next to useless, and people who do those tasks understandably don't get the hype. Tell a lab biologist or chemist to use an LLM to help them with their work and they'll get very little useful out of it. Ask an attorney to use it and…

Strongly in agreement. I've tried them and mostly come away unimpressed. If you work in a field where you have to get things right, and it's more work to double check and then fix everything done by the LLM, they're worse than useless. Sure, I've seen a few cases where they have value, but they're not much of my job. Cool is not the same as valuable.

If you think "it can't quite do what I need, I'll wait a little longer until it can" you may still be waiting 50 years from now.

Re: I genuinely don't understand why some people are still bullish about LLMs

#299

Post by t917910 »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

Perhaps you have already paid off your mortgage and saved up a million dollars for retirement? And you're not threatened by dismissal or salary reduction because supposedly "AI will replace everyone."

By the way, you don't need to be a 50+ year old nerd. Nerds are a special culture-pen where smart straight-A students from schools are placed so they can work, increase stakeholder revenues, and not even accidentally be able to do anything truly worthwhile that could redistribute wealth in society.

Re: I genuinely don't understand why some people are still bullish about LLMs

#300

Post by iambateman »

I think people confuse the power of the technology with the very real bubble we’re living in.

There’s no question that we’re in a bubble which will eventually subside, probably in a “dot com” bust kind of way.

But let me tell you…last month I sent several hundred million requests to AI, as a single developer, and got exactly what I needed.

Three things are happening at once in this industry… (1) executives are over promising a literal unicorn with AGI, that is totally unnecessary for the ongoing viability of LLM’s and is pumping the bubble. (2) the technology is improving and delivery costs are changing as we figure out what works and who will pay. (3) the industry’s instincts are developing, so it’s common for people to think “AI” can do something it absolutely cannot do today.

But again…as one guy, for a few thousand dollars, I sent hundreds of millions of requests to AI that are generating a lot of value for me and my team.

Our instincts have a long way to go before we’ve collectively internalized the fact that one person can do that.

Post reply on HN