Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI isn’t good enough

skventures.substack.com

Post reply on HN
Print view View original

201–210 of 374 posts

Re: AI isn’t good enough

#201

Post by Philpax »

RandomLensman wrote: post #195
auggierose wrote: post #192

Earlier quoted context omitted.

On their own, yes. But if you have an application where you can check the correctness of what they come up with, you are golden. Which is often the case in the hard sciences. It's almost like we need our AI's to have two brain parts. A fast one, for intuition, and a slow one, for correctness. ;-)

Unclear to me. The economics might not be so great as you might need (i) expensive people, (ii) there could be a lot to check for correctness, and (iii) checking could involve expensive things beyond people. Net productivity might not go up much then. For some industries where I understand the cost stacks with lower and higher skilled workers, I'd say it only takes out the "cheap" part and thereby not taking out a la…

My interpretation of the parent post is not that LLMs' output should be checked by humans, or that they are used in domains where physical verification is expensive; no, what they're suggesting is using a secondary non-stochastic AI system/verification solution to check the LLM's results and act as a source of truth.

An example that exists today would be the combination of ChatGPT and Wolfram [1], in which ChatGPT can provide the method and Wolfram can provide the execution. This approach can be used with other systems for other domains, and we've only just started scratching the surface.

[1] https://www.wolfram.com/wolfram-plugin-chatgpt/

Re: AI isn’t good enough

#202

Post by burnished »

worrycue wrote: post #143
lucubratory wrote: post #141

Earlier quoted context omitted.

Yes, I have noticed that a lot of extreme AI cynics have been arguing that any and every example of reasoning or thinking that an LLM displays is just some variant of memorisation.

The biggest evidence that LLMs can’t reason is hallucinations. If it could reason it would have rejected fictional generated output that make no sense.

Wouldn't that be evidence at most that its reasoning was flawed, or could contain errors?

Re: AI isn’t good enough

#203

Post by Philpax »

stormfather wrote: post #198
lewhoo wrote: post #184

Earlier quoted context omitted.

A system that could perfectly predict what I would do in response to any particular stimuli, as a continuing sequence, would be exactly as intelligent as me. That's certainly interesting but it's not a depiction of a LLM is it ? LLM's are not deterministic, and (perhaps) so are we so two non-deterministic systems can only occasionally align (or so I assume). Intuition says they may get "close enough", whatever that m…

LLMs are deterministic if the temperature parameter is set to 0. Randomness is artificially injected into their outputs otherwise in order to make them more interesting, but they're just a series of math operations.

To elucidate on this: the LLM can be viewed as a function that takes the context as an attachment and produces a probability distribution for the next token over all known tokens.

Most inference samples from that distribution using a composition of sampling rules or such, but there's nothing stopping you from just always taking the most probable token (temperature = 0) and being fully deterministic. The results are quite bland, but it's perfect for extraction tasks.

(note: GPT-4 is not fully deterministic; there's no details on this but the running theory is it is a mixture of experts model and that their expert routing algorithm is not deterministic/is dependent on the resources available)

Re: AI isn’t good enough

#204

Post by FrustratedMonky »

jug wrote: post #117
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

There has actually been research that found that there are strong diminishing returns in terms of at least expanding parameter sizes. While I think there are still breakthroughs to be made in terms of window sizes and workarounds like Mixture of Experts, I'm not sure how much farther we will get here in the long term in terms of raw performance of the LLM itself. FWIW, Sam Altman agrees and has a surprisingly similar…

How can research predict this wave is coming to an end, when research also didn't think this wave would happen either. It seems like there are always people saying 'it can't be done'. Then it happens. If there was a way to predict the future, then wouldn't that research need to know how something would be implemented, in order to know it can't be?

Re: AI isn’t good enough

#205

Post by abdussamit »

I haven't read the article (like many here) and I don't get why the writer would write this. LLMs/AI are not a stage where they can replace people or their jobs (broadly speaking), they still have to be taught and engineered to solve your problems. For example, if I enter a prompt "Write a Javascript function that does XYZ for me.", and don't get a desired result, it's unfair for me to say "ChatGPT bad, AI bad".

At this moment, you have to work with the limits, use better prompts, use ChatGPT as a guide and not as your personal robot. With that approach, I feel lots of revolutionary content is incoming in products. Use the tool better to get the most out of it.

Re: AI isn’t good enough

#206

Post by arrow7000 »

ChatGTP wrote: post #185
jillesvangurp wrote: post #112

By the time AIs stop hallucinating, it will be effectively super human (i.e. much better) because humans hallucinate all the time. And some people are barely coherent. We're holding AIs to a much higher standard than ourselves. And we move the goal posts all the time as well. Let's deconstruct that title. "AI isn't good enough". Good enough for what? Great example of moving the goal posts. Because anytime it fails to…

"Guy calls humans flawed why claiming the same flawed species will build superhuman intelligence...", pretty good for a bunch of dumb LLMs in "meat bags"...

You're pointing out the best that humanity is capable of. He's pointing out that people judge LLMs by their worst performances. You're not disagreeing with the parent comment.

Re: AI isn’t good enough

#207

Post by FrustratedMonky »

nine_k wrote: post #137
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

The current crop of LLMs is like Paleozoic megafauna, or like Egyptian Pyramids. It takes a few relatively simple approaches and stretches them wildly, using colossal computing resources. Live systems in nature seem to solve similar problems with way less compute available. There should be better architectures. Also, as somebody said, every exponential growth curve is a lower part of a sigmoid. LLMs will plateau at s…

Everyone has tunnel vision on LLM's like they are the only thing just because they are the latest big shiny thing.

"oh, they are at their limit". People were saying they were at their limit 5 years ago too.

There are a lot of AI techniques, some more advanced than LLM, just with other limitations so they don't look 'amazing'.

Seems like engineering at this point to tie them together for the next 'big thing'.

Re: AI isn’t good enough

#208

Post by vidarh »

ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

There's a second assumption: That "worker displacement" remains an unsolved problem. That assumes a society that chooses to do nothing to in the face of reducing the need for labour, turning something that could be substantial boon into a downside.

In other words: It assumes Marx was right about the central failure-mode of capitalism (an inability or unwillingness to find ways to distribute work and proceeds of work in ways that prevents productivity gains from eventually causing social upheaval).

There's no reason what they describe as "so-so" tech can not also be a significant societal advantage, but it requires structural change to ensure it does not deprive a significant number of people of a livelihood.

Re: AI isn’t good enough

#209

Post by andrepd »

Kiro wrote: post #69
andrepd wrote: post #39

Earlier quoted context omitted.

Why is it always the same reply? Yes, GPT4 is as useless as GPT3.5 on any non trivial task.

> Why is it always the same reply? When there's one definitive answer to something that people keep repeating there's a slight chance that it's actually true. Shocking, I know.

Well but it's not. Also, your reasoning would imply e.g. Qanon is true based on the number of people repeating the same thing, so that's pretty weak.

Re: AI isn’t good enough

#210

Post by andrepd »

sebzim4500 wrote: post #191
andrepd wrote: post #39

Earlier quoted context omitted.

Why is it always the same reply? Yes, GPT4 is as useless as GPT3.5 on any non trivial task.

>Why is it always the same reply? I keep going around telling people that 1+1=3, why do they always give me the same nonsense about the number '2'? I blame Sam Altman.

?
Post reply on HN