Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI isn’t good enough

skventures.substack.com

Post reply on HN
Print view View original

41–50 of 374 posts

Re: AI isn’t good enough

#41

Post by jstummbillig »

user3939382 wrote: post #27
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

I asked ChatGPT how to add a JSDoc type to a Vue 2 prop and it gave me a wrong answer. There have been several times where I’ve asked it questions and it sprinkles in well disguised misinformation. These tools are impressive but they definitely have limitations.

Everything has limitations. The linked piece is very much a statement about limited potential, not the current state. In this context, this stance is akin to saying "I asked a 6yo this math question and it gave me a wrong answer, there's definitely limitations". Sure – but who would use that observation to assess where the future potential cap of the kid is?

Re: AI isn’t good enough

#42

Post by intended »

astrange wrote: post #23
tuatoru wrote: post #16

Earlier quoted context omitted.

Just catching up to years of undershooting inflation. In real terms still worse than 1969.

Definitely not. You wouldn't like the food, cars and home appliances of 1969, and you really wouldn't like the healthcare.

Apples and oranges.

You would like a home from 1969.

You would like the salary saved from 1969.

Re: AI isn’t good enough

#43

Post by coldtea »

jasfi wrote: post #37
user3939382 wrote: post #27

Earlier quoted context omitted.

I asked ChatGPT how to add a JSDoc type to a Vue 2 prop and it gave me a wrong answer. There have been several times where I’ve asked it questions and it sprinkles in well disguised misinformation. These tools are impressive but they definitely have limitations.

Are you using GPT-4? If not, it's understandable. If you don't pay for ChatGPT, you get GPT-3.5. You can also get access to GPT-4 if you use the playground.

I'm using GTP-4 and it makes trivial errors all the time.

I asked it (actual names changed):

"I run the Linux command line program "foo". When I use the flags -xyz, I get results, but when I use -txyz I get nothing. What could this mean?"

And it told me: "The lack of results is because you didn't use the -t flag".

Or I ask it some very basic music theory questions and it gets stuff wrong all the time, giving impossible answers.

Re: AI isn’t good enough

#44

Post by dontupvoteme »

ryanklee wrote: post #2

"We are quickly reaching the limits of current AI" So many words in this piece, but nothing to back up this primary assertion.

This, how absurd. Do they think the future is always going to be in "just add more parameters"?

It's obviously going to be smaller, efficient and such. Even GPT4 itself is supposedly along these lines with MoE

Re: AI isn’t good enough

#45

Post by dontupvoteme »

system2 wrote: post #36
ryanklee wrote: post #2

"We are quickly reaching the limits of current AI" So many words in this piece, but nothing to back up this primary assertion.

For what it's worth, I can actually tell the hype around AI died down a little. 8-10 months ago you'd hear AI from grandmas and their interest in playing with gpt. That's gone. From programmer's perspective, it is still useless. I can't still get chatgpt to give me a useful ffmpeg or a basic php database script. I don't know who is really using ai for what, other than crappy chatbots. They are saying JSON works perfe…

Really? I have a lot of success with bash and ffmpeg scripts. I don't even try to write them myself these days.

I don't do JSON stuff much but either function calling or grammar based sampling should be able to force correct formatting if the issue is syntax.

ImageGen seemed like a solved problem to me with stable diffusion about 6 months ago or so, between LoRAs and the tons of models availble on sites like civitas..

SOTA Text to speech/voice cloning is scary good, but it's only available via SaaS like 11labs (perhaps just them).

One interesting weekend they allowed anyone to clone a voice (required only maybe 10-60s of audio) to say anything

I made Bob Ross read Neuromancer using 60 seconds of fuzzy audio from youtube, but 4chan made something like 20k fake audio recordings on vocaroo of everything horrible and hilarious you can think of. Then they shut that off for good reason on Monday.

There are some cool applications there. I watched an hour video about the talkshow wars between Leno and Conan and i couldn't figure out what the voice was but it sounded like a smart old guy, apparently it was a weird clone of Biden. Which means anyone can now make YouTube content with any kind of voice. Which is bad for impersonation, but good if, say, you don't like the sound of your voice and want to go for a different one (maybe an old timey radio broadcaster)

What definitely has become shittier is the quality of gpt4. But that's not an AI issue, that's OpenAI being OpenAI.

Re: AI isn’t good enough

#46

Post by Tanjreeve »

astrange wrote: post #23
tuatoru wrote: post #16

Earlier quoted context omitted.

Just catching up to years of undershooting inflation. In real terms still worse than 1969.

Definitely not. You wouldn't like the food, cars and home appliances of 1969, and you really wouldn't like the healthcare.

Why do you think there is a zero sum choice between real terms purchasing power increases and the technology we have today? (Also worth noting I'd happily take a lot of goods from a time where they were built to last)

Re: AI isn’t good enough

#47

Post by ryoshu »

user3939382 wrote: post #27
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

I asked ChatGPT how to add a JSDoc type to a Vue 2 prop and it gave me a wrong answer. There have been several times where I’ve asked it questions and it sprinkles in well disguised misinformation. These tools are impressive but they definitely have limitations.

True. I can't do a full table scan on an non-indexed column in a RDBMS with billions of rows on a single machine without it grinding to a halt.

Tools have limitations.

Re: AI isn’t good enough

#48

Post by nmca »

This is nuanced and interesting, despite most likely being completely wrong - as others have mentioned, it's predicted on LLMs being at the top of the sigmoid and they just ain't.

Also, this goal-post-moving did crack me up rather:

> Sure, it is terrific at using its statistical models to come up with textual passages that read better than the average human’s writing, but that’s not a particularly high hurdle.

Re: AI isn’t good enough

#49

Post by dontupvoteme »

intended wrote: post #35
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

I can take a bet that it haha already failed - the hype cycle has already made a promise that LLMs can’t keep. Hallucinations to the normal person are a bug. The issue is that only humans can hallucinate. We know there is a “reality”. For an LLM, everything it does is a hallucination. That’s why you have more POCs than production goods. Your “hallucination rate” is unknown. Yesterday Ars has an article that described…

Why not "simply" multigen every (important) query and take the statistical average?

Hallucinations are random, the truth isn't.

This is absurdly expensive with GPT4, cheaper with 3, and dirt cheap locally with LLaMA

Re: AI isn’t good enough

#50

Post by dontupvoteme »

user3939382 wrote: post #27
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

I asked ChatGPT how to add a JSDoc type to a Vue 2 prop and it gave me a wrong answer. There have been several times where I’ve asked it questions and it sprinkles in well disguised misinformation. These tools are impressive but they definitely have limitations.

You're asking a model which is just a big unfocused and highly alignment taxed mush of the entire internet across all language, regardless of if its natural or computer, then (maybe) sliced across 8 experts.

It can also simulate a Zizek vs Wittgenstein argument over Russian literature. the fact that it can usually write executing computer code is nothing short of magic to me.

What one would want is something like LLaMA2-Coding-Vue2, maybe with a LoRA for the library or concept.

I wrote a big word salad about all the other things you could do but suffice it to say there's no reason anyone should be limited to a single pass on a monolithic without automatic context window augmentation nor automatic code checking & regeneration & model escalation (e.g. query out to a 200B coding model or something).

Post reply on HN