Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI isn’t good enough

skventures.substack.com

Post reply on HN
Print view View original

111–120 of 374 posts

Re: AI isn’t good enough

#111

Post by roenxi »

usrbinbash wrote: post #67
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

I'm put in mind of the OpenAI DoTA bot that was winning 99% of its games and some people refused to admit that it knew how to play DoTA based on some esoteric interpretation of the word "play".

We're going to see exponential increases in processing power of the best GPU clusters and human brains are a stationary target. And there is precious little evidence that the average human is much more than an LLM. LLMs are already more likely to understand a topic to a high standard than a given human.

They're going to progress and if they aren't intelligent then intelligence is overrated and I'd rather have whatever they have.

Re: AI isn’t good enough

#112

Post by jillesvangurp »

By the time AIs stop hallucinating, it will be effectively super human (i.e. much better) because humans hallucinate all the time. And some people are barely coherent. We're holding AIs to a much higher standard than ourselves. And we move the goal posts all the time as well.

Let's deconstruct that title. "AI isn't good enough". Good enough for what? Great example of moving the goal posts. Because anytime it fails to do whatever, it's not good enough. But what about all the stuff it is good enough for that people are actually using it for already? It's passing tests at a level most humans only manage very briefly after lots of intensive preparation. The knowledge slips away quickly after that.

The way I see it, there's a long, rapidly growing list of stuff that AIs have nailed already and a list of things where it is clearly struggling. That list is shrinking.

Self driving gets cited a lot as something where AIs is failing. It's hard to say that in a world where multiple companies are now operating self driving vehicles commercially in several cities across the US, China, and soon Europe. Are they perfect? No. Do they drive better than the average jerk on the road? Definitely; not even close. Most people simply suck at driving. That's why traffic deaths are so common. Most of those deaths are caused by somebody with a drivers license proving that they shouldn't have received one. Not quite a solved problem but at this point the question of when these things will be on the road in my city is more a matter of when than if. It already works in other cities. I don't see what's so special in mine that it couldn't be replicated here. In other words, I expect the number of cities with autonomous vehicles to start exploding over the next few years. And who knows, Tesla might hit the good enough mark at some point as well. Again, the barrier is very low because humans aren't held to the same standards. They'll give drivers licenses to just about anyone.

Re: AI isn’t good enough

#113

Post by ChatGTP »

savolai wrote: post #101
beefield wrote: post #78

Earlier quoted context omitted.

> LLMs won't get intelligent. That's a fact based on their MO. I kind of agree. However, I see a real possibility that in the near future LLM behaviour would be practically indistinguishable from intelligent/sentient behavior. And at that point we (or at least I) are facing some really interesting/difficult questions, namely how do you know an intelligent looking thing actually is intelligent (or sentient). How do yo…

It seems to me that the real question here is what is true human intelligence. Ai has made it plain to see, by being able to replicate it so convincingly, that much of what we have considered intelligence has been pattern matching or acting as complex parrots. There is much more to the abilities of human body-mind-emotional-experiential being, but it is only slowly becoming mainstream. (Edit: Of course there are also…

I'm getting suspicious that there is a bit of a blind spot in understanding the world and the usefulness of what we call intelligence, as Noval Yuah Harari says, intelligence is overrated. Look at what we've done to the planet and our environment we have fucked it properly, yet we consider ourselves to be intelligent?

Could it be that intelligence is overrated and discovery of new ideas / thing is underrated? Our egos tell us it's intelligence that makes us special and creative and awesome but maybe most of the special stuff is already there for us to find and we conflate discovery with extrapolation. Maybe knowledge and experience are the "important bits" of intellect.

Example: Einstein didn't really invent anything, he discovered things about the world that blew our mind and changes our lives. Yes he was a great thinker and a courageous soul to go against the grain and he had the balls to be open minded enough to discover new things. We obviously believe Einstein to be intelligent but was he just a great explorer ?

I have a similar attitude towards technological progress, yes we've done amazing things but fundamentally the air we breathe, the water we drink and the beauty we are subjected too when looking at a sunset are taken for granted while we stare at our phones.

Re: AI isn’t good enough

#114

Post by alpaca128 »

usrbinbash wrote: post #67
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

> LLMs won't get intelligent

Even assuming that is true: LLMs aren't all that exists in AI research and just like LLMs are amazing in terms of language it's possible similar breakthroughs could be made in more abstracted areas that could use LLMs for IO.

If you think ChatGPT is nice, wait for ChatGPT as frontend for another AI that doesn't have to spend a single CPU cycle on language.

Re: AI isn’t good enough

#115

Post by glenstein »

usrbinbash wrote: post #67
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

>Which is countered by...the assertion that it won't?

No it's countered by principled restraint in not making an affirmative claim one way or the other.

I've heard this referred to as the overconfident pessimism problem. Which is that normal, well founded scientific discipline and evidence-based restraint go out of the window when people declare, without evidence that they know certain advances won't happen.

Because people get mentally trapped into this framing of either have to declare that it will happen or that it won't, seeming to forget that you can just adopt the position of modesty and say the dust hasn't yet settled.

Re: AI isn’t good enough

#116

Post by HPsquared »

dontupvoteme wrote: post #61
cbozeman wrote: post #32

Earlier quoted context omitted.

You've identified the problem inherent in these models right now, though. There's a lot of "extraneous" information and details that appear useless and unrelated, but that long-time developers have tucked away in their brains (or really anyone who has done something at a high level for a long time), that turns out to be incredibly useful; generally these people don't require examples - they just know what the right a…

That's where good money and societal upheaval will come up, private models for corporations trained on their entire codebase and documentation. Both young and old shall feel the pain then.

The codebase and documentation usually don't capture the full context of the development process. There is still a lot that isn't written down.

Re: AI isn’t good enough

#117

Post by jug »

ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

There has actually been research that found that there are strong diminishing returns in terms of at least expanding parameter sizes. While I think there are still breakthroughs to be made in terms of window sizes and workarounds like Mixture of Experts, I'm not sure how much farther we will get here in the long term in terms of raw performance of the LLM itself. FWIW, Sam Altman agrees and has a surprisingly similar quote as found here "I think we’re at the end of the era where it’s gonna be these giant models, and we’ll make them better in other ways": https://techcrunch.com/2023/04/14/sam-altman-size-of-llms-wo...

Re: AI isn’t good enough

#118

Post by SanderNL »

system2 wrote: post #36
ryanklee wrote: post #2

"We are quickly reaching the limits of current AI" So many words in this piece, but nothing to back up this primary assertion.

For what it's worth, I can actually tell the hype around AI died down a little. 8-10 months ago you'd hear AI from grandmas and their interest in playing with gpt. That's gone. From programmer's perspective, it is still useless. I can't still get chatgpt to give me a useful ffmpeg or a basic php database script. I don't know who is really using ai for what, other than crappy chatbots. They are saying JSON works perfe…

> Image generation is still terrible and useless

Image generation is thriving and now an indispensible part of the creative process. The tools are fresh and still evolving, but the impact even in this early stage is ridiculous. You can easily take rough sketches and have it spit out fully fleshed out colored prototypes. Depending on your perspective it's either an insane productivity-multiplier or scary as f-ck or both.

Look at https://civitai.com for a taste. Look around. You really believe this is terrible and useless? Most of these images came straight from SD with some minor tweaking, usually upscaling. I think it's quite something.

It's not just MidJourney, it's the entire ecosystem surrounding models like SD, ControlNet, etc. Some artists are swinging their fists, others shrug, but all are affected by it whether they like it or not and the damn thing is just getting started. SD came out in 2022. As a young aspiring artist I can imagine it's hard to not feel like this whole thing is a massive punch in the gut.

Like an experienced and highly-skilled artist, it's easy to shrug this off when you're an accomplished programmer. It can't replace me, right? Well, true. It's a gradual thing, but the impact it has on junior programmers is undeniable. What it means for our branch if leagues of young people lose interest in the meticulous acquiring of technical and often arcane knowledge through "manual programming" is anybody's guess. Maybe it's a good thing, maybe it's not.

> Text-to-speech neural still sucks big time

This is not my experience at all. I am actually practicing French with it and it can pick up my weird accent. It's ridiculous how good it has become in such a short time.

EDIT: Oh, sorry, I read it wrong. But in that case, the same thing applies. 11labs is insane and getting better. It fools many, many people on Youtube. The Bark project is also scary at times. This is just getting started.

> I can't still get chatgpt to give me a useful ffmpeg or a basic php database script.

I believe these issues are at least for a significant fraction caused by the severely castrated and surprisingly lacking UI efforts from OpenAI. Just free-form "talking" with an LLM might be a good way to get grandma to use it, but as a professional programmer (or professional anything, really) it's bordering on stupid.

The prompts and settings you use determine 90% of your success. OpenAI's interface gives you nothing. They just recently introduced something like "profiles", but it's amateurish at best. It makes me feel like programming without IDE (and Vi/Emacs).

Re: AI isn’t good enough

#119

Post by tlarkworthy »

glenstein wrote: post #115
usrbinbash wrote: post #67

Earlier quoted context omitted.

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

>Which is countered by...the assertion that it won't? No it's countered by principled restraint in not making an affirmative claim one way or the other. I've heard this referred to as the overconfident pessimism problem. Which is that normal, well founded scientific discipline and evidence-based restraint go out of the window when people declare, without evidence that they know certain advances won't happen. Because…

AI has been around the corner since the 1950s, this is the historical evidence for the pessimistic stance against over optimistic predictions.

LLMs are a huge stride forward, but AI does not progress like Moore's law. LLM have revealed a new wall. Combining multi agents is not working out as hoped.

Re: AI isn’t good enough

#120

Post by oceanplexian »

alpaca128 wrote: post #114
usrbinbash wrote: post #67

Earlier quoted context omitted.

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

> LLMs won't get intelligent Even assuming that is true: LLMs aren't all that exists in AI research and just like LLMs are amazing in terms of language it's possible similar breakthroughs could be made in more abstracted areas that could use LLMs for IO. If you think ChatGPT is nice, wait for ChatGPT as frontend for another AI that doesn't have to spend a single CPU cycle on language.

The next AI wave hasn't even started. Imagine an LLM the size of GPT-4 but it's trained on nothing but gene sequence completion.

All the models being used in academia are basically toys, none of those guys are running hardware at a scale that can even remotely touch Azure, Meta, etc, and right now there is a massive global shortage of GPU compute that's eventually going to clear up. We know models get A LOT better when they are scaled up and are fed more data, so why wouldn't the same be true for other problems besides text completion?

Post reply on HN