Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI isn’t good enough

skventures.substack.com

Post reply on HN
Print view View original

81–90 of 374 posts

Re: AI isn’t good enough

#81

Post by albertzeyer »

usrbinbash wrote: post #67
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

Look at the scaling laws.

We found that extrapolating the performance given a few data points with smaller models is actually very accurate. That's how they determined hyper parameters, by tuning them on multiple smaller scale models and then extrapolating. So far, all those predictions were quite good.

Together with a bigger model, we also need more data to get better performance. If we add video and audio to the text data, we have still a lot more data we can use, so this is also not really a problem.

It would be very unexpected that those scaling laws are suddenly not true anymore for the next order of magnitude in model and data size.

Re: AI isn’t good enough

#82

Post by golol »

usrbinbash wrote: post #67
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

>LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots.

This is absolutely wrong. There is nothing about their MO that stops them from being intelligent. Suppose I build a human LLM as follows: A random human expert is picked and he is shown the current context window. He is given 1 week to deliberate and then may choose the next word/token/character. Then you hook this human LLM into an auto-GPT style loop. There is no reason it couldn't operate with high intelligence on text data.

Not also that LLMs are not really about language at all anymore, the architectures can be used on any sequence data.

Right now we are compute limited. If compute was 100x cheaper we could have GPT-6, bring 100x bigger, we could have really large and complex agents using GPT-4 power models, or we could train on tupled text-video data of subtitles videos. Given the world model LLMs manage to learn out of text data, I am 100% certain that a sufficiently large transformer can learn a decent world model from text-video data. Then our agents could also have a good physical understanding.

Re: AI isn’t good enough

#83

Post by golol »

user3939382 wrote: post #27
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

I asked ChatGPT how to add a JSDoc type to a Vue 2 prop and it gave me a wrong answer. There have been several times where I’ve asked it questions and it sprinkles in well disguised misinformation. These tools are impressive but they definitely have limitations.

ChatGPT (3.5) is not state of the art

Re: AI isn’t good enough

#84

Post by thrwjud »

ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

Every deep learning tech had an exponential growth phase, followed by a slowdown followed by a platou the nothing could break until a fundamentally new architecture came along. People get excited about the first part, project it into the second and start companies by the time we're well into the third.

Re: AI isn’t good enough

#85

Post by coldtea »

dontupvoteme wrote: post #71
coldtea wrote: post #68

Earlier quoted context omitted.

When the "fine tuning" and focus on tooling begins, it usually means the "free" massive improvements have tapered off.

If it's anything like Stable Diffusion, it's actually where it becomes terrifyingly good at a specific thing. Like putting a field of artists out of a job, or copying a style so good that you can complete a persons piece before they do on a livestream. We're using a massive mush of the internet model which is taxed at 50% for alignment. That's going to be very dumb in the long run.

>Like putting a field of artists out of a job,

To be frank, between stock art and photos, pre-AI template based tools, and the massive oversupply of graphic design and photography work, the field was already massively redudant and kind of out of a job to begin with...

Re: AI isn’t good enough

#87

Post by squeaky-clean »

usrbinbash wrote: post #67
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

> LLMs won't get intelligent.

I think this sentence doesn't mean much unless we have a strict definition of what intelligence means.

Just today ChatGPT helped me solve a DNS issue that I would not have been able to solve on my own in one day, let alone an hour. I'd consider it already more intelligent than myself when it comes to DNS.

Re: AI isn’t good enough

#88

Post by thatjoeoverthr »

dontupvoteme wrote: post #49
intended wrote: post #35

Earlier quoted context omitted.

I can take a bet that it haha already failed - the hype cycle has already made a promise that LLMs can’t keep. Hallucinations to the normal person are a bug. The issue is that only humans can hallucinate. We know there is a “reality”. For an LLM, everything it does is a hallucination. That’s why you have more POCs than production goods. Your “hallucination rate” is unknown. Yesterday Ars has an article that described…

Why not "simply" multigen every (important) query and take the statistical average? Hallucinations are random, the truth isn't. This is absurdly expensive with GPT4, cheaper with 3, and dirt cheap locally with LLaMA

Just try it!

Re: AI isn’t good enough

#89

Post by Aperocky »

squeaky-clean wrote: post #87
usrbinbash wrote: post #67

Earlier quoted context omitted.

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

> LLMs won't get intelligent. I think this sentence doesn't mean much unless we have a strict definition of what intelligence means. Just today ChatGPT helped me solve a DNS issue that I would not have been able to solve on my own in one day, let alone an hour. I'd consider it already more intelligent than myself when it comes to DNS.

It's seen more DNS content than you and anybody else have seen in their entire lives, and are able to regurgitate what it read because it has far faster memory access than you did.

A dictionary contain knowledge but no intelligence.

Re: AI isn’t good enough

#90

Post by dontupvoteme »

coldtea wrote: post #85
dontupvoteme wrote: post #71

Earlier quoted context omitted.

If it's anything like Stable Diffusion, it's actually where it becomes terrifyingly good at a specific thing. Like putting a field of artists out of a job, or copying a style so good that you can complete a persons piece before they do on a livestream. We're using a massive mush of the internet model which is taxed at 50% for alignment. That's going to be very dumb in the long run.

> Like putting a field of artists out of a job, To be frank, between stock art and photos, pre-AI template based tools, and the massive oversupply of graphic design and photography work, the field was already massively redudant and kind of out of a job to begin with...

I was thinking of people who... well, take commissions to make lewd images of various themes (e.g. furry stuff) or new specific characters or such... not my cup of tea but i know from researching what is possible that you only need a few images of new season anime girl blue hair edition #9237191 from the Japanese sites to do a lora and be able to make mostly anything you want when it's done on the booru models. And you can pose people with controlnet i think. that's also how those super cool QR code images are made.

Those guys apparently used to make pretty good cash from twitter, usually using pen names so they wouldn't' be associated with their regular work

The scary bit here is you can also "clone" a person to make any image you want of them. Obviously there's a lot of problems coming from that in the future, but also neat applications, e.g. some guy made selfie pictures of himself in the past with this for internet dating.

Post reply on HN