Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rodney Brooks on limitations of generative AI

techcrunch.com

Post reply on HN
Print view View original

181–190 of 202 posts

Re: Rodney Brooks on limitations of generative AI

#181

Post by hbs18 »

derefr wrote: post #124
pestaa wrote: post #109

Earlier quoted context omitted.

It's more than enough data for a specialized tool, yes. It's not even remotely enough data for a statistical language processor.

Why are young children able to quickly surpass state-of-the-art ML models at arithmetic tasks, from only a few hours of lecturing and a "training dataset" (worksheets) consisting of maybe a thousand total examples? What is happening in the human learning process from those few thousand examples, to deduce so much more about "the rules of math" per marginal datapoint?

You can't reliably teach an LLM maths the same way you can't take a locomotive offroading.

Re: Rodney Brooks on limitations of generative AI

#182

Post by MathMonkeyMan »

MattPalmer1086 wrote: post #72
aurareturn wrote: post #15

Earlier quoted context omitted.

I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…

Haha, I'd be happy if outlook just integrated a search that actually works. Most of outlook search results aren't even relevant, and it regularly misses things I know are there. Literally the most useless search I've ever had to use.

Thunderbird's is bad, too. GMail's is pretty good.

Re: Rodney Brooks on limitations of generative AI

#183

Post by hanche »

grugagag wrote: post #130
hanche wrote: post #71

Earlier quoted context omitted.

> I probably would never listen to more than a few GBs of high fidelity music in my life time. FWIW, I currently have about 100 GB of music on my phone. And that is in a fairly high quality AAC format. Converted to lossless, it might be about five times that size? I don't even think of my music collection as all that extensive. But still, 160 TB would be a different ball game altogether. For sure, there is no mass ma…

You have a collection. Do or will you ever listen to it all end to end? I doubt it.

Of course not. I must have misunderstood what you meant, sorry.

Re: Rodney Brooks on limitations of generative AI

#184

Post by Propelloni »

vidarh wrote: post #165
Propelloni wrote: post #89

Earlier quoted context omitted.

Most data around is junk and the internet produces junk data faster then useful data and current GPT AIs basically regurgitate what someone already did somewhere on the internet. So I guess the more data we feed into GPTs the worse the results will get. My take to improve AI output is to heavily curate the data you feed your AI, much the like expert systems of old (which were lauded as "AI" also.) Maybe we can break…

There are multiple companies hiring people on contracts to curate and generate data for this. I do confidential contract work for two different ones at the moment, and while my NDAs limit how much I can say, it involves both identifying issues with captured prompt/response pairs that have been filtered, and writing synthetic ones from scratch aided by models (e.g. come up with a coding problem, and rewrite the respon…

Thanks for your insights! Would you say, this is an approach suited for "general" GPTs (not in the sense of AGI) or more for expert systems like Copilot?

Re: Rodney Brooks on limitations of generative AI

#185

Post by Eisenstein »

pyrale wrote: post #14
roenxi wrote: post #9

> He uses the iPod as an example. For a few iterations, it did in fact double in storage size from 10 all the way to 160GB. If it had continued on that trajectory, he figured out we would have an iPod with 160TB of storage by 2017, but of course we didn’t. I think Brooks' opinions will age poorly, but if anyone doesn't already know all the arguments for that they aren't interested in learning them now. This quote see…

> Overall I don't think this is a fair argument against exponential growth. Do we still need to make a fair claim against unrestricted exponential growth in 2024? Exponential growth claims have been made countless times in the past, and never delivered. Studies like the Limits to Growth report (1972) have shown the impossibility of unrestricted growth, and the underlying science can be used in other domains to show t…

Moore's law?

Re: Rodney Brooks on limitations of generative AI

#186

Post by lostmsu »

Jensson wrote: post #161
lostmsu wrote: post #75

Earlier quoted context omitted.

There was never universally recognized early success in FSD.

When self driving cars started driving around public in 2010 that was a universally recognized success of self driving cars. That shocked many people, as before then self driving cars was entirely science fiction. There was of course never a recognized success of FSD, since we don't have FSD, but when people started seeing cars drive themselves on public streets 2010 they assumed we would have FSD in 5-10 years, but…

Who are these people you are talking about?

99.9% of population never saw a self-driving car in their life, and only a few would treat a single spotting on media as indication of anything.

ChatGPT on the other hand is widespread.

Re: Rodney Brooks on limitations of generative AI

#187

Post by bongodongobob »

surfingdino wrote: post #177
bongodongobob wrote: post #140

Earlier quoted context omitted.

Yes this is already a thing. Copilot for m365 fine tunes on your entire orgs data.

So that's a hard stop for many orgs. Not everyone is supposed to see all of the org's data even if they work for it.

There are still controls on that, it doesn't give you access to everything. It took forever for legal to sign off on it.

Re: Rodney Brooks on limitations of generative AI

#188

Post by pyrale »

Eisenstein wrote: post #185
pyrale wrote: post #14

Earlier quoted context omitted.

> Overall I don't think this is a fair argument against exponential growth. Do we still need to make a fair claim against unrestricted exponential growth in 2024? Exponential growth claims have been made countless times in the past, and never delivered. Studies like the Limits to Growth report (1972) have shown the impossibility of unrestricted growth, and the underlying science can be used in other domains to show t…

Moore's law?

A slowdown already happened once (the original pace was doubling every year), and I believe it's widely accepted that we're not going to keep the current pace for long.

Re: Rodney Brooks on limitations of generative AI

#189

Post by talldayo »

grugagag wrote: post #134
threeseed wrote: post #95

Earlier quoted context omitted.

> My take to improve AI output is to heavily curate the data you feed your AI This is what OpenAI is doing with their relationships with companies like Reddit, News Corp etc: https://openai.com/index/news-corp-and-openai-sign-landmark-... Problem is that we have a finite amount of this type of information.

Massive surveilance, take out data and use it on training. Hope this will not come to frution.

Thankfully, we have stalwart and well-known defenders of our security like Apple and Microsoft to protect us. There's nothing of the sort to worry about.

Re: Rodney Brooks on limitations of generative AI

#190

Post by Eisenstein »

pyrale wrote: post #188
Eisenstein wrote: post #185

Earlier quoted context omitted.

Moore's law?

A slowdown already happened once (the original pace was doubling every year), and I believe it's widely accepted that we're not going to keep the current pace for long.

> Exponential growth claims have been made countless times in the past, and never delivered.

But one did though? Pronouncements of the future do not invalidate current results.

Post reply on HN