Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rodney Brooks on limitations of generative AI

techcrunch.com

Post reply on HN
Print view View original

101–110 of 202 posts

Re: Rodney Brooks on limitations of generative AI

#101

Post by locallost »

lostmsu wrote: post #76
locallost wrote: post #33

After using Copilot that is pretty bad at guessing what I exactly want to do, but still occasionally right on the money and often pretty close: AI is not really AI and it won't kill us all, but the realization is that a lot of work is just repetitive and really not that clever at all. If I think about all the work I did in my life it follows the same pattern: a new way of doing things comes along, then you start figu…

Copilot can not read your mind: that is an impressive argument against AI!

I actually said it can read my mind which I interpret as my work being repetitive and not that clever. It can still read my mind poorly most of the time, although it's in the ballpark, but on occasion it hammers one home which feels like magic. Whether or not it can get good at reading my mind or not I don't know. But whether it can get good at doing non repetitive tasks that have been done a billion times, well, it feels like a no right now.

Re: Rodney Brooks on limitations of generative AI

#102

Post by qeternity »

danparsonson wrote: post #39
qeternity wrote: post #35

The iPod analogy is a poor one. Instead of 160TB music players, we got general computers (iPhone) with effectively unlimited storage (wireless Internet). I don’t need to store all my music on my device. I can have it beamed directly to my ears on-demand.

That's the point though, an iPhone is not just a bigger iPod - scaling by itself isn't enough.

No, it's not. Scaling did work. The internet gives us (effectively) unlimited storage.

We just scaled in a slightly different way.

Re: Rodney Brooks on limitations of generative AI

#103

Post by mjburgess » 

    Were one to slice the corpus callosum, 
    and burn away the body, 
        and poke out the eyes.. 
    and pickle the brain, 
        and everything else besides... 
    and attach the few remaining neurones 
        to a few remaining keys.. 
    then out would come a program 
        like ChatGPT's --
    
    "do not run it yet!" 
        the little worker ants say
    dying on their hills, 
    out here each day:
        it's only version four -- 
        not five!
    Queen Altman has told us: 
        he's been busy in the hive --
        next year the AI will be perfect
            it will even self-drive!

    Thus comment the ants, 
    each day and each night:
        Gemini will save us, 
            Llama's a delight!
        One more gigawat, 
        One more crypto coin
        One soul to upload
        One artist to purloin
        One stock price to plummet
        Oh thank god
           -- it wasn't mine!

Re: Rodney Brooks on limitations of generative AI

#104

Post by caddemon »

qeternity wrote: post #102
danparsonson wrote: post #39

Earlier quoted context omitted.

That's the point though, an iPhone is not just a bigger iPod - scaling by itself isn't enough.

No, it's not. Scaling did work. The internet gives us (effectively) unlimited storage. We just scaled in a slightly different way.

The music part is just scaling in a different way, but that's not the reason the iPhone is so successful. The leap from iPod to iPhone was definitely not just scaling.

Re: Rodney Brooks on limitations of generative AI

#105

Post by hdhshdhshdjd »

moffkalast wrote: post #44
SideburnsOfDoom wrote: post #38

Earlier quoted context omitted.

> not smart > slightly smarter > way smarter > Last one, "slightly smarter" So, the the usual s-curve, that has an exponential phase, then topping out?

Pretty much, yep. There was definitely a more significant jump there in the middle where 7B models went from being a complete waste of time to actually useful. Then going from being able to craft a sensible response to 80% of questions to 90% is a much smaller apparent increase but takes a lot more compute to achieve as per the pareto principle.

I see giant models like Intel chips over the last decade: big, powerful, expensive, energy hogs.

Small models are like arm: you get much of the performance you actually need for common consumer tasks, very cheap to run, and energy efficient.

We need both but I personally spend most of my ML time on small models training and I’m very happy with the results.

Re: Rodney Brooks on limitations of generative AI

#106

Post by hdhshdhshdjd »

Animats wrote: post #8

That makes sense from Brooks' perspective. He's done his best work when he didn't try to overreach. The six-legged insect thing was great, but very dumb. Cog was supposed to reach human-level AI and was an embarrassing dud. The Roomba was simple, dumb, and useful. The military iRobot machines were good little remote-controlled tanks. The Rethink Robotics machines were supposed to be intelligent and learn manipulation…

[flagged]

Re: Rodney Brooks on limitations of generative AI

#107

Post by lucubratory »

oska wrote: post #61
bubblyworld wrote: post #57

Earlier quoted context omitted.

Look, I don't want to get into the weeds of this because personally I don't think it's relevant to the issue of intelligence, but here's a list of things I think are evident about consciousness: 1. People have different kinds of conscious experience (just talk to other humans to get the picture). 2. Consciousness varies, and can be present or not-present at any given moment (sleep, death, hallucinogenic drugs, anaest…

Your responses indicate (at least to me) that you are, philosophically, a Materialist or a Physicalist. That's fine; I accept that's a philosophy of existence that one can hold (even though I personally find it sterile and nihilistic). However many, like me, do not subscribe to such a philosophy. We can avoid any argument between ppl who hold different philosophies but still want to discuss machine learning productiv…

Every scientist has to be a materialist, it comes from interacting with the world as it is. Most of them then keep their materialism in their personal life, but unfortunately some come home from work and stop thinking critically, embracing dualism or some other unprovable fantasies. If you want people engaging with the world and having correct ideas, you need to tolerate materialism because that's how that happens. It offending your religious worldview is unrelated.

Re: Rodney Brooks on limitations of generative AI

#108

Post by singpolyma3 »

oska wrote: post #61
bubblyworld wrote: post #57

Earlier quoted context omitted.

Look, I don't want to get into the weeds of this because personally I don't think it's relevant to the issue of intelligence, but here's a list of things I think are evident about consciousness: 1. People have different kinds of conscious experience (just talk to other humans to get the picture). 2. Consciousness varies, and can be present or not-present at any given moment (sleep, death, hallucinogenic drugs, anaest…

Your responses indicate (at least to me) that you are, philosophically, a Materialist or a Physicalist. That's fine; I accept that's a philosophy of existence that one can hold (even though I personally find it sterile and nihilistic). However many, like me, do not subscribe to such a philosophy. We can avoid any argument between ppl who hold different philosophies but still want to discuss machine learning productiv…

I'm not sure generic terms like "artificial intelligence" which describe every enemy in a video game are bound by a particular philosophy.

Re: Rodney Brooks on limitations of generative AI

#109

Post by pestaa »

threeseed wrote: post #94
theteapot wrote: post #90

Earlier quoted context omitted.

I think the argument was, GPT4 can't learn to do Math from more data. I'd be surprised if that's not true.

ChatGPT makes mistakes doing basic arithmetic or sorting numbers. Pretty sure we have enough data for these fundamental tasks.

It's more than enough data for a specialized tool, yes.

It's not even remotely enough data for a statistical language processor.

Re: Rodney Brooks on limitations of generative AI

#110

Post by surfingdino »

aurareturn wrote: post #15
fxtentacle wrote: post #13

To me, this reads like a very reasonable take. He suggests to limit the scope of the AI problem, add manual overrides in case there are unexpected situations, and he (rightly, in my opinion) predicts that the business case for exponentially scaling LLM models isn't there. With that context, I like his iPod example. Apple probably could have made a 3TB iPod to stick to Moore's law for another few years, but after they…

I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…

> integrate an LLM into Slack

They are already training their models https://slack.com/intl/en-gb/trust/data-management/privacy-p...

> Microsoft to integrate an LLM into outlook

Unlikely to happen. Orgs that use MS products do not want content of emails leaking and LLMs leak. There is a real danger that an LLM will include information in the summary that does not come from the original email thread, but from other emails the model was trained on. You could learn from the summary that you are going to get fired, even though that was not a part of the original conversation. HR doesn't like that.

> However, the more data you feed into an LLM, the smarter it should be in the response

Not necessarily. At some point you are going to run out of current data and you might be tempted to feed it past data, except that data may be of poor quality or simply wrong. Since LLMs cannot tell good data from bad, they happily accept both leading to useless outputs.

Post reply on HN