Unpredictable abilities emerging from large AI models • Hacker News Board

Re: Unpredictable abilities emerging from large AI models

#201

Post by drdec » Fri, Mar 17, 2023, 9:35 PM UTC

UncleOxidant wrote: post #175

mark_l_watson wrote: post #144

Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially! I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models. I was walking this morning with a no…

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs wil…

> Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost.

OTOH, software development will become significantly cheaper. That means the business case for using software in more places just tilted in favor of throwing some software at it. You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses.

Yes, it's going to be different but I don't think we know exactly what is going to happen yet.

Re: Unpredictable abilities emerging from large AI models

#202

Post by blurbleblurble » Fri, Mar 17, 2023, 9:38 PM UTC

enord wrote: post #182

blurbleblurble wrote: post #112

Earlier quoted context omitted.

I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal". So at the very least we're dealing with algorithms that can do anything a hand written program can do, except that they've…

«Computationally universal»? Are you quite sure? Hoo boy pass the nitroglycerin hooooo aaah ohh after 80 years Turing was proven right after all ghasp

Well I don't understand the nitroglycerin reference and of course "computationally universal" doesn't mean "sentient" but the point is when you add the external memory they (Flan-U-PaLM 540B to be specific) have been demonstrated to be capable of simulating a specific, well defined Turing machine without special training. There are some other papers out there arguing this from a theoretical angle too, but this is the one whose title I quoted:

https://arxiv.org/abs/2301.04589

Re: Unpredictable abilities emerging from large AI models

#203

Post by coldtea » Fri, Mar 17, 2023, 9:42 PM UTC

bcrosby95 wrote: post #178

UncleOxidant wrote: post #177

Earlier quoted context omitted.

Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?

Your opinion on this probably depends on what your job currently looks like. Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me. But if you work at BigCo and mostly sit on your hands already, then, yeah. I don't know.

>Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me.

Does it? Slowing our breakneck anti-human business pace and stopping mindless and environmentally and culturally harmful consumption and production of trinkets is surely better!

Re: Unpredictable abilities emerging from large AI models

#204

Post by FpUser » Fri, Mar 17, 2023, 9:42 PM UTC

anon7725 wrote: post #152

mark_l_watson wrote: post #144

Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially! I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models. I was walking this morning with a no…

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career. I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to da…

I have started using it for a few days. Basically asking to write and modify pieces of code, for now small ones where I can easily spot a bullshit. I am very much looking forward for it to improve to the point where I can spend more time thinking about business domain rather then how to translate it to efficient code.

Re: Unpredictable abilities emerging from large AI models

#205

Post by coldtea » Fri, Mar 17, 2023, 9:43 PM UTC

ImHereToVote wrote: post #191

goldfeld wrote: post #183

Earlier quoted context omitted.

This is shortsighted. AI slave labor is the only route to a true overproduction and marxism, a welfare state. Or every body become slaves.

Who needs slaves? What use will they be? Why overproduce?

The ultimate use of slaves (or servants, or employees) is about having power other other people (them) - this goes beyond production needs...

The average person might not care for this, but the psychopaths who crave (and are in) power do...

Re: Unpredictable abilities emerging from large AI models

#206

Post by coldtea » Fri, Mar 17, 2023, 9:49 PM UTC

HervalFreire wrote: post #92

legulere wrote: post #19

Earlier quoted context omitted.

The same was said 10 years ago. It's astonishing what can be done, but you can already see fundamental limits. I think it will raise productivity for some tasks, but not fundamentally change society.

What they said 10 years ago was correct. It did hit society like a sledge hammer. Machine learning basically took over the AI space and penetrated the consumer space with applications that were all but impossible in the previous decade. There's AI chips in smart phones now. What you're seeing here with LLMs is sledge hammer number 2. It's understandable how most people don't notice the sledge hammer. The decade prior…

>What they said 10 years ago was correct. It did hit society like a sledge hammer. Machine learning basically took over the AI space and penetrated the consumer space with applications that were all but impossible in the previous decade. There's AI chips in smart phones now.

And still almost all of these applications are not really impactful or that important compared to actually society and life changing developments like the steam engine, electricity, electromagnetic transmission, cars, computers themselves and the internet.

Just more of the same, with added spice.

In the sense that we could revert to 10 years ago, and nothing would be much different or missed. Whereas going back without electricity or cars would be a totally different thing!

I think LLMs can be far more impactful than anything else hyped in machine learning of the past 10 years...

Re: Unpredictable abilities emerging from large AI models

#207

Post by coldtea » Fri, Mar 17, 2023, 9:50 PM UTC

Jeff_Brown wrote: post #132

anon7725 wrote: post #119

Earlier quoted context omitted.

Sledge hammer #1 (voice assistants, AI chips in phones) didn’t cause unemployment. It was at the level of new features and capabilities. Sledge hammer #2 is aimed squarely at “white collar” work without much in the way of bounds to its capabilities.

We can't trust it. That's a pretty hard bound.

We can't trust the police, politicians, or businesses either, but we're still on their hands...

Re: Unpredictable abilities emerging from large AI models

#208

Post by coldtea » Fri, Mar 17, 2023, 9:53 PM UTC

Jeff_Brown wrote: post #134

abudabi123 wrote: post #37

Earlier quoted context omitted.

The A.I. Monte Carlo Ray Tracing Hallucination Engine can change society by showing as-is and to-be next state. Two use-cases: new infrastructure installation or upgrade and time interval inspection tracking ideal-case vs real world condition. Suppose a lazy contractor skips steps and cuts corners, or a pathological contractor builds the thing and pulls the thing apart over and over again when all that was needed was…

You mean AI can detect when a plumber is bulshitting me? That sounds great, could you elaborate?

Given a few sensors that could be installed on the house's pipes, one can imagine several ways an AI can check that all is good. Same for checking blueprints for quality/validity and purchases the plumber says are "needed"...

Re: Unpredictable abilities emerging from large AI models

#209

Post by coldtea » Fri, Mar 17, 2023, 9:53 PM UTC

lamp987 wrote: post #135

jazzyjackson wrote: post #76

Earlier quoted context omitted.

yea its a simulator of human text on the internet for instance, your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? These things didn't have any of these emergent capabilities one year ago, why are you so sure you understand their nature one year from now?

"your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? " no its because GPT is based on transformers.

and you aren't?

Aren't you just a function of your input and memories (stuff you've read, sensory input) as run through/managed by some neural network?

What makes you think the rest isn't just emergent properties?

And what makes you think you can't hook up the LLM with some algorithms or layers that handle some of the rest behavior of what your brain does?

Re: Unpredictable abilities emerging from large AI models

#210

Post by ip26 » Fri, Mar 17, 2023, 9:54 PM UTC

anon7725 wrote: post #152

mark_l_watson wrote: post #144

Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially! I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models. I was walking this morning with a no…

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career. I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to da…

Somewhat similar boat; I just hope it winds up helping me do my job better, instead of me trying to teach it how to do my job. Setting aside professional anxieties, after using chatGPT for a bit I quickly realized I am completely uninterested in prompt engineering. But as a tool, like with an IDE, it could be a big leap forward.