I used ChatGPT to generate erotic stories. Now I want a model which can produce porn videos from prompts.
Huh; I had assumed it would refuse to do that?
Unpredictable abilities emerging from large AI models
171–180 of 326 posts
Re: Unpredictable abilities emerging from large AI models
#172Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially! I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models. I was walking this morning with a no…
Is that what it is? I've been struggling to figure out why people have had such difficulty seeing the clear impact is going to have this
Re: Unpredictable abilities emerging from large AI models
#173Earlier quoted context omitted.
So on the one hand, these newly publicized models can render convincing representations of realities we used to get from deterministic processes. On the other hand, it's probabilistic and fails to conform to logic quite often, and in a confident way. We're building systems capable of programing computers non-deterministically. I think this is huge. But not because ChatGPT23 will be a CEO or a politician. But because…
The LLM can act as a global cache for common solutions to common problems, with the ability to perform the integration work necessary to apply them. That prime number example is a little bit like when you put a functools.lru_cache decorator on a function in Python. It's faster than computing the function call because it's able to recall the return value for the parameters from the cache "memory". Of course, many skil…
>The LLM can act as a global cache for common solutions to common problems, with the ability to perform the integration work necessary to apply them.
In my opinion Stack Overflow does a fine job at that. And it's transparent in terms of solutions proposed being voted on and discussed. Turning that into sausage advice is a downgrade.
>But globally that's not super interesting if lots of people are constantly re-developing the same solutions to the same problems for their own benefit.
I'd argue this is how we train individuals thus globally quite relevant.
Re: Unpredictable abilities emerging from large AI models
#174I'd like to see posts on LLMs written from a different perspective. For me, the surprise comes not from the sudden emergent capability of language models, but that the understanding (and synthesis!) of ideas encoded in language has succumbed to literally nothing more than statistical analysis. Or at least come that much closer to doing so. That it bears so close a resemblance to actual thinking says more about the im…
I find the phrase "statistical analysis" a frustrating one nowadays as it seems to have become a signal for "I hold a particular philosophy of the mind".
I don't understand this use of "statistical" as a diminutive to describe these models.
Why can't incredibly complicated behavior be emergent from matrix multiplication subject to optimization in the same way that our biological matter has developed complicated emergent properties also being subject to optimization?
The loss function is very different, the optimization techniques as well, but the fundamental idea of complex behavior emerging out of a substrate subject to optimization seems common. I haven't seen a single good answer to that
Re: Unpredictable abilities emerging from large AI models
#175Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially! I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models. I was walking this morning with a no…
> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through.
Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. Part of me wonders if at least some of the layoffs we've seen in the last couple of months are because companies such as Google, Microsoft, Amazon, etc. (the ones that have pretty clear visibility into what's happening in the AI space) are realizing that they aren't going to need as many knowledge workers in the not-so-distant future.
I think there was always this idea in the back of our minds that this was going to happen someday. But someday was always like 15 to 20 years out. Looks like someday is knocking on our door now.
Re: Unpredictable abilities emerging from large AI models
#176I'd like to see posts on LLMs written from a different perspective. For me, the surprise comes not from the sudden emergent capability of language models, but that the understanding (and synthesis!) of ideas encoded in language has succumbed to literally nothing more than statistical analysis. Or at least come that much closer to doing so. That it bears so close a resemblance to actual thinking says more about the im…
This is what Stephen Wolfram concludes in a recent article about ChatGPT: > The specific engineering of ChatGPT has made it quite compelling. But ultimately (at least until it can use outside tools) ChatGPT is “merely” pulling out some “coherent thread of text” from the “statistics of conventional wisdom” that it’s accumulated. But it’s amazing how human-like the results are. And as I’ve discussed, this suggests some…
I wonder if different languages lead to different capabilities? If I ask the same question in English, Japanese, and German, will I reliably get “better” answers from one language over another.
Re: Unpredictable abilities emerging from large AI models
#177Earlier quoted context omitted.
> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career. I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to da…
I think it'll end at using chatgpt as a great tool. And that seems fine. I think much of the anxiety here is in the unknown. You have to learn your IDE, but it doesn't make programming harder. It makes it more fun actually. Chatgpt is going to require that devs are even better IMO because you have to catch its errors and know how to work with it.
Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?
Re: Unpredictable abilities emerging from large AI models
#178Earlier quoted context omitted.
I think it'll end at using chatgpt as a great tool. And that seems fine. I think much of the anxiety here is in the unknown. You have to learn your IDE, but it doesn't make programming harder. It makes it more fun actually. Chatgpt is going to require that devs are even better IMO because you have to catch its errors and know how to work with it.
Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?
Your opinion on this probably depends on what your job currently looks like. Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me.
But if you work at BigCo and mostly sit on your hands already, then, yeah. I don't know.
Re: Unpredictable abilities emerging from large AI models
#179Earlier quoted context omitted.
This is what Stephen Wolfram concludes in a recent article about ChatGPT: > The specific engineering of ChatGPT has made it quite compelling. But ultimately (at least until it can use outside tools) ChatGPT is “merely” pulling out some “coherent thread of text” from the “statistics of conventional wisdom” that it’s accumulated. But it’s amazing how human-like the results are. And as I’ve discussed, this suggests some…
I wonder if different languages lead to different capabilities? If I ask the same question in English, Japanese, and German, will I reliably get “better” answers from one language over another.
The models transfer knowledge between languages, so probably some difference in capabilities but not a ton in core capabilities.
It can solve a physics problem in Telugu close to as well as in English.
Re: Unpredictable abilities emerging from large AI models
#180I write this as a sceptic who always leaned towards the symbolic side of this debate. First seeing Open AI inventing tactics in Dota and now using GPT and other recent AI product has changed my mind. The rate of progress in the field is just staggering and seems to be still accelerating. Overall, I'd say we are running into too many "unkowns". We don't know our own cognition works which makes all arguments along the…
>We don't know our own cognition works which makes all arguments along the lines of "LLMs are just .." Sure, but there are very binary tests we can do to understand the first principles of what LLMs are vs. what they are not. Ask an LLM to play tic-tac-toe and it does great. Ask it to play tic-tac-toe on a 100x100 board, it get's confused. This is a very easy test to examine the limits of it's ability to do symbolic…
How does it work regarding queries in natural language? I mean, thinking on translating a natural language question to an SQL query in complex scenarios.