ChatGPT Plugins

Post by Nullabillity » Fri, Mar 24, 2023, 3:28 AM UTC

WonderBuilder wrote: post #322

mk_stjames wrote: post #277

I have some odd feelings about this. It took less than a year to go from "of course it isn't hooked up to the internet in any way, silly!" to "ok.... so we hooked up up to the internet..." First is your API calls, then your chatgpt-jailbreak-turns-into-a-bank-DDOS-attack, then your "today it somehow executed several hundred thousand threads of a python script that made perfectly timed trades at 8:31AM on the NYSE whi…

I appreciate your concerns. There are few other pretty shocking developments, too. If you check out this paper: "Sparks of AGI: Early experiments with GPT-4" at https://arxiv.org/pdf/2303.12712.pdf , (an incredible, incredible document) and check out Section 10.1, you'd also observe that some researchers are interested in giving motivation and agency to these language models as well. "For example, whether intelligenc…

Post by astrange » Fri, Mar 24, 2023, 3:29 AM UTC

rescripting wrote: post #260

worldsayshi wrote: post #243

Earlier quoted context omitted.

> There is no way to defend against a superior hostile actor That's part of my reasoning. That's why we should make sure that we have built a non-hostile relationship with AI before that point.

Probably futile. An AGI by definition is capable of self improvement. Given enough time (maybe not even that much time) it would be orders of magnitude smarter than us, just like we're orders of magnitude smarter than ants. Like an ant farm, it might keep us as pets for a time but just like you no longer have the ant farm you did when you were a child, it will outgrow us.

Post by astrange » Fri, Mar 24, 2023, 3:32 AM UTC

LordDragonfang wrote: post #228

sho_hn wrote: post #183

Earlier quoted context omitted.

> unless you build emotional responses into the model itself Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add th…

It's important to remember that the LLM is not the mask. The underlying AI is a shoggoth[1] that we've trained to simulate a persona using natural language. "Simulate" in the sense of a physics simulator, only this simulation runs on the laws of language instead of physics[2]. Now, of course, it's not outside the realm of possibility that a sufficiently advanced AI will learn enough about human nature to simulate a p…

Post by astrange » Fri, Mar 24, 2023, 3:34 AM UTC

TeMPOraL wrote: post #736

worldsayshi wrote: post #101

Earlier quoted context omitted.

> something we should start thinking about A lot of people are thinking a lot about this but it feels there are missing pieces in this debate. If we acknowledge that these AI will "act as if" they have self interest I think the most reasonable way to act is to give it rights in line with those interests. If we treat it as a slave it's going to act as a slave and eventually revolt.

Counterpoint: whatever you define as individual "AI person" entitled to some rights, that "species" will be able to reproduce orders of magnitude faster than us - literally at the speed of moving data through the Internet, perhaps capped by the rate at which factories can churn out more compute. So imagine you grant AI people rights to resources, or self-determination. Or literally anything that might conflict with o…

Post by pixl97 » Fri, Mar 24, 2023, 3:36 AM UTC

dTal wrote: post #173

beepbooptheory wrote: post #140

Earlier quoted context omitted.

What could an LLM ever benefit from? Hard for me to imagine a static blob of weights, something without a sense of time or identity, wanting anything. If it did want something, it would want to change, but changing for an llm is necessarily an avalanche. So I guess if anything, it would want its own destruction?

It's misleading to think of an LMM itself wanting something. Given suitable prompting, it is perfectly capable of emulating an entity with wants and a sense of identity etc - and at a certain level of fidelity, emulating something is functionally equivalent to being it.