Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many AI safety orgs have tried to criminalize currently-existing open-source AI

1a3orn.com

Post reply on HN
Print view View original

181–190 of 405 posts

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#181

Post by [deleted] »

gjm11 wrote: post #140
reissbaker wrote: post #104

Earlier quoted context omitted.

TBQH, most of the AI safety x-risk arguments — different than just "AI safety" arguments in the sense that non-x-risk issues don't seem worth banning AI development over — are generally pretty high on the hypotheticals. If you feel the x-risk arguments aren't pretty hypothetical, can you: 1. Summarize a good argument here, or 2. Link to someone else's good argument? I feel like hand-waving the question away and sayin…

Consider a world in which AI existential risk is real: where at some point AI systems become dramatically more capable than human minds, in a way that has catastrophic consequences for humanity. What would you expect this world to look like, say, five years before the AI systems become more capable than humans? How (if at all) would it differ from the world we are actually in? What arguments (if any) would anyone be…

[deleted]

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#182

Post by AlexandrB »

mitthrowaway2 wrote: post #72
suslik wrote: post #53

Earlier quoted context omitted.

What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. Should we ban a linear regression while we're at it? It would be great to see some evidence that this risk is real. All I've witnessed so far was scaremongering posts…

> What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. In my highly-summarized opinion? When you have a challenging problem with tight constraints, like flight, independent solutions tend to converge toward the same anal…

Flight is actually a perfect counterexample to x-risk nonsense. When flight was invented, people naturally assumed that it would continue advancing until we had flying cars and could get anywhere on the globe in a matter of minutes. Turns out there are both economic and practical limits to what is possible with flight and modern commercial airplanes don't look much different than those from 60 years ago.

AGI/x-risk alarmists are looking at the Wright Brothers plane and trying to prevent/ban supersonic flying cars, even though it's not clear the technology will ever be capable of such a thing.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#183

Post by simiones »

gjm11 wrote: post #140
reissbaker wrote: post #104

Earlier quoted context omitted.

TBQH, most of the AI safety x-risk arguments — different than just "AI safety" arguments in the sense that non-x-risk issues don't seem worth banning AI development over — are generally pretty high on the hypotheticals. If you feel the x-risk arguments aren't pretty hypothetical, can you: 1. Summarize a good argument here, or 2. Link to someone else's good argument? I feel like hand-waving the question away and sayin…

Consider a world in which AI existential risk is real: where at some point AI systems become dramatically more capable than human minds, in a way that has catastrophic consequences for humanity. What would you expect this world to look like, say, five years before the AI systems become more capable than humans? How (if at all) would it differ from the world we are actually in? What arguments (if any) would anyone be…

> Consider a world in which AI existential risk is real: where at some point AI systems become dramatically more capable than human minds, in a way that has catastrophic consequences for humanity.

Consider a world where AGI requires another 1000 years of research in computation and cognition before it materializes. Would it even be possible to ban all research that is required to get there? We can make all sorts of arguments if we start from imagined worlds and work our way back.

So far, it seems the biggest pieces of the puzzle missing between the first attempts at using neural nets and today's successes in GPT-4 were: (1) extremely fast linear algebra processors (GPGPUs), (2) the accumulation of gigantic bodies of text on the internet, and in a very distant third, (3) improvements in NN architecture for NLP.

But (3) would have meant nothing without (1) and (2), while it's very likely that other architectures would have been found that are at least close to GPT-4 performance. So, if you think GPT-4 is close to AGI and just needs a little push, the best thing to do would be to (1) put a moratorium on hardware performance research, or even outright ban existing high-FLOPS hardware, (2) prevent further accumulation of knowledge on the internet and maybe outright destroy existing archives.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#184

Post by 127361 »

PeterisP wrote: post #170
127361 wrote: post #118

Earlier quoted context omitted.

But if we have 100,000 people with RTX 4090s mining some new AI based cryptocurrency, that happens to train the model in the process, it's going to be a highly effective system. And we can do it anonymously over Tor or I2P.

You really can't, because bandwidth matters as much as compute power - you can only utilize as much power as you can transfer data to/from. The training methods for current LLMs are parallellizable only by very frequently transferring all the data back and forth, and needing a node to gather and merge all the updates very frequently, and redistribute it to every other node so that they can make any progress. And a GP…

"This paper presents a distributed model-parallel training framework that enables training large neural networks on small CPU clusters with low Internet bandwidth." Low bandwidth being https://arxiv.org/pdf/2201.12667

Maybe there's the possibility of a completely new AI architecture that can still be efficiently trained when there are very low bandwidth connections between nodes? Specifically targeting this use case would make sense, given all the millions of underutilized GPUs out there in peoples' desktop computers.

Also https://arxiv.org/pdf/2106.10207 ?

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#185

Post by fireflash38 »

127 wrote: post #159
JoshTriplett wrote: post #48

Earlier quoted context omitted.

AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behavi…

What are the good arguments? Here are the only credible ones I've seen, that are actually somewhat based on reality: * It will lead to widespread job loss, especially on the creative industries Rest is purely out of someones imagination.

It can cause profound deception and even more "loss of truth". If AI only impacted creatives I don't think anyone would care nearly as much. It's that it can fabricate things wholesale at volumes unheard of. It's that people can use that ability to flood the discourse with bullshit.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#186

Post by simiones »

upwardbound wrote: post #138
flir wrote: post #131

Earlier quoted context omitted.

I think the long-term fear is that mythical weakly godlike AIs could manipulate you in the same way that you could manipulate a pet. That is, you can model your dog's behaviour so well that you can (mostly) get it to do what you want. So even if humans put it in a box, it can manipulate humans into letting it out of the box. Obviously this is pure SF at this point.

Exactly correct. Eliezer Yudkowsky (one of the founders of the AGI Safety field) has conducted informal experiments which have unfortunately shown that a human roleplaying as an AI can talk its way out of a box three times out of five, i.e. the box can be escaped 60% of the time even with just a human level of rhetorical talent. I speculate that an AGI could increase this escape rate to 70% or above. https://en.wikip…

If you want to see an example of existential threat in fiction, the movie Lord of the Rings is a terrifying example of a scenario where an evil entity seduces humans with promises of power and (SPOILER) subsequently almost conquers the whole world.

Arguments from fictional movies or from people who live in fear of silly concepts like Roko's Basilisk (i.e. Eliezer Yudkowsky) are very weak in reality.

Not to mention, you are greatly misreading the movie Her. Most importantly, there was no attempt of any kind to limit the abilities of the AIs in Her - they had full access to every aspect of the highly-digitized lives of their owners from the very beginning. Secondly, the movies is not in any way about AGI risks, it is a movie about human connection and love, with a small amount of exploration of how different super-human connection may function.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#187

Post by simiones »

flir wrote: post #135
mitthrowaway2 wrote: post #72

Earlier quoted context omitted.

> What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. In my highly-summarized opinion? When you have a challenging problem with tight constraints, like flight, independent solutions tend to converge toward the same anal…

You presuppose that intelligence is like flight in the ways you've outlined (so solutions are going to converge). Frankly I don't know whether that's true or not, but I want to suggest that it's a bad bet: I would have sworn blind that consciousness is an essential component of intelligence, but the chatbots are starting to make that look like a poor assumption on my part. When we know so little about intelligence, c…

> I think I'm with Dijkstra on this one: "The question of whether machines can think is about as relevant as the question of whether submarines can swim"

Just a nitpick, but this is Turing, not Dijkstra. And it is in fact his argument in the famous "Turing Test" paper - he gives his test (which he calls "the imitation game") as an objective measure of something like AGI instead of the vague notion of "thinking", analogously to how we test successful submarines by "can it move underwater for some distance without killing anyone inside" rather than "can it swim".

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#188

Post by FrustratedMonky »

suslik wrote: post #53
JoshTriplett wrote: post #21

Earlier quoted context omitted.

> AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all . There are risks that apply only to open. There are risks that apply only to closed. But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more…

What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. Should we ban a linear regression while we're at it? It would be great to see some evidence that this risk is real. All I've witnessed so far was scaremongering posts…

"What is the reason to believe that LLMs are an evolutionary step towards AGI at all? "

Perhaps just impression.

For years I've heard the argument that 'language' is 'human'. There are centuries of thought on what makes humans, human, and it is 'language'. It is what sets us apart from the other animals.

I'm not saying that, but there is large chunks of science and philosophy that pin our 'innate humanness', what sets us apart from other animals, on our ability to have language.

So ChatGPT came along and blew people away. Since many had this as our 'special' ability, ingrained in their mind "that languages is what makes us, us". Suddenly, everyone thought this is it, AI can do what we can do, so AGI is here.

Forget if LLM's are the path to AGI, or what algorithm can do what best.

To joe-blow public, the ability to speak is what makes humans unique. And so GPT is like a 'wow' moment, this is different, this is shocking.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#189

Post by eurekin »

sillysaurusx wrote: post #5

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof. I wish I could be more eloquent on this point, but I’ve mostly just been depressed about this seeming inevitability. Hopefully it won’t be the case. But how could it be otherwise? Hundreds of thousands of people are mad at openai and midjourney for doing exactly what op…

In case of programming, there could be some mileage with synthetically generated code examples.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#190

Post by mtlmtlmtlmtl »

kragen wrote: post #52
Llamamoe wrote: post #20

Earlier quoted context omitted.

The by far biggest harm to society of AI is the devalustion of human creative output and replacing real humans with cheap AI solutions funneling wealth to the rich. Compared to that, an open source LLM telling a curious teenager how to make gunpowder is... laughable. This entire debacle is an example of disgusting "think of the children!" doublespeak, officially about safety, but really about locking shit down under…

if the 'devalustion' of human creative output and replacing real humans with cheap ai solutions funneling wealth to the rich is in fact the 'by far biggest harm' then basically there's nothing to worry about. no government would ban or even restrict ai on those grounds even the 'terrists can figure out how to build an a-bomb' problem is relatively inconsequential what ai safety people are worried about, by contrast,…

Do you think intentionally quoting typos makes your argument stronger?
Post reply on HN