Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

311–320 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#311

Post by adamsmith143 »

nawgz wrote: post #300
sp527 wrote: post #269

Earlier quoted context omitted.

“AI will probably most likely lead to the end of the world, but in the meantime, there'll be great companies” - Sam Altman But sure, let’s just let this guy who sounds like a madman keep going. What could possibly go wrong?

Do you honestly find this quote out-of-sorts with what a normal CEO says? Oil companies have been knowingly poisoning the globe for their own profit - no probabilities involved - since at least the 70s, probably earlier, and on the flip-side building massive propaganda and influence machines, and further to that end even buying and squashing green patents of all sorts. If you're going to cry that we should regulate A…

Do you really believe that people think AI is risky because of 1 Sam Altman quote?

Re: Pause Giant AI Experiments: An Open Letter

#312

Post by tetris11 »

4k3aa4 wrote: post #268
Quarrel wrote: post #257

Earlier quoted context omitted.

Forget countries? How will you stop me? I have models on my local computer. I can and will use them. They will get better. You don't put tech genie's back in their bottles, that's not how it works.

I mean, in an alternate universe where atomic weapons were a little easier to manufacture at home would it not have made sense for governments to aggressively crack down on anyone doing anything even remotely resembling building them? I guess the second question is - would they have succeeded or would we all just have died?

Here's Nick Bostromcs Vulnerable World Hypothesis that you were alluding to.

http://www.nickbostrom.com/papers/vulnerable.pdf

Re: Pause Giant AI Experiments: An Open Letter

#313

Post by [deleted] »

bungeonsBaggins wrote: post #262
pc_edwin wrote: post #250

I've been a staunch supporter of Elon throughout all his scandals, especially the Twitter ones but I'm unable to come up with any justification for this. Everything leads to him (and others) pressuring OpenAI to pause so they could catch up. It doesn't take much smarts to deduce the cats already out of the bag. There is no stopping this, only pressuring/slowing-down/targeting certain players (OpenAI) as a competition…

[flagged]

[deleted]

Re: Pause Giant AI Experiments: An Open Letter

#314

Post by permo-w »

stuckinhell wrote: post #244

The genie is out of the bottle. It's never gonna happen. Even my husband and I went out and paid $20k to create a homelab for AI training. My firm fast tracked 2 million just for hardware to create their own internal Chatgpt style AI.

if governments across the West came out now and made laws to stop this, then sure a few thousand experts on hacker news that can afford to drop $20k on hardware could keep developing them illegally, but all the universities and corporations would stop. and it's the universities and the corporations, openAI especially, that have capacity to develop the dangerous AIs

Re: Pause Giant AI Experiments: An Open Letter

#316

Post by adamsmith143 »

thomastjeffery wrote: post #302

The dumb criticize the blind. What an absurd situation! How did we get here? Here are the steps: 1. Large Language Models have been presented as "AI", which personifies them instead of describing how they work. 2. Goals for LLM development were set for the personified attributes, and not the actual functionally of the real thing. OpenAI brags about how GPT4 scores at human tests: as if that has any bearing on the mod…

At no point in your rambling, incoherent comment were you even close to anything that could be considered a rational thought. Everyone on this site is now dumber for having read it. May God have mercy on your soul.

Re: Pause Giant AI Experiments: An Open Letter

#317

Post by thomastjeffery »

permo-w wrote: post #308
thomastjeffery wrote: post #302

The dumb criticize the blind. What an absurd situation! How did we get here? Here are the steps: 1. Large Language Models have been presented as "AI", which personifies them instead of describing how they work. 2. Goals for LLM development were set for the personified attributes, and not the actual functionally of the real thing. OpenAI brags about how GPT4 scores at human tests: as if that has any bearing on the mod…

I'm not concerned about how they work or whether they're attaining personhood or not. I'm concerned at what people can and will use them for to the detriment of our civilisation

How they work is the most critical piece of that puzzle.

It isn't a mystery at all. LLMs implicitly model every pattern they can find in the text they are given. If the pattern aligns to language, humans categorize it as a feature. If it aligns to desired behavior, humans categorize it as a feature. If it aligns to undesired behavior, humans categorize it as a limitation. What humans categorize as features and limitations do not align with what the model is. A truth and a lie are structured with identical text patterns. The LLM can't categorize the two separately.

The only risk here is that people continue to personify LLMs, and place trust in their relatively limited ability to fulfill a human role. It's the same category of risk as bad security practices. The potential to make a mistake itself is not a danger: making the mistake is.

Re: Pause Giant AI Experiments: An Open Letter

#318

Post by FartyMcFarter »

ChatGTP wrote: post #252
kvetching wrote: post #233

Earlier quoted context omitted.

If China achieves AGI first, we are done. The first thing they'll do is create cyber-weapons. Their AGI could find vulnerabilities in our systems and brick all of our devices. It's a far greater near term existential risk for us to allow China to gain a lead here.

We could also just talk with China rather than kicking off a new arms race ?

Do you think that's likely to work?

If China doesn't even respect its own citizens (see Xinjiang, heavy censorship if you need evidence), why would they respect talks with other countries?

Re: Pause Giant AI Experiments: An Open Letter

#319

Post by 55555 »

55555 wrote: post #270
bungeonsBaggins wrote: post #262

Earlier quoted context omitted.

[flagged]

Don't you think this is needlessly argumentative? It's not relevant to the parent topic. His only purpose in stating the part you quoted was to say, "so me disagreeing with him now should really say something."

I will include some parts of the HN commenting guidelines here for my sibling commenters who disagree.

> Avoid generic tangents.

> Please don't use Hacker News for political or ideological battle. That tramples curiosity.

> Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead.

> [irrelevant] ... too common to be interesting.

> [irrelevant] ... it makes boring reading.

Do as you wish.

Re: Pause Giant AI Experiments: An Open Letter

#320

Post by zach_garwood »

paxys wrote: post #182

So every company should cease all AI development for an indefinite period of time until someone has done the research and deemed it safe. And who exactly will that someone be? Future of Life Institute (whoever they are)? Elon Musk, who has very conveniently signed the letter after his public beef with OpenAI? Your friendly local government that has no agenda whatsoever? The UN? Efforts like these are at best naive, a…

> So every company should cease all AI development for an indefinite period of time

The letter quite clearly asks for six months.

Post reply on HN