Sam and Greg's response to OpenAI Safety researcher claims • Hacker News Board

Re: Sam and Greg's response to OpenAI Safety researcher claims

#461

Post by komali2 » Sun, May 19, 2024, 2:09 PM UTC

Dalewyn wrote: post #317

komali2 wrote: post #241

Earlier quoted context omitted.

Can you explain, with your worldview, why the nature of geopolitics turned abruptly and completely upon the invention and proliferation of nuclear bombs? What was it about the nuclear bomb that caused it to be the defining structure of the cold war? Do you believe MAD can exist without nuclear bombs?

>What was it about the nuclear bomb that caused it to be the defining structure of the cold war? The only difference is you can kill more humans for a given span of time. The end result is the same: Cities are leveled, people are killed. Is it more comforting to have been killed by a conventional bomb than a nuke? I certainly don't think so. So practically speaking, there is no difference. >Do you believe MAD can exi…

Do you believe any country has enough conventional ordinance to end human life on the entire planet? You must know more about this than me, I've never heard of something like this.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#462

Post by hiatus » Sun, May 19, 2024, 2:23 PM UTC

hiAndrewQuinn wrote: post #377

huppeldepup wrote: post #363

I think the fear of AGI destroying humanity mostly stems from the suspicion that a logically thinking device would indeed come to the conclusion that we need to be exterminated.

It comes from the fact that a logically thinking device given an arbitrary goal will probably not develop "Thou shalt not kill"-type moralisms in the pursuit of that goal. If you make an AGI to run a paperclip factory, eventually you'll come back to find most of the carbon and iron in your galaxy refactored into paperclips. The extinction of any local flora or fauna there is just a byproduct of that drive. See [2] fo…

> If you make an AGI to run a paperclip factory, eventually you'll come back to find most of the carbon and iron in your galaxy refactored into paperclips.

Given the costs to run such an AGI, it seems unlikely one would be left unattended for long.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#463

Post by Terretta » Sun, May 19, 2024, 2:23 PM UTC

conception wrote: post #208

Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

https://www.theguardian.com/technology/article/2024/may/17/a... "...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the…

> ... map tool lets users “nominate targets of interest” for “the target nomination process” ... These blobs, she said, were civilian areas like hospitals and schools. ... where children sleep and families get medical treatment." ... Definitely not dangerous at all.

Have you typed "school" or "hospital" in Google Maps?

This argument reduces to: any knowledge that can be misused is dangerous.

More murders in America are committed with hammers than long guns, yet we don't regulate hardware stores.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#464

Post by Terretta » Sun, May 19, 2024, 2:31 PM UTC

Nevermark wrote: post #368

Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

Text continuation defines a task. It does not limit the sophistication of the solution. The difference between problems and solutions seems to trip up a lot of people. —- It gets even more nuanced. At its basic level all types of digital AI are “just” arithmetic, or “just” Boolean logic. But arithmetic’s and Boolean logic’s simplicity don’t limit what systems can be built with them, as they are Turing Complete. Likew…

> ... they are Turing Complete.

OK, so all software must be regulated?

Arguing that simple tools can be used for dangerous results and thus must be regulated or made “safe” is like saying since more murders in America are committed with hammers than long guns, you must show an ID to buy a hammer at Home Depot, and demanding "safety" training of text continuation models is like requiring hammers be redesigned to have pillows for hammerheads.

We already have laws against bad acts. Hold wrongdoers accountable. Leave hammers alone.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#465

Post by blksv » Sun, May 19, 2024, 2:35 PM UTC

0xDEAFBEAD wrote: post #392

blksv wrote: post #326

Earlier quoted context omitted.

But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. I agree, that AI…

>But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. One gorilla says…

> "Those human brains are just synapses firing. They depend on nature to survive. Not a problem"

When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs.

> Suppose we create a server emulation of a highly intelligent, manipulative serial killer How do you feel about this?

Quite indifferent: the only field I can see for such a simulation is game development, but that would be huge overkill.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#466

Post by ben_w » Sun, May 19, 2024, 2:46 PM UTC

RandomLensman wrote: post #445

ben_w wrote: post #439

Earlier quoted context omitted.

> If they were really super effective broadly, then Trump would have won any popular vote easily Trump is up against all the other equally effective communicators who are both allowed to and motivated to hold that specific office.

Yes, but again it shows there isn't necessarily a dominant way to communicate. Convincing people is hard.

This does not appear to dispute the claim that mere communication is an effective way to have a lot of power.

Do you think I was claiming someone might be able to demonstrate absolute power, rather than mere relative power? Even Stalin didn't have that, but he would've been one of the closest.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#467

Post by onthecanposting » Sun, May 19, 2024, 2:50 PM UTC

renegade-otter wrote: post #443

onthecanposting wrote: post #442

Earlier quoted context omitted.

The productive output of many people is also just mediocre. I think of chat AIs as near-zero cost interns or the boss's nephew. Probably doesn't know what it's talking about, but can get a lot done with supervision. I hope the net result will be a shift toward upskilling employees to achieve the expertise handle the difficult edge cases and less time spent on the routine or the trivial. Seems like a win-win. Incentiv…

I think it was on the Ezra Klein show where he dropped the statistic? Almost 70% of employees are using ChatGPT in one way or another. In many cases the bosses don't even know, but all it will do long-term is require everyone to produce more output. AI is not about to create 3-day work weeks and everyone living their best lives off of Universal Basic Income. Literally nothing will change - except that all of us will…

It might not be that bad. Auto workers produce more cars per employee, but some of the really repetitive activities are done by robots now. I think not having to do the same weld 8+hrs a day every day with consumers still getting their car is a net positive for owners, consumers, and labor.

Personally, I would love to deliver more projects. The vast majority of my time is spent on mind-numbing drudgery that makes the job extremely unsatisfying. If someone offered me a job where I got to focus on the fun parts of my work, and I didn't have to dump the drudgery on a poor intern (the AI does it), but at half my pay, I'd resign the same day.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#468

Post by RandomLensman » Sun, May 19, 2024, 2:55 PM UTC

ben_w wrote: post #466

RandomLensman wrote: post #445

Earlier quoted context omitted.

Yes, but again it shows there isn't necessarily a dominant way to communicate. Convincing people is hard.

This does not appear to dispute the claim that mere communication is an effective way to have a lot of power. Do you think I was claiming someone might be able to demonstrate absolute power, rather than mere relative power? Even Stalin didn't have that, but he would've been one of the closest.

None of it is mere communication: Need to organise events and travel schedules, create (or shape) organizations, find funds, etc. Communication is perhaps the mostly "visible" part, though.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#469

Post by sashank_1509 » Sun, May 19, 2024, 3:09 PM UTC

plaidfuji wrote: post #63

sashank_1509 wrote: post #54

Earlier quoted context omitted.

AGI seems like a stretch when none of the LLM’s right now can solve the prompt: “ Write a sentence that has an odd number of words, with the third word being "keyboard", the last word being "anyway", the next-to-last word being a number, and the first word being a palindrome “ Do we have to add these examples to the dataset now, so that they can solve this? Then maybe we find another prompt and add that too, as a hum…

Interesting, gpt-4o took seven tries to do this, and I had to give it very specific feedback about what it was doing wrong. Here was the answer: Level says keyboard broke down today, needing repair within five anyway.

I would prefer if it can give the number 5 instead of five. Also I don’t like to tell it what exactly it got wrong, just that it got it wrong. Saying things like think step by step, focus on rules etc is all fair game, while saying “keyboard is the second word, not third”, imo is not. I tried quite a bit when I saw this example on twitter and could never get it to work.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#470

Post by tarsinge » Sun, May 19, 2024, 3:31 PM UTC

sensanaty wrote: post #430

The talk of AI exterminating us via paperclips is an annoying one to me that detracts from the real dangers that are currently unfolding: - scammers are having the time of their lives, and their capabilities are only going to get better and easier - people are actively getting dumber as they trust the hallucinated outputs blindly - regular communication is starting to break down because you can't trust that people ar…

Depends which people and whose communications. I was in the farmers market of my small rural town this morning and it's funny how all of this simply don't exist here. No smartphones in sight, people are here physically, they interact, buy local food, laugh, have a drink, etc... Then they go eat at some friend's place and have some outdoor activities. On the other hand yes it looks bleak for terminally online people.