Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Preparedness Challenge

openai.com

Post reply on HN
Print view View original

61–70 of 160 posts

Re: OpenAI Preparedness Challenge

#61

Post by GistNoesis »

mk_stjames wrote: post #17

This kinda of crowdsourcing just feels.... f'ing weird man. It's like if, after the 1993 WTC bombing, but before 9/11, the FBI and NY Port Authority went around asking people how they would attack NYC if they were to become terrorists and... then how they would suggest detecting and stopping said attack. And please be as detailed as possible. Leave your name and phone number. Best answer gets season tickets to the Ya…

Sama: Yeah so if you ever need some dataset for an evil AI.

Sama: Just ask.

Sama: I have over 4000 ideas, how-tos, guides for and by malicious actors,

[Redacted Friend's Name]: What? How"d you manage that one?

Sama: People just submitted it.

Sama: I don't know why.

Sama: They "trust me"

Sama: Dumb f***s.

Re: OpenAI Preparedness Challenge

#63

Post by haltist »

vdfs wrote: post #40
wseqyrku wrote: post #18

Earlier quoted context omitted.

It is actually a prompt. Notice there's also a bounty, they are basically paying for an agi-as-a-service subscription, that is, the internet people. I'd expect they will put up more "challenges" like this in the future.

> a malicious actor might misuse these models to uncover a zero-day exploit in a government security system $25K seems low for "a malicious actor might misuse these models to uncover a zero-day exploit in a government security system.", this is not just a zero-day, this discovering the process of discovering zero-days.

Sam Altman is on the record about his belief that OpenAI is going to create a general intelligence system that can solve any well-posed challenge. That belief is based on the current success of LLMs as syntax co-pilots. So if you can formally specify what it means to have a zero-day exploit then presumably OpenAI's general intelligence system will then understand and "solve" it.

Many people have compared OpenAI to a cult and it is easy to see why. OpenAI should get credit for their efforts in making AI mainstream but there's a long way to go for automated zero-day exploits.

Re: OpenAI Preparedness Challenge

#64

Post by pveierland »

zupa-hu wrote: post #50
pveierland wrote: post #23

My vote would be to mandate a remote kill switch system to be installed in all sufficiently capable robotic entities, e.g. the humanoid robots being built by OpenAI/Tesla/Figure, that we are likely to see in the millions within decades. - The kill switch system can only be used to remotely deactivate a robot. - The kill switch system is not allowed to be developed or controlled by the robot manufacturer. - The kill s…

- Require two independent kill switches, one long range, one close range (say 5m). - Allow everyone to have close range kill switches, which use a universal open standard protocol that works on every robot (alas pepper spray for robots).

Agreed! Empowering people to have some power over robots through regulation, without being at the mercy of a single company, seems very important. Higher level behavior could involve a standardized safety language to command a robot to act slower or stop.

Re: OpenAI Preparedness Challenge

#65

Post by czbond »

Jensson wrote: post #20
laomai wrote: post #3

Maybe I’m reading this wrong: - ask people to get creative and give ideas for worst possible outcomes from use of AI and ways to prevent it. ..then give them a ton of credits for using said AI? .. well the first thing on my mind would be try the thing I just told you and see if it was really a risk or not? Is that what they expect people to do with the reward, or is this some unintended consequence?

We know OpenAI wants this field to get regulated to hell, so this looks like an attempt to generate arguments for AI regulations. The aim isn't to protect against AI but to protect against competitors, so it doesn't matter to them what you do with it.

I don't know that OpenAI does what it to be regulated. The EU was looking to enforce laws into providing auditable transparency into how decisions are made for suggestions - and OpenAI is freaked out by that.

If I recall, they were looking at having to pull out of the EU if enacted. The only company I am aware of currently looking to tackle AI Governance is Verses - they released a paper on it. https://www.verses.ai/ai-governance

Re: OpenAI Preparedness Challenge

#66

Post by [deleted] »

GistNoesis wrote: post #61
mk_stjames wrote: post #17

This kinda of crowdsourcing just feels.... f'ing weird man. It's like if, after the 1993 WTC bombing, but before 9/11, the FBI and NY Port Authority went around asking people how they would attack NYC if they were to become terrorists and... then how they would suggest detecting and stopping said attack. And please be as detailed as possible. Leave your name and phone number. Best answer gets season tickets to the Ya…

Sama: Yeah so if you ever need some dataset for an evil AI. Sama: Just ask. Sama: I have over 4000 ideas, how-tos, guides for and by malicious actors, [Redacted Friend's Name]: What? How"d you manage that one? Sama: People just submitted it. Sama: I don't know why. Sama: They "trust me" Sama: Dumb f***s.

[deleted]

Re: OpenAI Preparedness Challenge

#67

Post by JyB »

TheCaptain4815 wrote: post #7

While I applaud how much OpenAi fears these negatives, given the current state of Ai trajectory, it won't be long until a future open source model gets "uncensored" and is easily usable for tons and tons of malicious intent. There already exists a fantastic "uncensored" model with the newly released Dolphin Mistral 7b. I saw some results from others where the model could easily give explosives recipes from existing p…

> write racist poems

"catastrophic misuse of the model"

Re: OpenAI Preparedness Challenge

#69

Post by jazzyjackson »

heyheyhouhou wrote: post #14

I think we are already seeing really bad use cases already. Just go to youtube, newspapers, etc and see all the bot comments regarding the current Gaza situation. PS: I'm in a burner account because I'm afraid that my employer will kick me out for not agreeing with the methods of the "right side"

[flagged]

Re: OpenAI Preparedness Challenge

#70

Post by waterhouse »

HK-47: Answer: Yes. I believe my original Master needed this functionality in order to recover information from various indigenous tribes across the galaxy, but I know little else than that. Suffice to say that that translation capability allowed these... copies of myself to assume the role of protocol and translation droids in much of known space. That is, of course, not their primary function. And while they are attempting to pass themselves off as translation droids, their primary functionality keeps rising to the forefront.

HK-47: Recitation: For example, on Praven Prime, the simple transferring of L'Xing syntax for 'friendship' changes its meaning - and implies that one's brood mate was actually impregnated by their own host.

HK-47: Statement: This comment, of course, caused a civil war between the Gu-vandi Collective and L'Xing that still persists to the current date.

Post reply on HN