Live data from Hacker News

Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

lawzero.org

21–30 of 40 posts

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#21
post #11
post #7

Earlier quoted context omitted.

Won’t neutering a model by using only safe data for training create a safe model?

Can we call it general intelligence then? Is human intelligence not the sum of both good and bad people?

Maybe I'm looking at it very literally, but the above simply mentions "safe-by-design AI systems", there is no mention of the target being general intelligence.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#22

Earlier quoted context omitted.

> basically just prompting it extra hard If prompting got me into this mess, why can't it get me out of it?

https://en.wikipedia.org/wiki/Brandolini%27s_law

Hey, following that rule precisely, we just need 10x longer security prompts :)

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#23

A nonprofit, just like OpenAI ... I don't get the "safe AI" crowd, it's all ghost and mirrors IMO. It's been almost a year to the date since Ilya got his first billion. Later, another two billion came in. Nothing to show. I'm honestly curious since I don't think Ilya is a scammer, but I can't imagine what kind of product they pretend to bring to the market.

AI safety is a genuinely hard problem.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#24
post #7
post #4

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

Won’t neutering a model by using only safe data for training create a safe model?

No, because soon they will be able to learn. You'd need to project its thoughts or actions into a safe subspace as it learns and acts to make volitional disaster impossible, not unlikely. This would make it less intelligent, but still plenty capable.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#25
post #4

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

> basically just prompting it extra hard If prompting got me into this mess, why can't it get me out of it?

Prompting is like XML, which is like violence

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#26
Asimov's Zeroth Law of robotics:

  A robot may not harm humanity, or, by inaction, allow humanity to come to harm.
"Robots and Empire" is a nice discussion of the perils of LawZero. IMHO if successful it necessarily transfers human agency to bots, which we should be strenuously working to avoid, not accelerate.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#27
post #19

This guys annoys me a an entrepreneur because he gets a sh*t ton of government money and it starves the rest of the ecosystem in Montreal. The previous startup he made with that public money essentially failed. But he is some kind of hero of AI so it's an easy sell for politicians that need to demonstrate they are doing something about AI.

This is misinformation and you are sharing some very dangerous things online.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#28

A nonprofit, just like OpenAI ... I don't get the "safe AI" crowd, it's all ghost and mirrors IMO. It's been almost a year to the date since Ilya got his first billion. Later, another two billion came in. Nothing to show. I'm honestly curious since I don't think Ilya is a scammer, but I can't imagine what kind of product they pretend to bring to the market.

AI safety is a genuinely hard problem.

Indeed.

I just can't wrap my head about what the actual product/service is. Let alone something that could be sold for billions.

"Safe AI" is very ambiguous in terms of product.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#29
post #19

This guys annoys me a an entrepreneur because he gets a sh*t ton of government money and it starves the rest of the ecosystem in Montreal. The previous startup he made with that public money essentially failed. But he is some kind of hero of AI so it's an easy sell for politicians that need to demonstrate they are doing something about AI.

This is misinformation and you are sharing some very dangerous things online.

It reads like possibly slander, but dangerous? I don't understand how it could be dangerous

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#30
post #29

Earlier quoted context omitted.

This is misinformation and you are sharing some very dangerous things online.

It reads like possibly slander, but dangerous? I don't understand how it could be dangerous

The sentiment is real in Montréal for the rest of whomever wasn't holding on to the coattails of the government's golden-boy. $100M and what to show for it? A cool office in Rosemont? That company was fucked.
Post reply on HN