Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

lawzero.org

Post reply on HN
Print view View original

31–40 of 40 posts

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#31

Post by fidotron »

Sytten wrote: post #19

This guys annoys me a an entrepreneur because he gets a sh*t ton of government money and it starves the rest of the ecosystem in Montreal. The previous startup he made with that public money essentially failed. But he is some kind of hero of AI so it's an easy sell for politicians that need to demonstrate they are doing something about AI.

This is accurate, and what's impressive is how well this is scrubbed from the internet. For example: https://en.wikipedia.org/wiki/Element_AI

You'd have no idea about the fact most of the money came from the Quebec pension fund (which is then where the ServiceNow money went). For that you have to go to https://betakit.com/element-ai-announces-200-million-cad-ser... or https://www.cdpq.com/en/news/pressreleases/cdpq-expands-its-... Managing to spend $200M on AI in 2019 and having nothing to show for it in 2025. Quite impressive with hindsight.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#33

Post by rsfern »

glitchc wrote: post #10
nemomarx wrote: post #4

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

Yes it's unlikely that hard safety rules are possible for general intelligence. After billions of years of trying, the best biology has been able to do is incentivize certain behaviours. The only way to prevent seems to be to kill the organism for trying. I'm not sure if we can do better than evolution.

“Kill the [model] for trying” kind of sounds like using reinforcement learning to get models to behave a certain way

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#34

Post by saagarjha »

appleaday1 wrote: post #27
Sytten wrote: post #19

This guys annoys me a an entrepreneur because he gets a sh*t ton of government money and it starves the rest of the ecosystem in Montreal. The previous startup he made with that public money essentially failed. But he is some kind of hero of AI so it's an easy sell for politicians that need to demonstrate they are doing something about AI.

This is misinformation and you are sharing some very dangerous things online.

I think Hacker News is better when it doesn't involve vague threats.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#35

Post by jsnider3 »

moralestapia wrote: post #28
jsnider3 wrote: post #23

Earlier quoted context omitted.

AI safety is a genuinely hard problem.

Indeed. I just can't wrap my head about what the actual product/service is. Let alone something that could be sold for billions. "Safe AI" is very ambiguous in terms of product.

If you have a Safe AI, then becoming a billionaire is being an underachiever.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#36

Post by avmich »

glitchc wrote: post #10
nemomarx wrote: post #4

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

Yes it's unlikely that hard safety rules are possible for general intelligence. After billions of years of trying, the best biology has been able to do is incentivize certain behaviours. The only way to prevent seems to be to kill the organism for trying. I'm not sure if we can do better than evolution.

> I'm not sure if we can do better than evolution.

Surely we can, see aiplanes and rockets. There could be ideas why evolution didn't work in this case - like, too little time between humans getting power and conquering the planet - but in general, lack of proof isn't a proof of lack. So we still don't know if safety of this kind is possible.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#37

Post by moralestapia »

jsnider3 wrote: post #35
moralestapia wrote: post #28

Earlier quoted context omitted.

Indeed. I just can't wrap my head about what the actual product/service is. Let alone something that could be sold for billions. "Safe AI" is very ambiguous in terms of product.

If you have a Safe AI, then becoming a billionaire is being an underachiever.

Sure, but again, define "Safe AI" in terms of a product.

What exactly am I buying? How much I'm paying for it?

That's the thing I don't see.

Is it a model? `gpt-3.5-turbo-safe`?

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#38

Post by kbelder »

jsnider3 wrote: post #35
moralestapia wrote: post #28

Earlier quoted context omitted.

Indeed. I just can't wrap my head about what the actual product/service is. Let alone something that could be sold for billions. "Safe AI" is very ambiguous in terms of product.

If you have a Safe AI, then becoming a billionaire is being an underachiever.

Wouldn't all the money go to the unsafe AI, since it does more?

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#39

Post by afutjhkwsjug »

Sytten wrote: post #19

This guys annoys me a an entrepreneur because he gets a sh*t ton of government money and it starves the rest of the ecosystem in Montreal. The previous startup he made with that public money essentially failed. But he is some kind of hero of AI so it's an easy sell for politicians that need to demonstrate they are doing something about AI.

[dead]

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#40

Post by jsnider3 »

kbelder wrote: post #38
jsnider3 wrote: post #35

Earlier quoted context omitted.

If you have a Safe AI, then becoming a billionaire is being an underachiever.

Wouldn't all the money go to the unsafe AI, since it does more?

If someone invents an unsafe AI capable of making a billion dollars, then we will probably all die, which is why we should make safe AI instead.
Post reply on HN