Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#5

Post by Animats » Tue, Jun 03, 2025, 9:28 PM UTC

This seems to be a funding proposal for "Scientist AI."[1] Start reading around page 21. They're arguing for "model-based AI", with a "world model". But they're vague about what form that "world model" takes.

This is a good idea if you can do it. But people have been bashing their head against that problem for decades. That's what Cyc was all about - building a world model of some kind.

Is there any indication there that they actually know how to build this thing?

[1] https://arxiv.org/pdf/2502.15657

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#6

Post by didibus » Tue, Jun 03, 2025, 9:28 PM UTC

Interesting thing to keep an eye on.

Though personally, I'm not sure if I'm most scared of issues of safety with the models themselves, or more so in the impact these models will have on people's well being, lifestyles, and so on, which might fall under human law.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#7

Post by yumraj » Tue, Jun 03, 2025, 9:31 PM UTC

nemomarx wrote: post #4

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

Won’t neutering a model by using only safe data for training create a safe model?

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#8

Post by candiddevmike » Tue, Jun 03, 2025, 9:36 PM UTC

nemomarx wrote: post #4

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

> basically just prompting it extra hard

If prompting got me into this mess, why can't it get me out of it?

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#9

Post by fidotron » Tue, Jun 03, 2025, 9:37 PM UTC

Animats wrote: post #5

This seems to be a funding proposal for "Scientist AI."[1] Start reading around page 21. They're arguing for "model-based AI", with a "world model". But they're vague about what form that "world model" takes. This is a good idea if you can do it. But people have been bashing their head against that problem for decades. That's what Cyc was all about - building a world model of some kind. Is there any indication there…

> Is there any indication there that they actually know how to build this thing?

Nope. And it's exactly what they were trying to do at Element AI, where the dream was to build one model that knew everything, could explain everything, be biased in the exact required ways, and be tranferred easily to any application by their team of consultants.

At least these days the pretense of profit has been abandoned, but I hope it's not going to be receiving any government funding.

Re: Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

#10

Post by glitchc » Tue, Jun 03, 2025, 9:40 PM UTC

nemomarx wrote: post #4

Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

Yes it's unlikely that hard safety rules are possible for general intelligence. After billions of years of trying, the best biology has been able to do is incentivize certain behaviours. The only way to prevent seems to be to kill the organism for trying. I'm not sure if we can do better than evolution.