Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gavin Newsom vetoes SB 1047

gov.ca.gov

Post reply on HN
Print view View original

41–50 of 514 posts

Re: Gavin Newsom vetoes SB 1047

#41

Post by jpk »

JoshTriplett wrote: post #17
jprete wrote: post #6

Earlier quoted context omitted.

"The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are deployed in high-risk situations, he said in his veto message." That doesn't mean you're wrong , but it's not what Newsom signaled.

Only applying to the biggest models is the point ; the biggest models are the inherently high-risk ones. The larger they get, the more that running them at all is the "high-risk situation". Passing this would not have been a complete solution, but it would have been a step in the right direction. This is a huge disappointment.

> running them at all is the "high-risk situation"

What is the actual, concrete concern here? That a model "breaks out", or something?

The risk with AI is not in just running models, the risk is becoming overconfident in them, and then putting them in charge of real-world stuff in a way that allows them to do harm.

Hooking a model up to an effector capable of harm is a deliberate act requiring assurance that it doesn't harm -- and if we should regulate anything, it's that. Without that, inference is just making datacenters warm. It seems shortsighted to set an arbitrary limit on model size when you can recklessly hook up a smaller, shittier model to something safety-critical, and cause all the havoc you want.

Re: Gavin Newsom vetoes SB 1047

#42

Post by SonOfLilit »

atemerev wrote: post #20
SonOfLilit wrote: post #13

A bill laying the groundwork to ensure the future survival of humanity by making companies on the frontier of AGI research responsible for damages or deaths caused by their models, was vetoed because it doesn't stifle competition with the big players enough and because we don't want companies to be scared of letting future models capable of massive hacks or creating mass casualty events handle their customer support.…

Oh come on, the entire bill was against open source models, it’s pure business. “AI safety”, at least of the X-risk variety, is a non-issue.

I find it hard to believe that Google, Microsoft and OpenAI would oppose a bill against open source models.

Re: Gavin Newsom vetoes SB 1047

#43

Post by comp_throw7 »

tbrownaw wrote: post #18

https://legiscan.com/CA/text/SB1047/id/3019694 So this is the one that would make it illegal to provide open weights for models past a certain size, would make it illegal to sell enough compute power to train such a model without first verifying that your customer isn't going to train a model and then ignore this law, and mandates audit requirements to prove that your models won't help people cause disasters and can…

> this is the one that would make it illegal to provide open weights for models past a certain size

That's nowhere in the bill, but plenty of people have been confused into thinking this by the bill's opponents.

Re: Gavin Newsom vetoes SB 1047

#44

Post by whimsicalism »

akira2501 wrote: post #40
whimsicalism wrote: post #37

Earlier quoted context omitted.

This snide dismissiveness around “sci-fi” scenarios, while capabilities continue to grow, seems incredibly naïve and foolish. Many of you saying stuff like this were the same naysayers who have been terribly wrong about scaling for the last 6-8 years or people who only started paying attention in the last two years.

> seems incredibly naïve and foolish. We have electrical codes. These require disconnects just about everywhere. The notion that any system somehow couldn't be "turned off" with or without the consent of the operator is downright laughable. > were the same naysayers Now who's being snide and dismissive? Do you want to argue the point or are you just interested in tossing ad hominem attacks around?

> We have electrical codes. These require disconnects just about everywhere. The notion that any system somehow couldn't be "turned off" with or without the consent of the operator is downright laughable.

Not so clear when you are inferencing a distributed model across the globe. Doesn't seem obvious that shutdown of a distributed computing environment will always be trivial.

> Now who's being snide and dismissive?

Oh to be clear, nothing against being dismissive - just the particular brand of dismissiveness of 'scifi' safety scenarios is naive.

Re: Gavin Newsom vetoes SB 1047

#45

Post by SonOfLilit »

datavirtue wrote: post #36
SonOfLilit wrote: post #13

A bill laying the groundwork to ensure the future survival of humanity by making companies on the frontier of AGI research responsible for damages or deaths caused by their models, was vetoed because it doesn't stifle competition with the big players enough and because we don't want companies to be scared of letting future models capable of massive hacks or creating mass casualty events handle their customer support.…

The future survival of humanity involves creating machines that have all of our knowledge and which can replicate themselves. We can't leave the planet but our robot children can. I just wish that I could see what they become.

Sure, that's future survival. Is it of humanity though? Kinda no by definition in your scenario. In general, depends at least if they share our values...

Re: Gavin Newsom vetoes SB 1047

#46

Post by zamadatix »

whimsicalism wrote: post #37
akira2501 wrote: post #33

Earlier quoted context omitted.

> and mandates audit requirements to prove that your models won't help people cause disasters Audits cannot prove anything and they offer no value when planning for the future. They're purely a retrospective tool that offers insights into potential risk factors. > and can be turned off. I really wish legislators would operate inside reality instead of a Star Trek episode.

This snide dismissiveness around “sci-fi” scenarios, while capabilities continue to grow, seems incredibly naïve and foolish. Many of you saying stuff like this were the same naysayers who have been terribly wrong about scaling for the last 6-8 years or people who only started paying attention in the last two years.

I don't think GP is dismissing the scenarios themselves, rather espousing their belief these answers will do nothing to prevent said scenarios from eventually occuring anyways. It's like if we invented nukes but found out they were made out of having a lot of telephones instead of something exotic like refining radioactive elements a certain way. Sure - you can still try to restrict telephone sales... but one way or another lots of nukes are going to be built around the world (power plants too) and, in the meantime, what you've regulated away is the convenience of having a better phone from the average person as time goes on.

The same battle was/is had around cryptography - telling people they can't use or distribute cryptography algorithms on consumer hardware never stopped bad people from having real time functionally unbreakable encryption.

The safety plan must be around somehow handling the resulting problems when they happen, not hoping to make it never occur even once for the rest of time. Eventually a bad guy is going to make an indecipherable call, eventually an enemy country or rogue operator is going to nuke a place, eventually an AI is going to ${scifi_ai_thing}. The safety of all society can't rest on audits and good intention preventing those from ever happening.

Re: Gavin Newsom vetoes SB 1047

#47

Post by comp_throw7 »

jprete wrote: post #6
JoeAltmaier wrote: post #2

Perhaps worried that draconian restriction on new technology is not gonna help bring Silicon Valley back to preeminence.

"The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are deployed in high-risk situations, he said in his veto message." That doesn't mean you're wrong , but it's not what Newsom signaled.

He's dissembling. He vetoed the bill because VCs decided to rally the flag; if the bill had covered more models he'd have been more likely to veto it, not less.

It's been vaguely mindblowing to watch various tech people & VCs argue that use-based restrictions would be better than this, when use-based restrictions are vastly more intrusive, economically inefficient, and subject to regulatory capture than what was proposed here.

Re: Gavin Newsom vetoes SB 1047

#48

Post by jart »

jprete wrote: post #6
JoeAltmaier wrote: post #2

Perhaps worried that draconian restriction on new technology is not gonna help bring Silicon Valley back to preeminence.

"The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are deployed in high-risk situations, he said in his veto message." That doesn't mean you're wrong , but it's not what Newsom signaled.

If you read Gavin Newsom's statement, it sounds like he agrees with Terrance Tao's position, which is that the government should regulate the people deploying AI rather than the people inventing AI. That's why he thinks it should be stricter. For example, you wouldn't want to lead people to believe that AI in health care decisions is OK so long as it's smaller than 10^26 flops. Read his full actual statement here: https://www.gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Ve...

Re: Gavin Newsom vetoes SB 1047

#49

Post by pkage »

jpk wrote: post #41
JoshTriplett wrote: post #17

Earlier quoted context omitted.

Only applying to the biggest models is the point ; the biggest models are the inherently high-risk ones. The larger they get, the more that running them at all is the "high-risk situation". Passing this would not have been a complete solution, but it would have been a step in the right direction. This is a huge disappointment.

> running them at all is the "high-risk situation" What is the actual, concrete concern here? That a model "breaks out", or something? The risk with AI is not in just running models, the risk is becoming overconfident in them, and then putting them in charge of real-world stuff in a way that allows them to do harm. Hooking a model up to an effector capable of harm is a deliberate act requiring assurance that it doesn…

There is no concrete concern past "models that can simulate thinking are scary." The risk has always been connecting models to systems which are safety critical, but for some reason the discourse around this issue has been more influenced by Terminator than OSHA.

As a researcher in the field, I believe there's no risk beyond overconfident automation---and we already have analogous legislation for automations, for example in what criteria are allowable and not allowable when deciding whether an individual is eligible for a loan.

Post reply on HN