Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gavin Newsom vetoes SB 1047

gov.ca.gov

Post reply on HN
Print view View original

91–100 of 514 posts

Re: Gavin Newsom vetoes SB 1047

#91

Post by llamaimperative »

gdiamos wrote: post #87
llamaimperative wrote: post #82

Earlier quoted context omitted.

If you spent $100MM+ on training.

Advanced technology will drop the cost of training. The flop targets in that bill would be like saying “640KB of memory is all we will ever need” and outlawing anything more. Imagine what other countries would have done to us if we allowed a monopoly like that on memory in 1980.

No, there are two thresholds and BOTH must be met.

One of those is $100MM in training costs.

The other is measured in FLOPs but is already larger than GPT-4, so the “think of the small guys!” argument doesn’t make much sense.

Re: Gavin Newsom vetoes SB 1047

#92

Post by Cupertino95014 »

worstspotgain wrote: post #52

Excellent move by Newsom. We have a very active legislature, but it's been extremely bandwagon-y in recent years. I support much of Wiener's agenda, particularly his housing policy, but this bill was way off the mark. It was basically a torpedo against open models. Market leaders like OpenAI and Anthropic weren't really worried about it, or about open models in general. Its supporters were the also-rans like Musk [1]…

[flagged]

Re: Gavin Newsom vetoes SB 1047

#93

Post by Loughla »

akira2501 wrote: post #33
tbrownaw wrote: post #18

https://legiscan.com/CA/text/SB1047/id/3019694 So this is the one that would make it illegal to provide open weights for models past a certain size, would make it illegal to sell enough compute power to train such a model without first verifying that your customer isn't going to train a model and then ignore this law, and mandates audit requirements to prove that your models won't help people cause disasters and can…

> and mandates audit requirements to prove that your models won't help people cause disasters Audits cannot prove anything and they offer no value when planning for the future. They're purely a retrospective tool that offers insights into potential risk factors. > and can be turned off. I really wish legislators would operate inside reality instead of a Star Trek episode.

>I really wish legislators would operate inside reality instead of a Star Trek episode.

What are your thoughts about businesses like Google and Meta providing guidance and assistance to legislators?

Re: Gavin Newsom vetoes SB 1047

#94

Post by Terr_ »

jart wrote: post #48
jprete wrote: post #6

Earlier quoted context omitted.

"The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are deployed in high-risk situations, he said in his veto message." That doesn't mean you're wrong , but it's not what Newsom signaled.

If you read Gavin Newsom's statement, it sounds like he agrees with Terrance Tao's position, which is that the government should regulate the people deploying AI rather than the people inventing AI. That's why he thinks it should be stricter. For example, you wouldn't want to lead people to believe that AI in health care decisions is OK so long as it's smaller than 10^26 flops. Read his full actual statement here: ht…

> the government should regulate the people deploying AI rather than the people inventing AI

Yeah, there's no point having system that is made the most scrupulous of standards and then someone else deploys it in an evil way. (Which in some cases can be done simply by choosing to do the opposite of whatever a good model recommends.)

Re: Gavin Newsom vetoes SB 1047

#95

Post by SonOfLilit »

hn_throwaway_99 wrote: post #80
SonOfLilit wrote: post #66

Earlier quoted context omitted.

why would Google, Microsoft and OpenAI oppose a torpedo against open models? Aren't they positioned to benefit the most?

Yeah, I think the argument that "this just hurts open models" makes no sense given the supporters/detractors of this bill. The thing that large companies care the most about in the legal realm is certainty . They're obviously going to be a big target of lawsuits regardless, so they want to know that legislation is clear as to the ways they can act - their biggest fear is that you get a good "emotional sob story" in f…

Sob stories are definitely not covered by the text of the bill. The "critical harm" clause (ctrl-f this comment section for a full quote) is all about nuclear weapons and massive hacks and explicitly excludes "just" someone dying or getting injured with very clear language.

Re: Gavin Newsom vetoes SB 1047

#96

Post by trog »

akira2501 wrote: post #33
tbrownaw wrote: post #18

https://legiscan.com/CA/text/SB1047/id/3019694 So this is the one that would make it illegal to provide open weights for models past a certain size, would make it illegal to sell enough compute power to train such a model without first verifying that your customer isn't going to train a model and then ignore this law, and mandates audit requirements to prove that your models won't help people cause disasters and can…

> and mandates audit requirements to prove that your models won't help people cause disasters Audits cannot prove anything and they offer no value when planning for the future. They're purely a retrospective tool that offers insights into potential risk factors. > and can be turned off. I really wish legislators would operate inside reality instead of a Star Trek episode.

> Audits cannot prove anything and they offer no value when planning for the future. They're purely a retrospective tool that offers insights into potential risk factors.

Uh, aren't potential risk factors things you want to consider when planning for the future?

Re: Gavin Newsom vetoes SB 1047

#97

Post by dale_glass »

KoolKat23 wrote: post #60
jpk wrote: post #41

Earlier quoted context omitted.

> running them at all is the "high-risk situation" What is the actual, concrete concern here? That a model "breaks out", or something? The risk with AI is not in just running models, the risk is becoming overconfident in them, and then putting them in charge of real-world stuff in a way that allows them to do harm. Hooking a model up to an effector capable of harm is a deliberate act requiring assurance that it doesn…

Well this is exactly why there's a minimum scale of concern. Below a certain scale it's less complicated and answers are more predictable and alignment can be ensured. Bigger models how do you determine your confidence if you don't know what's it's thinking? There's already evidence in o1 red-teaming, the model was trying to game the researcher's checks.

Yeah, but what if you take a stupid, below the "certain scale" limit model and hook it up to something important, like a nuclear reactor or a healthcare system?

The point is that this is a terrible way to approach things. The model itself isn't what creates the danger, it's what you hook it up to. A model 100 times larger than the current available that's just sending output into /dev/null is completely harmless.

A small, below the "certain scale" model used for something important like healthcare could be awful.

Re: Gavin Newsom vetoes SB 1047

#98

Post by SonOfLilit »

handfuloflight wrote: post #72
SonOfLilit wrote: post #55

I wondered if the article was over-dramatizing what risks were covered by the bill, so I read the text: (g) (1) “Critical harm” means any of the following harms caused or materially enabled by a covered model or covered model derivative: (A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties. (B) Mass casualties or at least five hundred million d…

Does Newsom believe that an AI model can do this damage autonomously or does he understand it must be wielded and overseen by humans to do so? In that case, how much of an enabler is an AI to meet the destructive ends, when, if the humans can use AI to conduct the damage, they can surely do it without the AI as well. The potential for destruction exists either way but is the concern that AI makes this more accessible…

“Critical harm” does not include any of the following: (A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative.

The bogeyman is not these models, it's future agentic autonomous ones, if and when they can hack major infrastructure or build nukes. The quoted text is very very clear on that.

Re: Gavin Newsom vetoes SB 1047

#99

Post by derektank »

marshray wrote: post #65
zamadatix wrote: post #46

Earlier quoted context omitted.

I don't think GP is dismissing the scenarios themselves, rather espousing their belief these answers will do nothing to prevent said scenarios from eventually occuring anyways. It's like if we invented nukes but found out they were made out of having a lot of telephones instead of something exotic like refining radioactive elements a certain way. Sure - you can still try to restrict telephone sales... but one way or…

It's an interesting analogy. Nukes are a far more primitive technology (i.e., enrichment requires only more basic industrial capabilities) than AI hardware, yet they are probably the best example of tech limitations via international agreements. But the algorithms are mostly public knowledge, datacenters are no secret, and the chips aren't even made in the US. I don't see what leverage California has to regulate AI b…

>So it seems like the only thing such a bill would achieve is to incentivize AI research to avoid California.

Which, incidentally, would be pretty bad from a climate change perspective since many of the alternative locations for datacenters have a worse mix of renewables/nuclear to fossil fuels in their electricity generation. ~60% of VA's electricity is generated from burning fossil fuels (of which 1/12th is still coal) while natural gas makes up less than 40% of electricity generation in California, for example

Re: Gavin Newsom vetoes SB 1047

#100

Post by SonOfLilit »

worstspotgain wrote: post #73
SonOfLilit wrote: post #66

Earlier quoted context omitted.

why would Google, Microsoft and OpenAI oppose a torpedo against open models? Aren't they positioned to benefit the most?

If there was just one quasi-monopoly it would have probably supported the bill. As it is, the market leaders have the competition from each other to worry about. Getting rid of open models wouldn't let them raise their prices much.

So if it's not them, who is the hidden commercial interest sponsoring an attack on open source models that cost >$100mm to train? Or does Wiener just genuinely hate megabudget open source? Or is it an accidental attack, aimed at something else? At what?
Post reply on HN