Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gavin Newsom vetoes SB 1047

gov.ca.gov

Post reply on HN
Print view View original

451–460 of 514 posts

Re: Gavin Newsom vetoes SB 1047

#451

Post by ldbooth »

iluvcommunism wrote: post #447
ldbooth wrote: post #391

Earlier quoted context omitted.

Same reason the governor appointed public utility commission has allowed PG&E to raise rates 4 times in a single year without legitimate oversight. Yea unfortunately all roads point to his donors with this smooth talker, cost of living be damned.

On the east coast we don’t need the government to control electricity prices. And our electricity is cheaper. Go figure.

Companies like Dominion Energy, Duke Energy, and Consolidated Edison are regulated by state utility commissions, same as in California.

Re: Gavin Newsom vetoes SB 1047

#452

Post by taurath »

burningChrome wrote: post #450
inferiorhuman wrote: post #375

Earlier quoted context omitted.

Newsom vetoed the bill as a nod to his donors plain and simple. Same reason he just signed a bill allowing a specific customers at a single venue to be served alcohol later than 2 AM. Same reason he carved out a minimum wage exemption for Panera. Same reason he signed a bill to carve out a junk fee exemption specifically for restaurants. He's just planning for a post-governor career.

>> He's just planning for a post-governor career. After this year, many democrats are as well - which is why Harris had such a hard time finding a VP and took Walz who was like the last kid you pick on your dodgeball team. The presidential race in 2028 for the Democrats is going to have one of the deepest benches for talent I've seen in a long time. Newsom and Shapiro will be at the top of the list for sure. But I ag…

> Newsom and Shapiro will be at the top of the list for sure.

Neither has genuine appeal. Shapiro is a really, really poor speaker and has few credentials except as a moderate. Newsom is the definition of coastal elite. Both have spoken, neither have been heard.

Re: Gavin Newsom vetoes SB 1047

#453

Post by JoshTriplett »

pkage wrote: post #49
jpk wrote: post #41

Earlier quoted context omitted.

> running them at all is the "high-risk situation" What is the actual, concrete concern here? That a model "breaks out", or something? The risk with AI is not in just running models, the risk is becoming overconfident in them, and then putting them in charge of real-world stuff in a way that allows them to do harm. Hooking a model up to an effector capable of harm is a deliberate act requiring assurance that it doesn…

There is no concrete concern past "models that can simulate thinking are scary." The risk has always been connecting models to systems which are safety critical, but for some reason the discourse around this issue has been more influenced by Terminator than OSHA. As a researcher in the field, I believe there's no risk beyond overconfident automation---and we already have analogous legislation for automations, for exa…

> There is no concrete concern

This is false. You are dismissing the many concrete concerns people have expressed. Whether you agree with those concerns is immaterial. Feel free to argue against those concerns, but claiming there are no concerns is a false and unsupported assertion.

> but for some reason the discourse around this issue has been more influenced by Terminator than OSHA.

1) Claiming that concerns about AGI are in any way about "Terminator" is dismissive rhetoric that doesn't take the actual concerns seriously.

2) There are also, separately, risks about using models and automation unthinkingly in ways that harm people. Those risk should also be addressed. Those efforts shouldn't subvert or co-opt the efforts to prevent models from getting out of control, which was the point of this bill.

Re: Gavin Newsom vetoes SB 1047

#454

Post by JoshTriplett »

dale_glass wrote: post #97
KoolKat23 wrote: post #60

Earlier quoted context omitted.

Well this is exactly why there's a minimum scale of concern. Below a certain scale it's less complicated and answers are more predictable and alignment can be ensured. Bigger models how do you determine your confidence if you don't know what's it's thinking? There's already evidence in o1 red-teaming, the model was trying to game the researcher's checks.

Yeah, but what if you take a stupid, below the "certain scale" limit model and hook it up to something important, like a nuclear reactor or a healthcare system? The point is that this is a terrible way to approach things. The model itself isn't what creates the danger, it's what you hook it up to. A model 100 times larger than the current available that's just sending output into /dev/null is completely harmless. A s…

> A model 100 times larger than the current available that's just sending output into /dev/null is completely harmless.

That's certainly a hypothesis. What level of confidence should be required of that hypothesis before risking all of humanity on it? Who should get to evaluate that confidence level and make that decision?

One way of looking at this: If a million smart humans, thinking a million times faster, with access to all knowledge, were in this situation, could they break out? Are there any flaws in the chip they're running on? Will running code on the system emitting any interesting RF, and could nearby systems react to that RF in any useful fashion? Across all the code interacting with the system, would any possible single-bit error open up any avenues for exploit? Are other AI systems with similar/converged goals being used to design the systems interacting with this one? What's the output actually going to, because any form of analysis isn't equivalent to /dev/null, and may be exploitable.

Re: Gavin Newsom vetoes SB 1047

#455

Post by inferiorhuman »

ldbooth wrote: post #391
inferiorhuman wrote: post #375

Earlier quoted context omitted.

Newsom vetoed the bill as a nod to his donors plain and simple. Same reason he just signed a bill allowing a specific customers at a single venue to be served alcohol later than 2 AM. Same reason he carved out a minimum wage exemption for Panera. Same reason he signed a bill to carve out a junk fee exemption specifically for restaurants. He's just planning for a post-governor career.

Same reason the governor appointed public utility commission has allowed PG&E to raise rates 4 times in a single year without legitimate oversight. Yea unfortunately all roads point to his donors with this smooth talker, cost of living be damned.

Oh yeah I forgot this one — basically making it easier to force neighborhoods to abandon their natural gas infrastructure. Something I'd be in favor of were it not for the constant stream of electric rate hikes.

https://www.kqed.org/news/12006711/newsom-signs-bill-to-help...

Re: Gavin Newsom vetoes SB 1047

#456

Post by dale_glass »

JoshTriplett wrote: post #454
dale_glass wrote: post #97

Earlier quoted context omitted.

Yeah, but what if you take a stupid, below the "certain scale" limit model and hook it up to something important, like a nuclear reactor or a healthcare system? The point is that this is a terrible way to approach things. The model itself isn't what creates the danger, it's what you hook it up to. A model 100 times larger than the current available that's just sending output into /dev/null is completely harmless. A s…

> A model 100 times larger than the current available that's just sending output into /dev/null is completely harmless. That's certainly a hypothesis. What level of confidence should be required of that hypothesis before risking all of humanity on it? Who should get to evaluate that confidence level and make that decision? One way of looking at this: If a million smart humans, thinking a million times faster, with ac…

> That's certainly a hypothesis. What level of confidence should be required of that hypothesis before risking all of humanity on it? Who should get to evaluate that confidence level and make that decision?

We can have complete confidence because we know how LLMs work under the hood, what operations they execute. Which isn't much. There's just a lot of them.

> One way of looking at this: If a million smart humans, thinking a million times faster, with access to all knowledge, were in this situation, could they break out? Are there any flaws in the chip they're running on?

No. LLMs don't execute arbitrary code. They execute a whole lot of matrix multiplications.

Also, LLMs don't think. ChatGPT isn't plotting your demise in between requests. It's not doing anything. It's purely a receive request -> process -> output sort of process. If you're not asking it to do anything, it's not doing anything.

Fearing big LLMs is like fearing a good chess engine -- it sure computes a lot more than a weaker one, but in the end all that it's doing is computing chess moves. No matter how much horsepower we spend on that it's not going to ever do anything but play chess.

Re: Gavin Newsom vetoes SB 1047

#457

Post by worstspotgain »

burningChrome wrote: post #450
inferiorhuman wrote: post #375

Earlier quoted context omitted.

Newsom vetoed the bill as a nod to his donors plain and simple. Same reason he just signed a bill allowing a specific customers at a single venue to be served alcohol later than 2 AM. Same reason he carved out a minimum wage exemption for Panera. Same reason he signed a bill to carve out a junk fee exemption specifically for restaurants. He's just planning for a post-governor career.

>> He's just planning for a post-governor career. After this year, many democrats are as well - which is why Harris had such a hard time finding a VP and took Walz who was like the last kid you pick on your dodgeball team. The presidential race in 2028 for the Democrats is going to have one of the deepest benches for talent I've seen in a long time. Newsom and Shapiro will be at the top of the list for sure. But I ag…

2032, nice try though. Besides, we're not going to need to vote anymore otherwise, remember? The 2028 Democratic Primary would be a pro-forma affair between Barron vs. Mayor McCheese.

Re: Gavin Newsom vetoes SB 1047

#458

Post by woah »

alkonaut wrote: post #296

The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothing can be trusted. I'd be all for a law (however clumsy) that made image, audio or video content produced by models with over X parameters to be marked with metadata saying it's AI generated. Creating models that don't tag their output as such would be banned.…

> The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothing can be trusted. I'd be all for a law (however clumsy) that made image, audio or video content produced by models with over X parameters to be marked with metadata saying it's AI generated.

Just make a law that makes it so that AI content has to be tagged if it is being used for fraud

Re: Gavin Newsom vetoes SB 1047

#459

Post by SonOfLilit »

wslh wrote: post #183
SonOfLilit wrote: post #170

Earlier quoted context omitted.

Startups that spend >$100mm on one training run...

There are startups and startups, the ones that you read on media are just a fraction of the worldwide reality.

You misread my intention, I think. I was pointing out that startups that don't raise enough to train a GPT-4 are excluded from the bill, that requires >$100mm expenses for a model to be covered.
Post reply on HN