Live data from Hacker News

SB-1047 will stifle open-source AI and decrease safety

answer.ai

31–40 of 201 posts

Re: SB-1047 will stifle open-source AI and decrease safety

#31
post #30

Earlier quoted context omitted.

Miami has a population that is extremely ideologically opposed to laws like this.

They don’t seem to have any reason that there would be AI experts there though. No big comp sci universities or existing research labs and tech companies that you can pull talent from.

No but people go where money is. It sucks that California covers 80% of the West coast.

Re: SB-1047 will stifle open-source AI and decrease safety

#32
post #3

The politicians are gunning extremely hard for open source AI. It's crazy. "Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose…

I don’t really understand the long term plan, or maybe I don’t believe lawmakers understand where we are going long-term with this stuff.

We’re still in the very early days.

Unless the academic community really drops the ball, in 5 or so years they’ll be training models around the quality of the current state of the art on professors’ research clusters (probably not just at R1 universities).

I’d be shocked if, in the long term, anyone who can get access a library’s worth of text won’t be able to put together a useable model.

There’s nothing magical about our brains, so I imagine at some point you’ll be able to teach a computer to read and write with about as many books as it takes to teach a human. I mean maybe they’ll be, like, 10x as dumb as us. A typical American might read hundreds of books over the course of their life, what are they going to do, require a license to own more than a couple thousand e-books?

Re: SB-1047 will stifle open-source AI and decrease safety

#33

The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.

Open-source models runnable on consumer hardware now achieve comparable benchmark performance to the "absurd" models from a couple of years ago.

Re: SB-1047 will stifle open-source AI and decrease safety

#34
> The creator of a model can not ensure that a model is never used to do something harmful – any more so that the developer of a web browser, calculator, or word processor could. Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams.

This matches my thoughts on why this is ultimately a bad piece of legislation. It is virtually impossible to ensure that a piece of technology will not be used for "harmful purposes". I agree that such stipulations will be just another roadblock keeping everyone except "big businesses with well funded legal teams" from working on LLMs.

Re: SB-1047 will stifle open-source AI and decrease safety

#35
this feels a lot like Uber/Lyft whining about rideshare regulation, or Musk whining about self-driving regulation. Howard does a miserable job attempting to explain what, exactly, this legislation will do to impact open source development and clearly hasnt read the specifics for open-source development in the bill. things like dual use/ITAR apply explicitly to commercial products, but he attempts to conflate them with open source instead and uses inference to try and define them. they are very well defined trade concepts at the federal level.

from the bill, the specific applications to open source are:

Appoint and consult with an advisory committee for open-source artificial intelligence that shall do all of the following:

(A) Issue guidelines for model evaluation for use by developers of open-source artificial intelligence models that do not have hazardous capabilities.

(B) Advise the Frontier Model Division on the creation and feasibility of incentives, including tax credits, that could be provided to developers of open-source artificial intelligence models that are not covered models.

(C) Advise the Frontier Model Division on future policies and legislation impacting open-source artificial intelligence development.

nowhere does it state open source developers need "required shutdowns" or burdensome reporting for open source. The states position to regulate trade is sacrosanct and in such, the bill applies almost entirely to commercial products. it would affect Jeremys business and as a business owner, he doesnt like that.

here is the actual bill

https://legiscan.com/CA/text/SB1047/id/2919384

Re: SB-1047 will stifle open-source AI and decrease safety

#36
post #30

Earlier quoted context omitted.

They don’t seem to have any reason that there would be AI experts there though. No big comp sci universities or existing research labs and tech companies that you can pull talent from.

No but people go where money is. It sucks that California covers 80% of the West coast.

I don’t think that worked when Miami was being hyped over the last few years and I don’t expect that to work in the future. Needs a base of research and big tech to start.

Re: SB-1047 will stifle open-source AI and decrease safety

#37
I disagree on "Focus on Usage, Not Development".

> Regulate the use of AI in high-risk areas such as healthcare, criminal justice, and critical infrastructure, where the potential for harm is greatest

This suggests, for example, image generation should be unregulated, but potential for harm of deepfake is great. In general, regulation needs to be feasible to implement, and even if it is ideal to regulate use not development, it can make a sense to regulate development due to feasibility concerns.

Re: SB-1047 will stifle open-source AI and decrease safety

#38
While this article makes some valid points, it basically just ignores the reasons why the law is being passed, that is the potential for open-models to enable bio-attacks, cyberattacks, election manipulation, automated personalised scams, and who knows what else.

One might question why that is. Perhaps it's the case that Jeremy has an excellent response to these points which he has somehow neglected to raise. Or perhaps it's because these threats are very inconvenient for an open source developer.

I'm sure he'd say that open-sourcing models means that all actors have access to defensive systems and that the good guys outnumber the bad guys and it'll all work out well.

And that could be true. Or it could be false. It's not like we really know that everything would work out fine. It's not that we've run the experiment. I mean maybe it works out like that, or maybe one guy creates a virus and then it doesn't really matter how many folk on the other side, but we still get kind of screwed because we can only produce vaccines that fast. It's that's what going to happen? I don't really know, but it's at least plausible. I mean, maybe we'll automate all aspects of vaccine production and be able to respond much faster, but that's dependent on when we develop this technology vs. when AI starts significantly helping with bioweapons with someone then using it for an attack. And at that point it's all so uncertain and up in the air that it's seems rather strange for someone to suggest that it'll all be fine.

Re: SB-1047 will stifle open-source AI and decrease safety

#39

The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.

> least 10^26 floating point ops

It's dumb to create laws based on arbitrary technological limits. Computing power is still increasing exponentially, yesterday's supercomputer is tomorrows gaming gpu.

Re: SB-1047 will stifle open-source AI and decrease safety

#40
post #3

The politicians are gunning extremely hard for open source AI. It's crazy. "Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose…

I don’t really understand the long term plan, or maybe I don’t believe lawmakers understand where we are going long-term with this stuff. We’re still in the very early days. Unless the academic community really drops the ball, in 5 or so years they’ll be training models around the quality of the current state of the art on professors’ research clusters (probably not just at R1 universities). I’d be shocked if, in the…

> I don’t really understand the long term plan

The long term plan for any lawmaker is winning the next election. Anything further in the future doesn't matter much.

The long term plan for incumbents here might be building a large moat by regulatory capture.

Maybe incumbents are helping lawmakers. Do ut des.

Post reply on HN