Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SB-1047 will stifle open-source AI and decrease safety

answer.ai

Post reply on HN
Print view View original

181–190 of 201 posts

Re: SB-1047 will stifle open-source AI and decrease safety

#181

Post by jahewson »

casebash wrote: post #38

While this article makes some valid points, it basically just ignores the reasons why the law is being passed, that is the potential for open-models to enable bio-attacks, cyberattacks, election manipulation, automated personalised scams, and who knows what else. One might question why that is. Perhaps it's the case that Jeremy has an excellent response to these points which he has somehow neglected to raise. Or perh…

As someone who has studied both computer science and molecular biology at postgraduate level I can tell you that the chance of LLMs leading to higher probability of a “bio-attack” compared with a quick Google search is zero.

Do you know how much skill, practice, resourcing and time it takes to develop bio-anything?

Re: SB-1047 will stifle open-source AI and decrease safety

#182

Post by stult »

adamc wrote: post #129
ubj wrote: post #34

> The creator of a model can not ensure that a model is never used to do something harmful – any more so that the developer of a web browser, calculator, or word processor could. Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams. This matches my thoughts on why this is ultimately a b…

Someone who creates very dangerous items needs to take responsibility for them. Or their production needs to be very heavily regulated. That is just a reality. We don't let companies sell grenades on street corners. The running away from responsibility is one of the things I like least about big tech.

Sure, ultra-hazardous activities are regulated differently from other activities, including under tort law, but generic AI tools are not ultra-hazardous by nature. No piece of software is, until it is connected in some way to real world effects. Take an object-detection algorithm. There's absolutely nothing inherently dangerous about identifying objects in a video stream. But once you use the algorithm to create an automatic targeting system for a drone with a grenade strapped to it, it does become hazardous. But that's no reason to regulate the algorithm as if it were hazardous itself, at least no more so than it is to regulate the drone. As you point out, we regulate hand grenades. We do not regulate the boxes hand grenades are delivered in, or the web framework used for building a website that can be used to purchase hand grenades.

Re: SB-1047 will stifle open-source AI and decrease safety

#183

Post by andy99 »

wokwokwok wrote: post #147
andy99 wrote: post #121

Earlier quoted context omitted.

That's because you don't understand it.

Please don’t. It’s unhelpful to the argument when you do this, and it makes our side look like a bunch of smug self entitled assholes. The reality is that AI is disruptive but we don’t know how disruptive. The parent post is clearly hyperbole; but let’s push back on what is clearly nonsense (ie. AI being more dangerous than nuclear weapons) in a logical manner hm? Understanding AI is not the issue here; the issue so…

I disagree. Debating gives legitimacy, especially when one begins to debate a throwaway comment that doesn't even put an argument forward. The right answer is outright dismissal.

Re: SB-1047 will stifle open-source AI and decrease safety

#184

Post by humansareok1 »

phkahler wrote: post #105
humansareok1 wrote: post #54

Earlier quoted context omitted.

Bro given that the cost of frontier models is already at or past $100M I think that boat has already sailed. Unless you have a completely cracked team that can raise like $1B upfront you have no chance at competing.

Costs are coming down significantly every year.

At the same time compute required is increasing like 10x each generation, its scaling faster than Moore's law.

Re: SB-1047 will stifle open-source AI and decrease safety

#185

Post by brigadier132 »

ctoth wrote: post #20

This is a very, very bad article that reeks of LLM-generation. If you want to actually understand this bill, read Zvi's analysis: https://thezvi.substack.com/p/on-the-proposed-california-sb-...

Can we please focus on the substance of the article instead of trying to derail the discussion.

Re: SB-1047 will stifle open-source AI and decrease safety

#186

Post by photochemsyn »

ubj wrote: post #34

> The creator of a model can not ensure that a model is never used to do something harmful – any more so that the developer of a web browser, calculator, or word processor could. Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams. This matches my thoughts on why this is ultimately a b…

The definition of harm is buried low in the bill, here's the list:

(A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties. (B) At least five hundred million dollars ($500,000,000) of damage through cyberattacks on critical infrastructure via a single incident or multiple related incidents. (C) At least five hundred million dollars ($500,000,000) of damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human. (D) Other threats to public safety and security that are of comparable severity to the harms described in paragraphs (A) to (C), inclusive.

That means AI for drug discovery and materials science development, AI for managing electricity grids and broadband traffic, AI in the financial and health services sectors, etc. Then there's the military-industrial side, which this legislation might not even touch if only federal contracts are involved. Classified military AI development seems reckless, hasn't anyone seen War Games?

https://technologymagazine.com/top10/top-10-military-technol...

At least with open source, the capabilities are more immediately visible.

Re: SB-1047 will stifle open-source AI and decrease safety

#187

Post by ryandrake »

echelon wrote: post #71
sanxiyn wrote: post #49

Earlier quoted context omitted.

As I understand this law does not mandate you to ensure anything. It requires you to follow best practices (to be determined), report safety incidents, etc. You are not even liable for safety incidents, you just need to report them, although it may be embarrassing. Overall, it seems highly reasonable.

> requires you to follow best practices (to be determined) Trigger happy regulation for a field that hasn't even come into full swing. It's indicative of an over-active immune system; lawmakers with nothing better to do. Pass laws against improper use and go after the malicious users. Don't ban the technology, the research, or even the applications. (Of which there will be abundant good uses. Many of which we've yet…

> Trigger happy regulation for a field that hasn't even come into full swing. It's indicative of an over-active immune system; lawmakers with nothing better to do.

I guess they are damned if they do and damned if they don't.

We constantly complain about slow lawmaking, "Look at how out of touch Congress are! XYZ technology is moving so fast, and they're always 10-20 years behind!" Finally, someone is actually on the ball and up-to-date with a current technology, and now the other complainers complain that they're jumping the gun and regulating too soon. Lawmakers can't win.

Re: SB-1047 will stifle open-source AI and decrease safety

#188

Post by hendrycks »

ubj wrote: post #34

> The creator of a model can not ensure that a model is never used to do something harmful – any more so that the developer of a web browser, calculator, or word processor could. Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams. This matches my thoughts on why this is ultimately a b…

1. The new Frontier Model Division is just receiving information and issuing guidelines. It’s not a licensing regime and isn’t investigating developers.

2. Folks aren’t automatically liable if their highly capable model is used to do bad things, even catastrophic things. The question is whether they took reasonable measures to prevent that. This bill could have used strict liability, where developers would be liable for catastrophic harms regardless of fault, but that's not what the bill does.

3. Overall it seems pretty reasonable that if your model can cause catastrophic harms (which is not true of current models, but maybe true of future models), then you shouldn’t be releasing models in a way that can predictably allow folks to cause those catastrophic harms.

If people want a detailed write up of what the bill does, I recommend this thorough writeup by Zvi. In my opinion this is a pretty narrow proposal focused at the most severe risks (much more narrow than, e.g., the EU AI act). https://thezvi.substack.com/p/on-the-proposed-california-sb-...

Re: SB-1047 will stifle open-source AI and decrease safety

#189

Post by hendrycks »

A few things that I’m seeing folks in the comments misunderstanding about the bill (full disclosure: I’ve been one of a group of folks advising Senator Wiener on SB 1047)

1. The new Frontier Model Division is focused on receiving information and issuing guidelines. It’s not a licensing regime and isn’t investigating developers.

2. Folks aren’t automatically liable if their highly capable model is used to do bad things, even catastrophic things. The question is whether they took reasonable measures to prevent that. This bill could have used strict liability, where developers would be liable for catastrophic harms regardless of fault, but that's not what the bill does.

3. The bill requires developers to test their models and report whether they have hazardous capabilities (and the answer can obviously be yes or no). Even if the model does have hazardous capabilities, the developer can still deploy it if they take reasonable precautions, as outlined in the bill. For perjury, you would need to intentionally lie—good faith errors would not be covered. I get that models can have unforeseen capabilities, but this isn’t about that. If you are knowingly releasing something that could have demonstrably catastrophic consequences, it seems fair to have consequences for that. Some things which already require folks to certify under penalty of perjury: lobbying disclosures, companies’ financial disclosures, immigration compliance forms.

4. Overall it seems pretty reasonable that if your model can cause catastrophic harms (which is not true of current models, but maybe true of future models), then you shouldn’t be releasing models in a way that can predictably allow folks to cause those catastrophic harms.

If people want a writeup of what the bill does I recommend this one by the law firm DLA Piper (https://www.dlapiper.com/en/insights/publications/2024/02/ca...). In my opinion this is a pretty narrow proposal focused at the most severe risks (much more narrow than, e.g., the EU AI act).

Re: SB-1047 will stifle open-source AI and decrease safety

#190

Post by 65a »

drcode wrote: post #72
65a wrote: post #66

Earlier quoted context omitted.

Laws should be about the outcome, not about processes that may lead to an outcome. It is already illegal in California to produce your own nuclear weapon. Instead of outlawing books, because they allow research into building giant gundam robots, just outlaw giant gundam robots.

> Laws should be about the outcome, not about processes that may lead to an outcome. Some outcomes are pretty terrible, I think there are valid instances where we might also want to prevent precursor technology from being widely disseminated to prevent them.

There are certainly types of data that are already prohibited for export and dissemination. In this case, I would argue no new law is needed, the existing laws cover the export or dissemination of dual use technologies. If the LLM becomes dual-use/export-restricted/etc because it was trained on export-restricted/sensitive/etc data, it is already illegal to disseminate it. Enforce the existing law, rather than use taxpayer money to ban and police private LLM training because this might happen.
Post reply on HN