Live data from Hacker News

SB-1047 will stifle open-source AI and decrease safety

answer.ai

21–30 of 201 posts

Re: SB-1047 will stifle open-source AI and decrease safety

#21

I’m worried that regulations like these will create a lock in effect that benefits existing leading AI companies and makes it impossible for new entrants

That is called regulatory capture, which could definitely be happening here.

Politics are depressing :/

Has the tech industry always been so cutthroat, or is this a new trend? Maybe I just didn't hear about tech companies lobbying for power in the past?

Re: SB-1047 will stifle open-source AI and decrease safety

#22
post #10

The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.

Important point, but I don't really have a reference for that number. Do we know what GPT-4 or other models would be in comparison?

No, it’s not a good point. It could be 10^100000000 flops and it wouldn’t fucking matter. There’s no evidence that would do anything at all. “AGI” may not even be possible with those flops. You’re talking about what would require unprecedented control over computing to ease fears over a scary robot fanfic. None of the “safety” concerns are real. A GPT4 open source would do nothing but hurt Sam Altman’s bottom line - YOU ARE BEING DUPED.

Re: SB-1047 will stifle open-source AI and decrease safety

#23
(f) “Covered model” means an artificial intelligence model that meets either of the following criteria:

(1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations.

(2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models.

---

f.1: anyone who has taken a basic cpu arch class knows that int and float are significantly different computational effort. One could see an entity using this in court to greatly lower the threshold for qualification after the fact. i.e. lawyers play word/text games and one could say something to the effect that 1 float is 10 int ops, so the limit is 10^26 int or 10^25 float ops

f.2: future proofing against better algorithms based on today's benchmarks... to the point where effort no longer matters. They seem to be drawing the threshold at today's benchmarks, whether or not they are reflective of capability. I could see a small model be trained to do poorly on these benchmarks while excelling at the problems they are concerned with, like making nuclear weapons...

Re: SB-1047 will stifle open-source AI and decrease safety

#24
post #3

The politicians are gunning extremely hard for open source AI. It's crazy. "Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose…

> The politicians are gunning extremely hard for open source AI. It's crazy.

I hope someone has / can do some investigative journalism to check out their links with commercial / closed source AI; I can imagine the investors and those that benefit from companies like "Open"AI have close links with politicians. There's probably no direct links, they've become really good at obscuring those and plausible deniability.

Re: SB-1047 will stifle open-source AI and decrease safety

#25
post #20

This is a very, very bad article that reeks of LLM-generation. If you want to actually understand this bill, read Zvi's analysis: https://thezvi.substack.com/p/on-the-proposed-california-sb-...

> reeks of LLM-generation.

> Answer.AI is a new kind of AI R&D lab which creates practical end-user products based on foundational research breakthroughs.

It's very likely the company / author is dogfooding.

Re: SB-1047 will stifle open-source AI and decrease safety

#26
post #16

Which states would you speculate will be best long term for AI startups? I would’ve guessed California but it’s looking more important to pick a state less likely to get over their skis with regulation. Washington doesn’t seem yet to be doing this, and no state tax with lots of AI engineers/researchers in Seattle/redmond which is why I’m here. Texas probably won’t add regulations and has similar pros if you’re in Aus…

Miami has a population that is extremely ideologically opposed to laws like this.

Re: SB-1047 will stifle open-source AI and decrease safety

#27

mmmmmmm regulatory capture, wouldn't be america without it - nothing says freedom and open markets like sawing off the rungs from the bottom of the ladder after you've already climbed to the top

Open source AI is not at the bottom of the ladder; like closed source AI, some of the brightest, well-educated and arguably well-paid people in the world are the driving force behind it. OpenAI, arguably at the top of the ladder, was supposed to be open source as well.

Re: SB-1047 will stifle open-source AI and decrease safety

#28

The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.

Its not an absurd number, depending on how it is calculated.

I've seen that number thrown around, and by some calculations it would already apply to open source models like LLama.

No, we don't need to ban or regulate LLama or any existing open source models. If someone wants to be worried about GPT-6, fine. But there is no need to regulate the stuff thats already out there.

Re: SB-1047 will stifle open-source AI and decrease safety

#29
post #3

The politicians are gunning extremely hard for open source AI. It's crazy. "Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose…

> The politicians are gunning extremely hard for open source AI. It's crazy. I hope someone has / can do some investigative journalism to check out their links with commercial / closed source AI; I can imagine the investors and those that benefit from companies like "Open"AI have close links with politicians. There's probably no direct links, they've become really good at obscuring those and plausible deniability.

You'll hear about it in 10 years in an article about "why did open source Ai die? ... once blossoming along closed source AI, open source AI disappeared after XXYYZ AI ACT. Turns out, Senator X and Senator Y were both in the pocket of closed source AI (and ended up in cushy jobs in Microsoft and Xai, not a coincidence) ... "

Re: SB-1047 will stifle open-source AI and decrease safety

#30
post #16

Which states would you speculate will be best long term for AI startups? I would’ve guessed California but it’s looking more important to pick a state less likely to get over their skis with regulation. Washington doesn’t seem yet to be doing this, and no state tax with lots of AI engineers/researchers in Seattle/redmond which is why I’m here. Texas probably won’t add regulations and has similar pros if you’re in Aus…

Miami has a population that is extremely ideologically opposed to laws like this.

They don’t seem to have any reason that there would be AI experts there though. No big comp sci universities or existing research labs and tech companies that you can pull talent from.
Post reply on HN