Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-weight AI is having its Kubernetes moment

tobi.knaup.me

Post reply on HN
Print view View original

291–300 of 346 posts

Re: Open-weight AI is having its Kubernetes moment

#291

Post by thayne »

ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

> So, any solution to this “problem” must include ALL open-weight models.

I think that is what the "leading AI labs" actually want. They don't care where the open weight models come from, they just don't want to compete with them. The fact that a lot of the open weight models come from china is just a convenient circumstance they can leverage to get the government to give them what they really want.

Re: Open-weight AI is having its Kubernetes moment

#292

Post by mindwok »

RussianCow wrote: post #64
YetAnotherNick wrote: post #27

Earlier quoted context omitted.

2026 is the year for vercel and Render.

I haven't used them but aren't they basically modernized Heroku? What's different?

Pretty close, they're very close to Heroku in spirit except the new wave of hosted backends (Vercel, and to a lesser extent Supabase and Cloudflare Workers) are much more tightly integrated with the app layer. You can almost think of them as libraries that run in the cloud which you can just use inside your app.

Re: Open-weight AI is having its Kubernetes moment

#293

Post by tangotaylor »

curious_cat_163 wrote: post #7

> The government should use procurement to create demand for portable, interoperable systems rather than permanent dependence on one API vendor. Now, here is an idea that I have not heard before... and I think there is some merit to this. This is also the sort of thing that a state (looking at you CA, CO, IL, NY) could do, instead of just the federal government.

I dunno about the other states I feel like we need a better coalition lobbying California lawmakers about AI innovation because I feel like they're doing a lackluster job.

For example, Buffy Wicks introduced AB 2023 (passed the Assembly) which will effectively cause chatbot operators to ban minors because of the huge liability risk that the law introduces. This is a great way to kneecap innovation by excluding curious children and teens who are often the most innovative. California already has laws on the books to address chatbots encouraging self harm (BPC §§ 22601-22606) so I don't know why on earth they're doing this.

Then there's AB 2169 introduced by Lowenthal, which would have mandated interoperability between chatbot platforms to help people migrate to competing ones easier. I thought this was awesome but it didn't even get a vote in the Assembly.

Maybe the newly-created Little Tech Association can help here.

Re: Open-weight AI is having its Kubernetes moment

#294

Post by cgio »

satvikpendem wrote: post #171
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

It's simple, the US government will put any Chinese open model companies on the entity list which blocks any company which does business with the US from also doing business with the Chinese companies. This creates a chilling effect where even if it may be harder to tell, no US company will be able to provide or use any overt Chinese open model and won't even risk trying to go around as the punishments for trying to…

Probably a mechanism like what you describe. This is one more move that will push things towards a two speed world economy. One US sanctioned and one not. The question will be eventually which speed will end up faster. The challenge is when to do that with a meaningful chance of success, and while I don’t like the answer, the objective answer seems to be as soon as possible. Most probably the game is already lost though, so that soon may be too late, and the strategy wrong for US as a power while still valid for the AI houses that will squeeze what they can on the downward trajectory. I don’t expect people to actually calculate though.

Re: Open-weight AI is having its Kubernetes moment

#295

Post by ecocentrik »

thayne wrote: post #291
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

> So, any solution to this “problem” must include ALL open-weight models. I think that is what the "leading AI labs" actually want. They don't care where the open weight models come from, they just don't want to compete with them. The fact that a lot of the open weight models come from china is just a convenient circumstance they can leverage to get the government to give them what they really want.

It would be so simple if businesses could choose their competition or simplify disallow modes of competition they found inconvenient. I can't imagine any amount of regulation would make that possible in today's competitive landscape with the existing laws and the enormous volume of prior art.

Re: Open-weight AI is having its Kubernetes moment

#296

Post by killingtime74 »

LPisGood wrote: post #290
killingtime74 wrote: post #288

Earlier quoted context omitted.

It will go as well as the banning of music piracy and BitTorrent sites. They can't even take down those open scientific paper sites. 0% chance they can ban these models practically

The amount of money and strategic (national security) interest involved in AI seems to be at least a few orders of magnitude above scientific journal publication fees and the music industry combined.

How will they physically enforce the ban. They would have to go into everyone's house to search them. Using an llm also doesn't give off any signatures like BitTorrent or even visiting a papers site might. Can you propose even one method of enforcing it (I cannot)

Re: Open-weight AI is having its Kubernetes moment

#297

Post by LPisGood »

killingtime74 wrote: post #296
LPisGood wrote: post #290

Earlier quoted context omitted.

The amount of money and strategic (national security) interest involved in AI seems to be at least a few orders of magnitude above scientific journal publication fees and the music industry combined.

How will they physically enforce the ban. They would have to go into everyone's house to search them. Using an llm also doesn't give off any signatures like BitTorrent or even visiting a papers site might. Can you propose even one method of enforcing it (I cannot)

Practically, they could enforce this in similar ways they enforce CSAM restrictions. Dialing back the severity, restrictions on any cloud hosting provider from serving these models is probably going to cut usage down tremendously.

Re: Open-weight AI is having its Kubernetes moment

#299

Post by nareshshah139 »

andriy_koval wrote: post #260
antonvs wrote: post #256

Earlier quoted context omitted.

Achieving determinism with LLMs and other neural network models is actually a hard problem that people spend a lot of time on, when they need that. It doesn’t happen by accident. Issues include accumulated floating point errors happening in different orders due to distributed and parallel computation, CUDA kernels that deliberately sacrifice determinism for speed, and several other such issues.

It happened that I am working on OSS LLM -> finetuning -> benchmark with 100k tests pipeline, and unless I do some data augmentation, result is 100% deterministic. I think you likely right, that some parts of stack could induce some marginal float point error, but converged model can mitigate it, and on some principal set of knowledge can give deterministic result with high probability. Which leads me to believe if y…

The major cause for non-determism is batching.
Post reply on HN