Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-weight AI is having its Kubernetes moment

tobi.knaup.me

Post reply on HN
Print view View original

281–290 of 346 posts

Re: Open-weight AI is having its Kubernetes moment

#281

Post by monocasa »

moffkalast wrote: post #185
amarant wrote: post #182

Earlier quoted context omitted.

But that's just the thing with open weights: you're not doing any business with company that made the model. They might publish the weights to a, say, European host, and then you download the model from Europe and and run it on your servers in America, and suddenly it's very hard to tell where the model was originally created.

Yep, add a few blank layers, fine tune it a tiny bit and the weight checksums nor parameter counts won't match with anything, while the model will be practically the exact same. Time and time again random startups have tried passing established open models as their own. "You made this? I made this." Of course a conspicuous architecture would still give it away.

Or just perform a form of distillation, where you don't actually change the hyperparameters, but maybe shift around the embeddings or something.

You could even have another model watch the distillation process to check for goofy backdoors (which is about the best you're going to be able to do since detection of backdoors is np hard IIRC).

Re: Open-weight AI is having its Kubernetes moment

#282

Post by monocasa »

andriy_koval wrote: post #206
wyrdcurt wrote: post #198

Earlier quoted context omitted.

Except that even the exact same model won't output the exact same results, that's a fundamental aspect of how LLMs work. They're probabilistic/stochastic, not deterministic.

Models are weights for matrix operations, they are determenistics.

The output is a probability distribution for all potential tokens. Then a "temperature" is applied to weight the sampling randomly (unless the temperature is zero in which case the stack can be deterministic and simply the highest probability token is selected).

They go through this rigamarole because a little bit of randomness gives better results from a Turing test kind of perspective.

Re: Open-weight AI is having its Kubernetes moment

#283

Post by andriy_koval »

monocasa wrote: post #282
andriy_koval wrote: post #206

Earlier quoted context omitted.

Models are weights for matrix operations, they are determenistics.

The output is a probability distribution for all potential tokens. Then a "temperature" is applied to weight the sampling randomly (unless the temperature is zero in which case the stack can be deterministic and simply the highest probability token is selected). They go through this rigamarole because a little bit of randomness gives better results from a Turing test kind of perspective.

thank you, I already know basics.

Re: Open-weight AI is having its Kubernetes moment

#284

Post by KludgeShySir »

kloop wrote: post #132
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

> “The source described leading AI labs or their allies approaching the administration every 3-5 months with an idea to ban open-source models.” That's going to hit first amendment grounds pretty quick, the same way that software in general did. The modern version of the decss flag will be a character that says "I think good weights are {...weights go here...}" They could, however, ban any payment to a chinese entity…

There are already plenty of [illegal numbers](https://en.wikipedia.org/wiki/Illegal_number).

We have numbers that you can't possess without proper license/authorization, and numbers that you can't yell at a crowded movie theater.

Re: Open-weight AI is having its Kubernetes moment

#285

Post by KludgeShySir »

nunez wrote: post #244
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

It'll be pretty easy. Some US gov entity will create a list of models from Hugging Face, decree "thou shalt not provide access to these models", wrap it around some scary legalese for the pirates who try and that will be that. Though the legalese might not even be necessary. The list alone will make sure that no American company runs these on their servers, including the hosting providers.

Yeah, all the corporations will then be forced to pay for proprietary models, and then Big Model will be satisfied.

Just like typically individuals can get away with using pirated software, but IP owners don't make much of a stink as long as they've got the sweet, sweet enterprise license fees rolling in.

Re: Open-weight AI is having its Kubernetes moment

#286

Post by culi »

arczyx wrote: post #276
georgeburdell wrote: post #271

Earlier quoted context omitted.

There is no way Deepseek is making money even on inference

Pretty sure they are making money since on OpenRouter, there are other providers for DeepSeek V4 flash that are charging even less than DeepSeek themselves (eg DeepInfra and Digital Ocean). https://openrouter.ai/compare/deepseek/deepseek-v4-flash/ten...

That doesn't mean they're making money. Other providers could just be doing it more efficiently. DeepSeek is trying to make its model optimized for Hauwei chips instead of NVIDIA so it has its own constraints

Re: Open-weight AI is having its Kubernetes moment

#288

Post by killingtime74 »

ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

It will go as well as the banning of music piracy and BitTorrent sites. They can't even take down those open scientific paper sites.

0% chance they can ban these models practically

Re: Open-weight AI is having its Kubernetes moment

#289

Post by Covenant0028 »

SilverElfin wrote: post #159

What’s interesting is no one is talking about political censorship in models and how DeepSeek, Kimi, and the rest have to abide by CCP rules. It’s a big opportunity for China to control information.

The primary customers of these models are enterprises, and the most common use cases are office work and coding. How often do the questions of Tiananmen Square or the Uighurs become relevant in those contexts?

Re: Open-weight AI is having its Kubernetes moment

#290

Post by LPisGood »

killingtime74 wrote: post #288
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

It will go as well as the banning of music piracy and BitTorrent sites. They can't even take down those open scientific paper sites. 0% chance they can ban these models practically

The amount of money and strategic (national security) interest involved in AI seems to be at least a few orders of magnitude above scientific journal publication fees and the music industry combined.
Post reply on HN