Live data from Hacker News

Open source AI: Red Hat's point-of-view

redhat.com

61–70 of 71 posts

Re: Open source AI: Red Hat's point-of-view

#61
post #45

To me, the ML situation looks roughly like this. (1) Model weights are something like a bytecode blob. You can run it in a conformant interpreter, and be able to do inference. (2) Things like llama.cpp are the "bytecode interpreter" part, something that can load the weights and run inference. (3) The training setup is like a custom "compiler" which turns training data to the "bytecode" of the model weights. (4) The a…

I think these endless debates about whether open-weights models qualify for a particular piece of terminology are... tiring. That said, I think the debates would benefit from discussing model training and model inference as two separate systems, because that's what they are. It's possible for model training to be closed-source while model inference is open-source, and vice versa.

Consider recent Mistral-Small release. The model training is almost totally closed-source. You can't replicate it. However, the model inference is fully open source: the code and weights are Apache licensed. Not only that, but Mistral released both the base model and the instruction-tuned model, so you have a good foundation to work from (the base model) should you prefer to do your own instruction tuning. In fact, Mistral has also open-sourced code to aid in the fine-tuning process as well. So you really have everything you need* to use and customize this inference system. And for most practical purposes, even if you had the original training data, it would be of no use to you.

It's also worth considering the inverse scenario. Suppose Meta were to release a big blob of pre-training data and scripts for Llama 405B, but no weights. This clearly qualifies as open source, but it is basically useless unless you have many millions of dollars to do something with it. It would do very little to democratize access to AI.

* Asterisk: There is one situation where having access to the original training data would be really, really useful -- model distillation. Nobody can match Meta's ability to distill Llama 405B into an 8B size, because that process works best when you can do it on identically distributed data.

Re: Open source AI: Red Hat's point-of-view

#62
post #45

To me, the ML situation looks roughly like this. (1) Model weights are something like a bytecode blob. You can run it in a conformant interpreter, and be able to do inference. (2) Things like llama.cpp are the "bytecode interpreter" part, something that can load the weights and run inference. (3) The training setup is like a custom "compiler" which turns training data to the "bytecode" of the model weights. (4) The a…

> Model weights are something like a bytecode blob

Can you update a bytecode blob as easily as finetuning and prompting models? It only takes a few input-output pairs and a few dollars worth of compute. They are more like an operating system and fine-tuning/prompting is like scripting on top. Similarly with Linux, you can download a LLM and run it locally.

Re: Open source AI: Red Hat's point-of-view

#63
post #9

Earlier quoted context omitted.

i mean... it's IBM so what did we really expect?

I was thinking of trying Fedora (currently using Debian) and this comment made me look up who owns red-hat. ibm now owns red-hat, and apparently vanguard owns a huge chunk of ibm. I wonder how much influence any of the sponsors have over what goes into the os and what direction it takes.

> vanguard owns a huge chunk of ibm

Vanguard and other large institutions own a huge chunk of everything because most investors don't buy stocks directly, they buy them through mutual funds, ETFs, etc...

Re: Open source AI: Red Hat's point-of-view

#65
post #45

To me, the ML situation looks roughly like this. (1) Model weights are something like a bytecode blob. You can run it in a conformant interpreter, and be able to do inference. (2) Things like llama.cpp are the "bytecode interpreter" part, something that can load the weights and run inference. (3) The training setup is like a custom "compiler" which turns training data to the "bytecode" of the model weights. (4) The a…

I think these endless debates about whether open-weights models qualify for a particular piece of terminology are... tiring. That said, I think the debates would benefit from discussing model training and model inference as two separate systems , because that's what they are. It's possible for model training to be closed-source while model inference is open-source, and vice versa. Consider recent Mistral-Small releas…

For me, the attacks on ML that are possibly by poisoning the training data preclude considering models without freely distributable and modifiable training as open-source or libre models.

Re: Open source AI: Red Hat's point-of-view

#66
post #57
post #56

I largely agree with these points, however it is an awkward position coming from Red Hat which is the best funded Linux distribution there is, and -still- not part of the reproducible builds project or investing in full source bootstrapping which means no one can exactly reproduce their published artifacts from source or prove they were not tampered with. (Same with Fedora) Glass houses.

> (Same with Fedora) ?? https://docs.fedoraproject.org/en-US/reproducible-builds/ https://pagure.io/fedora-reproducible-builds/project/issues https://fedoraproject.org/wiki/Releases/41/ChangeSet#Reprodu...

From that first link "In the Fedora ecosystem, we cannot achieve reproducibility by the reproducible-builds.org definition"

Good to see they are slowly closing some blockers every year or so, but fundamentally today they do builds and signing centrally. There is no way to readily get the same hash of a central fedora supplied rpm locally.

Supply chain integrity is simply not a priority. They just trust the central build farm, or the compilers it uses, or everyone with access to it will never be compromised.

Re: Open source AI: Red Hat's point-of-view

#67
post #66
post #57

Earlier quoted context omitted.

> (Same with Fedora) ?? https://docs.fedoraproject.org/en-US/reproducible-builds/ https://pagure.io/fedora-reproducible-builds/project/issues https://fedoraproject.org/wiki/Releases/41/ChangeSet#Reprodu...

From that first link "In the Fedora ecosystem, we cannot achieve reproducibility by the reproducible-builds.org definition" Good to see they are slowly closing some blockers every year or so, but fundamentally today they do builds and signing centrally. There is no way to readily get the same hash of a central fedora supplied rpm locally. Supply chain integrity is simply not a priority. They just trust the central bu…

This is a touch dramatic. The hash of the payload and the hash of the RPM header are still reproducible and can be verified. It's just that the existence of internal signatures makes it impossible to do a simple checksum of the file.

Re: Open source AI: Red Hat's point-of-view

#68
post #67
post #66

Earlier quoted context omitted.

From that first link "In the Fedora ecosystem, we cannot achieve reproducibility by the reproducible-builds.org definition" Good to see they are slowly closing some blockers every year or so, but fundamentally today they do builds and signing centrally. There is no way to readily get the same hash of a central fedora supplied rpm locally. Supply chain integrity is simply not a priority. They just trust the central bu…

This is a touch dramatic. The hash of the payload and the hash of the RPM header are still reproducible and can be verified. It's just that the existence of internal signatures makes it impossible to do a simple checksum of the file.

And thus RPM was not designed with easy user reproduction and signing by multiple independent parties for high accountability in mind. Most other package managers do not have this problem. This is a flaw that should be corrected.

Also, it takes a ton of work and testing and bug fixes and patches to get software reproducible. Assume most packages are not reproducible until proven otherwise. Arch, debian, nix, guix, all do that work and publish the proof, for several years, with far less resources than redhat or fedora. Stagex even has 100% (shameless plug)

Easy user hash for hash reproducibility with published reproduction testing proofs is the standard baseline for years now, and even that is nowhere near good enough.

Multiple independently signed reproduction proofs with full source bootstrapping is IMO a bare minimum for any distro that expects other people to be able to trust it for more than hobby use cases.

Supply chain attacks are becoming very common, and no one should have to trust a single engineer somewhere with a god signing key for a major distro.

Also just to spot check a popular package in Fedora, rust, I just confirmed it still downloads a non-reproducible binary rust compiler to build its own rust package, so it is certainly not reproducible from source even putting aside the rpm signing format problems. Fedora blindly trusts whoever builds the binaries on the rust team. I can only assume RHEL does the same.

https://src.fedoraproject.org/rpms/rust/blob/8e04e725bbf4eb9...

Re: Open source AI: Red Hat's point-of-view

#69
post #68
post #67

Earlier quoted context omitted.

This is a touch dramatic. The hash of the payload and the hash of the RPM header are still reproducible and can be verified. It's just that the existence of internal signatures makes it impossible to do a simple checksum of the file.

And thus RPM was not designed with easy user reproduction and signing by multiple independent parties for high accountability in mind. Most other package managers do not have this problem. This is a flaw that should be corrected. Also, it takes a ton of work and testing and bug fixes and patches to get software reproducible. Assume most packages are not reproducible until proven otherwise. Arch, debian, nix, guix, al…

It's a problem that can be easily fixed with tooling that's smart enough to just look inside the file. Detached signatures aren't necessarily better, just different.

Re: Open source AI: Red Hat's point-of-view

#70
post #36

Earlier quoted context omitted.

Also they simultaneously hold the ownership rights as well as equivalent owenrship liabilities, so they own shit squat in net terms (excpet maybe their management fees).

It’s even weirder than that! Vanguard has an odd corporate structure where it’s owned by the funds that it manages, so it’s effectively a co-op owned by its customers.

Interesting, that's inline with Bogle's mission for low (0%) mgmt fees.

I think it probably doesn't apply to other majority holders like BlackRock though.

Post reply on HN