Earlier quoted context omitted.
Most of the diffs are probably innocuous. I suspect the most common diff would be the version line of Cargo.toml, both from CI that automatically updates that line, and people who forgot to update it before making a tag in git.
As someone with a crate that's in the 50MM plus range, this happens all the time. I really should automate this via a GH action.
I've compared nearly all Rust crates.io crates to contents of their Git repos
41–50 of 65 posts
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#42Earlier quoted context omitted.
First pass gpt?
Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#43Earlier quoted context omitted.
First pass gpt?
Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?
However, LLMs are incredibly naive, so they could be easily fooled by a malicious actor (probably as easy as adding a comment that this is definitely NOT a backdoor).
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#44How crazy would it be to have a package repository that also builds the artifacts it distributes? You’d need a high barrier to entry to save on costs and time sifting through garbage. Perhaps it’s this high barrier that would prevent such a repository from taking off though. Perhaps this is just a really dumb step on a path leading back to simple checksum validations… though with those, you’re only validating that wh…
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#45Earlier quoted context omitted.
Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?
when your goal is to improve security, the unreliability that comes with LLMs is not the answer.
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#46Earlier quoted context omitted.
I'm not sure what is meant by "downloading from Git", I assume you mean downloading from Github. And Github is far less secure than what crates.io does, because crates.io is immutable (once published, uploaders can't change anything without opening a support ticket which will get rejected if they don't have a good reason), whereas Github history is trivially rewriteable. This means that if you rely on "v1.2.3" of a l…
Go modules can be hosted in any Git repository. The Go toolchain also keeps hashes of the selected tag so if you've reviewed it once it will never change without you explicitly giving it the ok.
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#47How could you rank them for review priority? Use a combination of repo popularity multiplied by amount of significant differences? Where significant differences are determined by excluding non-code files?
However, publishing a list that basically says "these are the least trustworthy Rust users" would cause quite a stir, so I'm not doing that.
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#48This is why I like what Go does, where you're downloading from Git directly (optionally proxied through Google, yes)
I'm not sure what is meant by "downloading from Git", I assume you mean downloading from Github. And Github is far less secure than what crates.io does, because crates.io is immutable (once published, uploaders can't change anything without opening a support ticket which will get rejected if they don't have a good reason), whereas Github history is trivially rewriteable. This means that if you rely on "v1.2.3" of a l…
https://central.sonatype.org/publish/requirements/immutabili...
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#49Earlier quoted context omitted.
As someone with a crate that's in the 50MM plus range, this happens all the time. I really should automate this via a GH action.
Interested to see the crate, and maybe I can help?
Re: I've compared nearly all Rust crates.io crates to contents of their Git repos
#50Earlier quoted context omitted.
Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?
when your goal is to improve security, the unreliability that comes with LLMs is not the answer.
If you just want to see if there is incidence of valid differences, this seems fine. But I wouldn’t use it as a guarantee.