Live data from Hacker News

I've compared nearly all Rust crates.io crates to contents of their Git repos

mastodon.social

41–50 of 65 posts

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#41
post #18

Earlier quoted context omitted.

Most of the diffs are probably innocuous. I suspect the most common diff would be the version line of Cargo.toml, both from CI that automatically updates that line, and people who forgot to update it before making a tag in git.

As someone with a crate that's in the 50MM plus range, this happens all the time. I really should automate this via a GH action.

Interested to see the crate, and maybe I can help?

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#42

Earlier quoted context omitted.

First pass gpt?

Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?

when your goal is to improve security, the unreliability that comes with LLMs is not the answer.

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#43

Earlier quoted context omitted.

First pass gpt?

Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?

It could work for classifying honest/innocent differences.

However, LLMs are incredibly naive, so they could be easily fooled by a malicious actor (probably as easy as adding a comment that this is definitely NOT a backdoor).

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#44

How crazy would it be to have a package repository that also builds the artifacts it distributes? You’d need a high barrier to entry to save on costs and time sifting through garbage. Perhaps it’s this high barrier that would prevent such a repository from taking off though. Perhaps this is just a really dumb step on a path leading back to simple checksum validations… though with those, you’re only validating that wh…

That's what nixpkgs does for Nix/NixOS. The package set is continuously built by a CI system and made publicly available: https://github.com/NixOS/nixpkgs#continuous-integration-and-...

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#45
post #42

Earlier quoted context omitted.

Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?

when your goal is to improve security, the unreliability that comes with LLMs is not the answer.

I don't think this is a relevant take. Your goal is to implement a system to automatically scan countless packages and run a heuristic to determine if a package is suspicious or not. You're complaining about false positives/false negatives while ignoring that packages that not checking packages at all is not an improvement.

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#46
post #28
post #22

Earlier quoted context omitted.

I'm not sure what is meant by "downloading from Git", I assume you mean downloading from Github. And Github is far less secure than what crates.io does, because crates.io is immutable (once published, uploaders can't change anything without opening a support ticket which will get rejected if they don't have a good reason), whereas Github history is trivially rewriteable. This means that if you rely on "v1.2.3" of a l…

Go modules can be hosted in any Git repository. The Go toolchain also keeps hashes of the selected tag so if you've reviewed it once it will never change without you explicitly giving it the ok.

Also any svn/hg repository afaik

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#47
post #6

How could you rank them for review priority? Use a combination of repo popularity multiplied by amount of significant differences? Where significant differences are determined by excluding non-code files?

I'd use popularity (how many people are using the crate indirectly) divided by trust level in the publisher of the crate.

However, publishing a list that basically says "these are the least trustworthy Rust users" would cause quite a stir, so I'm not doing that.

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#48
post #22

This is why I like what Go does, where you're downloading from Git directly (optionally proxied through Google, yes)

I'm not sure what is meant by "downloading from Git", I assume you mean downloading from Github. And Github is far less secure than what crates.io does, because crates.io is immutable (once published, uploaders can't change anything without opening a support ticket which will get rejected if they don't have a good reason), whereas Github history is trivially rewriteable. This means that if you rely on "v1.2.3" of a l…

It’s interesting you even have to point this out. Maven solved this and other problems literally decades ago but the repository packaging wheel keeps getting reinvented. For example, here’s the page on Maven Central’s immutability policy:

https://central.sonatype.org/publish/requirements/immutabili...

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#49

Earlier quoted context omitted.

As someone with a crate that's in the 50MM plus range, this happens all the time. I really should automate this via a GH action.

Interested to see the crate, and maybe I can help?

https://crates.io/crates/ctor

Patches welcome!

Email in profile as well.

Re: I've compared nearly all Rust crates.io crates to contents of their Git repos

#50
post #42

Earlier quoted context omitted.

Heavily down voted, which is fair because I didn't really explain what I meant, which was: Would using LLM's to parse the generated diffs, as a first pass, be useful/efficient for spotting and interpreting discrepancies?

when your goal is to improve security, the unreliability that comes with LLMs is not the answer.

Not necessarily, it reduces false positives. It just doesn’t do anything for false negatives (arguably makes the problem worse).

If you just want to see if there is incidence of valid differences, this seems fine. But I wouldn’t use it as a guarantee.

Post reply on HN