Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The GMP library's repository is under attack by a single GitHub user

gmplib.org

Post reply on HN
Print view View original

51–60 of 91 posts

Re: The GMP library's repository is under attack by a single GitHub user

#51

Post by tedunangst »

jeroenhd wrote: post #33
judofyr wrote: post #15

More context: https://github.com/BtbN/FFmpeg-Builds/issues/278 The FFmpeg-Builds repo has a GitHub Actions Workflow which clones the Mercurial repo. However, this runs as a (daily?) cronjob. In addition, this repo has 700 forks and now all of them are running the same workflow. This is out of control for the original author of the repo as there’s no way to change those 700 forks… EDIT: Also relevant is that there’s n…

I'm confused by this statement: > GMP is at this point the only dependency that does not offer a sane way to clone its repository. The latest release is from years ago and https://gmplib.org/download/gmp/gmp-6.2.1.tar.lz should contain every file necessary. Why would you need access to the commit history in a build script? In fact, why would you need to download a fresh copy of a dependency that almost never gets upd…

The idea that you build software from a fixed archive is passé. Everything must be a git clone. Or if they don't use git, you muddle along with hg.

Re: The GMP library's repository is under attack by a single GitHub user

#52

Post by judofyr »

jeroenhd wrote: post #33
judofyr wrote: post #15

More context: https://github.com/BtbN/FFmpeg-Builds/issues/278 The FFmpeg-Builds repo has a GitHub Actions Workflow which clones the Mercurial repo. However, this runs as a (daily?) cronjob. In addition, this repo has 700 forks and now all of them are running the same workflow. This is out of control for the original author of the repo as there’s no way to change those 700 forks… EDIT: Also relevant is that there’s n…

I'm confused by this statement: > GMP is at this point the only dependency that does not offer a sane way to clone its repository. The latest release is from years ago and https://gmplib.org/download/gmp/gmp-6.2.1.tar.lz should contain every file necessary. Why would you need access to the commit history in a build script? In fact, why would you need to download a fresh copy of a dependency that almost never gets upd…

6.2.1 was released on 2020-11-14, but there's been additional bug fixes, features and speedups since then: https://gmplib.org/repo/gmp/. Not quite sure why they are not cutting a new release when there's been no development in the last 7 months.

Re: The GMP library's repository is under attack by a single GitHub user

#53

Post by miyuru »

aseipp wrote: post #18

The repository in question, based on the email from Mike Blacker, sent to gmp-devel[1]: https://github.com/BtbN/FFmpeg-Builds There's a bunch of build variants here, for various platforms, in different ways. Classic "you have to create the cartesian product of all variables" approach. Here is the single build matrix for a single "Build FFMPeg" job: https://github.com/BtbN/FFmpeg-Builds/actions/runs/530351117... You c…

Some more information here. The GitHub user has already removed repo from the code and uses a tarball now.

https://github.com/BtbN/FFmpeg-Builds/commit/d75466340a9a598...

Re: The GMP library's repository is under attack by a single GitHub user

#54

Post by zX41ZdbW »

> I've already switched GMP back to use an outdated release tarball because of how annoyingly flakey their mercurial server is.

We had a similar problem. Step one - don't allow submodules from outside of github.com: https://github.com/ClickHouse/ClickHouse/blob/master/utils/c...

If there is a dependency from external git repository, it can be forked to github.com.

But we still have a problem with: - Docker Hub; - Cargo. Sometimes they are offline. Also we should not use them to avoid supply-chain attacks.

The solution probably should be: - copy all docker images as tarballs; - or use ECR. Still not sure what to do with Cargo: https://github.com/ClickHouse/ClickHouse/issues/48575

Re: The GMP library's repository is under attack by a single GitHub user

#55

Post by jasonjayr »

542458 wrote: post #28
burkaman wrote: post #21

Earlier quoted context omitted.

It's totally reasonable to say that it isn't malicious and they aren't going to stop it, but it's unreasonable and unhelpful to add "seems like your servers just suck". They also missed or failed to mention that there are 700 forks of the project running the same thing at the same time.

I’m a bit confused as to why 700 automated clone requests per day would be an unreasonable amount of traffic for a project like GMP. That doesn’t sound like much traffic to me, especially since they reportedly have a 24 core CPU with 256GB of RAM. Is cloning a mercurial repo highly server intensive?

It's 700 automated clone requests at the same moment due to inheriting a cronjob.

Re: The GMP library's repository is under attack by a single GitHub user

#56

Post by CtrlAltDelete51 »

eddythompson80 wrote: post #36
matheusmoreira wrote: post #30

Earlier quoted context omitted.

I'm confused as well. Seems like a negligible amount of traffic at first glance. The administrator mentioned compression, could it really have such a big impact?

Especially if you’re running powerful server-class hardware and great connectivity to the Internet.

^ So much this. From the assumption of malicious activity (which is a far leap given such a predictable pattern of calls), to bragging about “server-class hardware” and “great connectivity”, this doesn’t seem like a super solid setup.

In before “it’s just one guy”: there are a thousand ways to solve this that are not expensive or complicated.

Should the offending party have been a better consumer? Yes.

Is it fair to entirely blame the user versus implementing additional caches and safeguards to make it an annoyance and not the end of the world emergency GMP is making it out to be? Definitely not.

Re: The GMP library's repository is under attack by a single GitHub user

#57

Post by orf »

eddythompson80 wrote: post #48
orf wrote: post #41

Earlier quoted context omitted.

Yes?

Why would you expect a cron task to work differently because a repo is a fork or not? Maybe Actions/Workflows shouldn’t be enabled by default on forks, which used to be the case but that has its problems too. You can disable actions on your forks.

Why would I expect pressing the fork button would suddenly enable redundant and, in aggregare, expensive cronjobs?

Most people fork to make a pull request and then never delete the fork after. Why are any scheduled actions required for that?

Re: The GMP library's repository is under attack by a single GitHub user

#58

Post by miyuru »

chongli wrote: post #42
jeroenhd wrote: post #37

Earlier quoted context omitted.

Those 700 clones would all hit at exactly the same time. That's quite a load for a single server, especially since Mercurial can't be cached that easily.

So if they're all hitting at exactly the same time, why not set up a firewall rule to block traffic from GitHub at exactly that time of day?

Setting up a rate limiter is the correct way forward, and using IP block as the nuclear option.

Re: The GMP library's repository is under attack by a single GitHub user

#59

Post by aseipp »

folkrav wrote: post #29
aseipp wrote: post #18

The repository in question, based on the email from Mike Blacker, sent to gmp-devel[1]: https://github.com/BtbN/FFmpeg-Builds There's a bunch of build variants here, for various platforms, in different ways. Classic "you have to create the cartesian product of all variables" approach. Here is the single build matrix for a single "Build FFMPeg" job: https://github.com/BtbN/FFmpeg-Builds/actions/runs/530351117... You c…

> It already is cached by the docker image. The problem why it's SO MANY requests is because hundreds of people have forked this repository, and now at the same time (due to the workflow cron) start an image build. 99% of the days, it'll just use the cached image and not cause any network requests whatsoever. > If it builds new images, it'll be at max 10 requests to fetch the latest snapshot in parallel from my end (…

Interesting, I didn't look far back enough. And I didn't realize it cloned from Mercurial first instead of using the tarball either; that'll hurt a lot more, too!

Re: The GMP library's repository is under attack by a single GitHub user

#60

Post by [deleted] »

samwillis wrote: post #7

So another classic case of someone's constant integration system DoS'ing an open source project. We've all become so blinded by big tech and how they operate we fail to see how our actions can affect smaller projects we depend on CI providers really should do more to prevent and mitigate these when they happen. They should have outbound firewalls, and the ability to request a rate limit on IPs. Having to resort to a…

[deleted]
Post reply on HN