Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The GMP library's repository is under attack by a single GitHub user

gmplib.org

Post reply on HN
Print view View original

81–90 of 91 posts

Re: The GMP library's repository is under attack by a single GitHub user

#81

Post by cxr »

burkaman wrote: post #21
CtrlAltDelete51 wrote: post #16

Earlier quoted context omitted.

That seems like a perfectly reasonable response. They even provided the project so GMP could reach out.

It's totally reasonable to say that it isn't malicious and they aren't going to stop it, but it's unreasonable and unhelpful to add "seems like your servers just suck". They also missed or failed to mention that there are 700 forks of the project running the same thing at the same time.

> it's unreasonable and unhelpful to add "seems like your servers just suck"

What is the basis for this quote? Ctrl+F-ing on what look like the relevant pages turns up nothing. Maybe I missed it, but Google seems only to know about 1 instance of phrase from the last week (and it's the one in your comment).

Re: The GMP library's repository is under attack by a single GitHub user

#82

Post by eddythompson80 »

CtrlAltDelete51 wrote: post #56
eddythompson80 wrote: post #36

Earlier quoted context omitted.

Especially if you’re running powerful server-class hardware and great connectivity to the Internet.

^ So much this. From the assumption of malicious activity (which is a far leap given such a predictable pattern of calls), to bragging about “server-class hardware” and “great connectivity”, this doesn’t seem like a super solid setup. In before “it’s just one guy”: there are a thousand ways to solve this that are not expensive or complicated. Should the offending party have been a better consumer? Yes. Is it fair to…

Hyperbole never ceases to amaze me. On one hand, I know how wasteful "CI" systems are. We offer a docker image from a custom docker registry. Since we started offering it 3 years ago, we've had 1 billion pulls. 990 million pulls were from CI systems. But to claim that the operator of those CI systems is "attacking" us would be pretty bizarre. Though bizarre seems to govern online discourse when it comes to most things

Re: The GMP library's repository is under attack by a single GitHub user

#83

Post by marmoute1 »

jeroenhd wrote: post #37
542458 wrote: post #28

Earlier quoted context omitted.

I’m a bit confused as to why 700 automated clone requests per day would be an unreasonable amount of traffic for a project like GMP. That doesn’t sound like much traffic to me, especially since they reportedly have a 24 core CPU with 256GB of RAM. Is cloning a mercurial repo highly server intensive?

Those 700 clones would all hit at exactly the same time. That's quite a load for a single server, especially since Mercurial can't be cached that easily.

[dead]

Re: The GMP library's repository is under attack by a single GitHub user

#84

Post by wpollock »

eddythompson80 wrote: post #48
orf wrote: post #41

Earlier quoted context omitted.

Yes?

Why would you expect a cron task to work differently because a repo is a fork or not? Maybe Actions/Workflows shouldn’t be enabled by default on forks, which used to be the case but that has its problems too. You can disable actions on your forks.

The problem of concurrent cron jobs isn't new. Solutions existed for decades. Fedora cron used eth0's first IP address and a urandom number to stagger cron jobs starting times. Google addressed this issue by having a single enterprise wide cron service. Why does Github have such a simplistic cron service in this day and age?

Re: The GMP library's repository is under attack by a single GitHub user

#85

Post by eddythompson80 »

wpollock wrote: post #84
eddythompson80 wrote: post #48

Earlier quoted context omitted.

Why would you expect a cron task to work differently because a repo is a fork or not? Maybe Actions/Workflows shouldn’t be enabled by default on forks, which used to be the case but that has its problems too. You can disable actions on your forks.

The problem of concurrent cron jobs isn't new. Solutions existed for decades. Fedora cron used eth0's first IP address and a urandom number to stagger cron jobs starting times. Google addressed this issue by having a single enterprise wide cron service. Why does Github have such a simplistic cron service in this day and age?

> Fedora cron used eth0's first IP address and a urandom number to stagger cron jobs starting times.

What?

> Google addressed this issue by having a single enterprise wide cron service.

What???

What does any of that even mean my guy? Neither of these were a cron CI shell script services. What does any of that even mean? Did you just read the word “cron” and get excited?

Re: The GMP library's repository is under attack by a single GitHub user

#86

Post by Mathiasdm »

jeroenhd wrote: post #37
542458 wrote: post #28

Earlier quoted context omitted.

I’m a bit confused as to why 700 automated clone requests per day would be an unreasonable amount of traffic for a project like GMP. That doesn’t sound like much traffic to me, especially since they reportedly have a 24 core CPU with 256GB of RAM. Is cloning a mercurial repo highly server intensive?

Those 700 clones would all hit at exactly the same time. That's quite a load for a single server, especially since Mercurial can't be cached that easily.

Mercurial clones can be cached quite easily with 'clonebundles', a Mercurial feature that allows redirecting a clone to instead download a single 'bundle' (which could come from a different server or set of servers).

See https://wiki.mercurial-scm.org/ClonebundlesExtension

Re: The GMP library's repository is under attack by a single GitHub user

#87

Post by rwmj »

cxr wrote: post #79
rwmj wrote: post #8

Earlier quoted context omitted.

The title is a bit confusing. It seems the GMP project has a self-hosted Mercurial repo ( https://gmplib.org/devel/repo-usage ) which is being hit by a user at Microsoft that they've somehow tied back to a single github account. I'm sure it's going to be CI or mirroring or some other automated process.

Why is this a response to the person you sent this reply to?

Github can't easily cache a remotely hosted repo that some random CI test (essentially a script) accesses, at least not without some very hairy proxying.

Re: The GMP library's repository is under attack by a single GitHub user

#88

Post by cxr »

rwmj wrote: post #87
cxr wrote: post #79

Earlier quoted context omitted.

Why is this a response to the person you sent this reply to?

Github can't easily cache a remotely hosted repo that some random CI test (essentially a script) accesses, at least not without some very hairy proxying.

Is GitHub's CI infrastructure (and cloud compute, generally) not hairy? (A basic attempt at straightforward proxying seems like it would be the _least_ hairy aspect of the stack, besides.)

Re: The GMP library's repository is under attack by a single GitHub user

#89

Post by wpollock »

eddythompson80 wrote: post #85
wpollock wrote: post #84

Earlier quoted context omitted.

The problem of concurrent cron jobs isn't new. Solutions existed for decades. Fedora cron used eth0's first IP address and a urandom number to stagger cron jobs starting times. Google addressed this issue by having a single enterprise wide cron service. Why does Github have such a simplistic cron service in this day and age?

> Fedora cron used eth0's first IP address and a urandom number to stagger cron jobs starting times. What? > Google addressed this issue by having a single enterprise wide cron service. What??? What does any of that even mean my guy? Neither of these were a cron CI shell script services. What does any of that even mean? Did you just read the word “cron” and get excited?

I understood the issue was caused by lots of cron jobs firing at the same moment. If Github had a better cron, the 700 or whatever network downloads would not have occurred at the same moment and there would have been no problem. Sorry if I misunderstood something.

Re: The GMP library's repository is under attack by a single GitHub user

#90

Post by capitainenemo »

capitainenemo wrote: post #49
coder543 wrote: post #25

Earlier quoted context omitted.

FWIW, it’s not a git repo, it’s a mercurial repo.

And even if it was a git repo, seems this would still be a problem if everything wasn't on github. I guess the solution is to put all code on github...

BTW, for anyone reading this later, I asked some mercurial folks and putting an http cache in front of clones does indeed work just fine, although using the repo bundles is more performant (but might not be something you could ask folks to do). Given this repo seems fairly low activity (last commit 7 months ago) I doubt there'd be much of a problem with caching reads.

Although given how they are using this repo, they'd probably be better off with a shallow clone, or a subversion proxy, or a snapshot tarball...likely just need a set of files..

Post reply on HN