Earlier quoted context omitted.
I’m a bit confused as to why 700 automated clone requests per day would be an unreasonable amount of traffic for a project like GMP. That doesn’t sound like much traffic to me, especially since they reportedly have a 24 core CPU with 256GB of RAM. Is cloning a mercurial repo highly server intensive?
It's 700 automated clone requests at the same moment due to inheriting a cronjob.
The GMP library's repository is under attack by a single GitHub user
71–80 of 91 posts
Re: The GMP library's repository is under attack by a single GitHub user
#72Earlier quoted context omitted.
Initially, I wanted to reply to you and say it's unfair to judge the person as incompetent; I still think it's a bit unfair because they develop a specialized library and their choice of hosting shouldn't need to be modernized at every step. However, reading through their response thread [0] I do think they aren't appearing particularly mature . It feels like they are refusing to characterize what is actually going o…
What part isn't mature? The bit where they set limitations for use of their hardware?
- The fair use update is reactive, and won't help anything
- Not separating microsoft/github from users of the platform (I agree the line is an worthwhile discussion)
- Not following up on the reasonable technical solutions that are being proposed
- Not seeming to ack that while the firewall ban might be the best they can do now, it'll block interested users of their own (GMP) library.
That said, the part in the GitHub reply dismissing GMP's hardware is also immature.
Re: The GMP library's repository is under attack by a single GitHub user
#73Earlier quoted context omitted.
What part isn't mature? The bit where they set limitations for use of their hardware?
- Threatening lawyers - The fair use update is reactive, and won't help anything - Not separating microsoft/github from users of the platform (I agree the line is an worthwhile discussion) - Not following up on the reasonable technical solutions that are being proposed - Not seeming to ack that while the firewall ban might be the best they can do now, it'll block interested users of their own (GMP) library. That said…
[1] https://gmplib.org/list-archives/gmp-devel/2023-June/006169....
Re: The GMP library's repository is under attack by a single GitHub user
#74Earlier quoted context omitted.
They've clearly worried enough about it to have somebody investigate the initial report and rootcause it. (And the motivation for that wasn't just some kind of a huge public outcry on the first report, because as far as I can tell that sank without a trace at least on HN). If the traffic from that customer is within the typical norms of git usage, rather than clearly abusive, then what further action would you expect…
Initially, I wanted to reply to you and say it's unfair to judge the person as incompetent; I still think it's a bit unfair because they develop a specialized library and their choice of hosting shouldn't need to be modernized at every step. However, reading through their response thread [0] I do think they aren't appearing particularly mature . It feels like they are refusing to characterize what is actually going o…
Granlund is a world class expert and apart from gmp has developed (with Montgomery) and implemented several fast arithmetic algorithms in gcc.
This is standard mailing list talk that always includes a bit of posturing. It predates the joyless, grey, boring, judgemental GitHub hypocrites. You are all so mature!
Re: The GMP library's repository is under attack by a single GitHub user
#75Earlier quoted context omitted.
They've clearly worried enough about it to have somebody investigate the initial report and rootcause it. (And the motivation for that wasn't just some kind of a huge public outcry on the first report, because as far as I can tell that sank without a trace at least on HN). If the traffic from that customer is within the typical norms of git usage, rather than clearly abusive, then what further action would you expect…
Yes. Repeated duplicate requests from MSFT ips are very clearly caused by gmp's server misconfiguration.
1. They did not provide enough information for anyone at Microsoft reading the message (probably due to being forwarding it to actually investigate the problem. Like, IP addresses, timestamps, the requested URLs. Hell, they even talk of just "identical requests" rather than identifying them as repository clones. Everything about the message is basically custom made to make investigating it as hard as possible.
2. If the goal was to pressure Microsoft into changing something by rousing up a mob with torches and pitchforks, they should have provided enough information for the actual audience (i.e. the mob) to judge the severity of this "attack". Like, the original complaint was about "thousands" of requests. Over what time period? How long was it sustained? (The logs they pasted in the email thread showed 15 requests over a time period of several minutes.)
3. If the goal was to convince Microsoft to take some action due to some reason other than public pressure, than the aggressive and uninformed communications style (copied by their user community) was not going to do them any favors. If you want people to help you, it helps for them to like you or at least have a neutral opinion. This kind of theatrical posturing might make you feel good, but it's going to actively harm your chances of affecting any change.
4. Reacting by blocking traffic from MS IPs to all ports. Given they weren't under attack, there was no reason to block SMTP and potentially block the main communications channel. Or block the website that contains the message they wanted Microsoft to read. But that's what they chose to do. This appears to actually have been like single-digit qps at most. There really was no reason for a layer 3 firewall to be the first solution to try, but even if it were the chose solution you should be able to target it to just hg with less collateral damage.
5. Running an underprovisioned service (they brag about the big real server hardware they have, but then it turns out the repository serving has access to a single core) and then sticking to it. Increasing capacity to soak the problematic traffic is like step one, and would clearly have been an option here. It moves the incident from an emergency to something that could be managed with less stress. And even if IP blocks turned out to be the chosen long-term solution, having a bit of overprovisioning means that things don't break when it turns out you'd missed an IP.
6. Trying to block IPs one by one. It's a server hosting a code repository for God's sake. Literally the first thing anyone should have thought of is that it's a CI pipeline, or something similar, and that if it was MS address space that it'd be GitHub. And the first search result for "github egress ips" will lead to GitHub's official, up to date, list of egress IPs [0], that they could then have blocked in one batch rather than play whack a mole.
So no, despite your snark I think's it's fair to say there was a clear lack of competence in dealing with a potential abuse issue. Given all those signs and the complete lack of details, why would we believe there analysis of this being an attack or abusive traffic were correct? Because, you know, it wasn't.
(Now, is it reasonable to expect everyone to be competent at this? Of course not. But it is reasonable to not just accept at face value the analysis of somebody who clearly does not have the expertise to do it correctly.)
[0] https://docs.github.com/en/authentication/keeping-your-accou...
Re: The GMP library's repository is under attack by a single GitHub user
#76Earlier quoted context omitted.
- Threatening lawyers - The fair use update is reactive, and won't help anything - Not separating microsoft/github from users of the platform (I agree the line is an worthwhile discussion) - Not following up on the reasonable technical solutions that are being proposed - Not seeming to ack that while the firewall ban might be the best they can do now, it'll block interested users of their own (GMP) library. That said…
To be fair, the amount of traffic we are talking about is not really that significant. GMP only allocating one core [1] for these requests and being frustrated that after so long they might have to increase the resource allocation or change something else is a bit unreasonable too. [1] https://gmplib.org/list-archives/gmp-devel/2023-June/006169....
Re: The GMP library's repository is under attack by a single GitHub user
#77Earlier quoted context omitted.
I'm confused by this statement: > GMP is at this point the only dependency that does not offer a sane way to clone its repository. The latest release is from years ago and https://gmplib.org/download/gmp/gmp-6.2.1.tar.lz should contain every file necessary. Why would you need access to the commit history in a build script? In fact, why would you need to download a fresh copy of a dependency that almost never gets upd…
The idea that you build software from a fixed archive is passé. Everything must be a git clone. Or if they don't use git, you muddle along with hg.
Re: The GMP library's repository is under attack by a single GitHub user
#78Earlier quoted context omitted.
It's totally reasonable to say that it isn't malicious and they aren't going to stop it, but it's unreasonable and unhelpful to add "seems like your servers just suck". They also missed or failed to mention that there are 700 forks of the project running the same thing at the same time.
I’m a bit confused as to why 700 automated clone requests per day would be an unreasonable amount of traffic for a project like GMP. That doesn’t sound like much traffic to me, especially since they reportedly have a 24 core CPU with 256GB of RAM. Is cloning a mercurial repo highly server intensive?
Re: The GMP library's repository is under attack by a single GitHub user
#79Earlier quoted context omitted.
Github could cache it then to improve this?
The title is a bit confusing. It seems the GMP project has a self-hosted Mercurial repo ( https://gmplib.org/devel/repo-usage ) which is being hit by a user at Microsoft that they've somehow tied back to a single github account. I'm sure it's going to be CI or mirroring or some other automated process.
Re: The GMP library's repository is under attack by a single GitHub user
#80Earlier quoted context omitted.
It's 700 automated clone requests at the same moment due to inheriting a cronjob.
I dunno. To me it seems like if a web server can't deal with 700 simulatenous requests it should probably call it quits. They're of course within their rights to just block the connections but it certainly doesn't look like a denial of service attack to me. Submitting a link to HN probably causes even bigger traffic spikes.