Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Repositories held for ransom by using valid credentials

about.gitlab.com

Post reply on HN
Print view View original

111–120 of 158 posts

Re: Repositories held for ransom by using valid credentials

#111

Post by gitlab-security »

mikeash wrote: post #93
avar wrote: post #17

> We believe that no data has been lost, unless the [...] GitLab copy was the only one. One difference between how GitLab and GitHub run their infrastructure is that GitLab doesn't keep reflogs, and uses git's default "gc" settings. As a result they won't have the data in question anymore in many cases[1]. Well, I don't 100% know that for sure, but it's the default configuration of their software, and I'm assuming th…

That is a crazy-ass quote. “We believe that no data has been lost... well, except for the data we keep. But you weren’t actually relying on us to save any data, right?” I know, back up everything at least twice. But still, when somebody loses one of your copies, they don’t get to say “it’s cool, no data was lost, you have other copies, right?”

To provide additional context, on GitLab.com, we maintain two weeks' backups. The last time we restored a single project repo, it was a significant effort that utilized many hours of an SRE's time to complete.

Re: Repositories held for ransom by using valid credentials

#112

Post by commandersaki »

vasco wrote: post #6

The current title "Gitlab.com Was Compromised" doesn't seem accurate. There's someone (or a group) currently attacking online repositories (gitlab is not the only affected provider) using passwords found in scans for files like .gitconfig's and the like. Unless new information comes to light about gitlab specifically being compromised, I'd say this is more about individual private repos being on the sights of a targe…

So git doesn't let you add the `.git` to the index. Most reports I've seen mention that SourceTree was used as a git client. Is it possible that SourceTree committed .git and pushed it to remotes which were then scraped?

Re: Repositories held for ransom by using valid credentials

#113

Post by jobigoud »

ctennis1 wrote: post #43
yoz-y wrote: post #16

Earlier quoted context omitted.

Out of all things to hold for ransom git repos seem like a bad idea. Most of the time there are multiple clones lying around anyways. I agree that having the source code leak can be bad news, but the code itself being secret should not be a critical part of the business.

Probably not, but I suspect some people will still pay the ransom anyway. Keep in mind, many people also do keep "secrets" used for other authenticating to other services in their git repos - even though it's a terrible idea.

These secrets are now compromised anyway.

Re: Repositories held for ransom by using valid credentials

#114

Post by ak217 »

masklinn wrote: post #25
avar wrote: post #17

> We believe that no data has been lost, unless the [...] GitLab copy was the only one. One difference between how GitLab and GitHub run their infrastructure is that GitLab doesn't keep reflogs, and uses git's default "gc" settings. As a result they won't have the data in question anymore in many cases[1]. Well, I don't 100% know that for sure, but it's the default configuration of their software, and I'm assuming th…

I'm surprised github runs regular git. I'd always assumed they were emulating it, especially with the lag we've observed between github-api and github-git at $DAYJOB (update repo 1 via api, update repo 2 via api, fetch repo 1 and repo 2 via git, we've had cases where the repo 2 update was visible but not the repo 1).

I think github runs https://github.com/libgit2/libgit2 which is not regular git.

Re: Repositories held for ransom by using valid credentials

#115

Post by theWheez »

A client of mine was hit by this. Makes me think I aught to go into security.. I brought up many concerns, and specifically about git access (using very weak mechanisms), a few months ago, to which I was told to "clean things up when I can" but that it wasn't a priority.

Re: Repositories held for ransom by using valid credentials

#116

Post by gcb0 »

avar wrote: post #17

> We believe that no data has been lost, unless the [...] GitLab copy was the only one. One difference between how GitLab and GitHub run their infrastructure is that GitLab doesn't keep reflogs, and uses git's default "gc" settings. As a result they won't have the data in question anymore in many cases[1]. Well, I don't 100% know that for sure, but it's the default configuration of their software, and I'm assuming th…

you are assuming mistakenly deleted data.

circumventing this is very trivial for an attacker.

Re: Repositories held for ransom by using valid credentials

#117

Post by insomniacity »

gitlab-security wrote: post #111
mikeash wrote: post #93

Earlier quoted context omitted.

That is a crazy-ass quote. “We believe that no data has been lost... well, except for the data we keep. But you weren’t actually relying on us to save any data, right?” I know, back up everything at least twice. But still, when somebody loses one of your copies, they don’t get to say “it’s cool, no data was lost, you have other copies, right?”

To provide additional context, on GitLab.com, we maintain two weeks' backups. The last time we restored a single project repo, it was a significant effort that utilized many hours of an SRE's time to complete.

"many hours" is too high. Regardless of this incident, if backups take too long to restore, you may as well not have them.

Re: Repositories held for ransom by using valid credentials

#118

Post by GL-pharrison »

insomniacity wrote: post #117
gitlab-security wrote: post #111

Earlier quoted context omitted.

To provide additional context, on GitLab.com, we maintain two weeks' backups. The last time we restored a single project repo, it was a significant effort that utilized many hours of an SRE's time to complete.

"many hours" is too high. Regardless of this incident, if backups take too long to restore, you may as well not have them.

I agree, the longer it takes to recover the less valuable the backups are. However, in this context where we're restoring individual repo's (and all the metadata baggage involved) it's much different than a DB restore or other disaster recovery/prevention mechanisms.

The per-repo restore time today is not where it should be. We're working to speed this up so we can help users recover and get back to a productive state quickly.

Re: Repositories held for ransom by using valid credentials

#119

Post by tjr225 »

username223 wrote: post #110
bostik wrote: post #102

Earlier quoted context omitted.

And a hundred times so for any public repos. There are bots feeding on the GitHub firehose, scavenging for accidentally committed credentials. A few years back (2015 or so) the average time from push-to-repo to AWS account compromise was 6 minutes . Surely that time has only gone down, and the number of different credentials identified has gone up.

> the average time from push-to-repo to AWS account compromise was 6 minutes . Wow, I didn't realize it had become so efficient, but I shouldn't be surprised. I never really understood the value in hosting non-public software in the public, and if it's open source, it shouldn't be getting anywhere near secrets that can be used to extract money from its developers. I remember thinking, back when it became trendy for p…

I had accidentally pushed an AWS credential out a month or two ago- within about a minute and a half AWS had disabled the IAM user, and automatically emailed me(as well as my entire org- how embarrassing!)- when we were going through the access logs it looked like it had taken only a minute and a half longer for some other, presumably malicious, system to attempt to access my compromised user. Probably between 2 or 3 minutes total. I'm not a huge Amazon fan but props to AWS for saving my butt.

Re: Repositories held for ransom by using valid credentials

#120

Post by edbaskerville »

soulofmischief wrote: post #54
antoineMoPa wrote: post #36

Earlier quoted context omitted.

I wonder if one could monetize this tendency.

Monetyzer: The world is your oyster, it's about time you start collecting pearls.

Maybe this comment was inspired by WellDeserved, a truly underappreciated but still important app:

https://www.youtube.com/watch?v=WoK4_dQbfuU

Post reply on HN