Live data from Hacker News

Repositories held for ransom by using valid credentials

about.gitlab.com

121–130 of 158 posts

Re: Repositories held for ransom by using valid credentials

#121
post #6

The current title "Gitlab.com Was Compromised" doesn't seem accurate. There's someone (or a group) currently attacking online repositories (gitlab is not the only affected provider) using passwords found in scans for files like .gitconfig's and the like. Unless new information comes to light about gitlab specifically being compromised, I'd say this is more about individual private repos being on the sights of a targe…

So git doesn't let you add the `.git` to the index. Most reports I've seen mention that SourceTree was used as a git client. Is it possible that SourceTree committed .git and pushed it to remotes which were then scraped?

https://twitter.com/bad_packets/status/1124429828680085504

Looks like someone was scraping for `.git/config`

Re: Repositories held for ransom by using valid credentials

#122
While the fault lies with the users for not following security best practices, including enabling 2FA there are things gitlab/any site can do to help defend against these sorts of attacks. Some suggestions: Treat logins from datacenters as suspicious. (In this case the IP block identified belongs to World Hosting Farm Limited). Treat logins from a new/different ISP as suspicious. Limit access to the account and verify the login via email. It’s not foolproof but as part of a defense in depth strategy it can be quite effective.

Re: Repositories held for ransom by using valid credentials

#123
post #96

Earlier quoted context omitted.

They do keep disk snapshots for 2 weeks though (created twice a day). https://about.gitlab.com/handbook/engineering/infrastructure...

Thanks for linking to that. I’m asking inside the company about the comment that the repositories are lost. I think it is a lot of work to restore individual repositories as opposed to restoring a disk, so maybe that is why we said that.

Also any commits pushed between the last snapshot and the deletion would be lost too.

Re: Repositories held for ransom by using valid credentials

#124
post #102
post #48

Earlier quoted context omitted.

> Wouldn't that that imply every secret accidentally committed and then 'deleted' is still accessible This should be a moot point because anyone (in IT) should realize that an accidentally committed secret is now 100% public for all eternity and needs to be rendered irrelevant to restore secure operations.

And a hundred times so for any public repos. There are bots feeding on the GitHub firehose, scavenging for accidentally committed credentials. A few years back (2015 or so) the average time from push-to-repo to AWS account compromise was 6 minutes . Surely that time has only gone down, and the number of different credentials identified has gone up.

> A few years back (2015 or so) the average time from push-to-repo to AWS account compromise was 6 minutes. Surely that time has only gone down, and the number of different credentials identified has gone up.

I don't doubt that a second and I'd like to use that as a quote. I'd like to be prepared if someone doubts it, so: Do you have a primary source for this?

Re: Repositories held for ransom by using valid credentials

#125
post #102

Earlier quoted context omitted.

And a hundred times so for any public repos. There are bots feeding on the GitHub firehose, scavenging for accidentally committed credentials. A few years back (2015 or so) the average time from push-to-repo to AWS account compromise was 6 minutes . Surely that time has only gone down, and the number of different credentials identified has gone up.

> A few years back (2015 or so) the average time from push-to-repo to AWS account compromise was 6 minutes. Surely that time has only gone down, and the number of different credentials identified has gone up. I don't doubt that a second and I'd like to use that as a quote. I'd like to be prepared if someone doubts it, so: Do you have a primary source for this?

I'll need to find the talk I lifted it from. Not easy... but looks like downthread a sibling comment gives a relatively decent update about the current speed of compromise.

Re: Repositories held for ransom by using valid credentials

#126
post #33
post #17

> We believe that no data has been lost, unless the [...] GitLab copy was the only one. One difference between how GitLab and GitHub run their infrastructure is that GitLab doesn't keep reflogs, and uses git's default "gc" settings. As a result they won't have the data in question anymore in many cases[1]. Well, I don't 100% know that for sure, but it's the default configuration of their software, and I'm assuming th…

This tendency of Hacker News users to want to monetize everything is sickening.

To explain some of my sickening tendencies: I'm not associated with any such for-profit hosting site, so I have nothing to gain from this. I just use them.

Implementing such a feature would cost resources that someone would have to pay for. Storage costs would go up, it's not atypical that e.g. a repo that's 100MB on disk might be 1.5x or 2x that (or beyond) if you were keeping every version of every ref ever. Think e.g. accumulating throwaway topic branches with library imports you never ended up using.

So how do you pay for running such a thing, nevermind the initial development cost?

You could just make it "free", but then you'd need to roll the cost onto customers across the board. Or you could only enable such "backups" for opt-in paying customers, but most people aren't going to think to enable/pay for that, or think "I won't need this", until they day they do.

So wouldn't it be neat to have such a service on in the background, funded by high premiums to recover the data in case their backup version is your last option?

I've certainly permanently lost personal data by accident where I'd wished I could have paid hundreds of dollars to get back, nevermind someone for whom such a thing might be of critical business importance.

Think about it as being able to pay money after-the-fact to undo the car crash you just got into. With technology that becomes feasible in some cases, and in particular due to how git stores data & what people tend to store there it's relatively cheap compared to some other types of storage.

Re: Repositories held for ransom by using valid credentials

#127
post #52

I don't get the ransom thing: users of a git repository have a clone of the repo that contains the whole history, no? So isn't it trivial to recreate the repository?

The attacker is also threatening to make these private repos public, or misuse their access to the repos in other ways (likely additional types of breaches).

Re: Repositories held for ransom by using valid credentials

#128
post #52

I don't get the ransom thing: users of a git repository have a clone of the repo that contains the whole history, no? So isn't it trivial to recreate the repository?

For some repositories the code may be of little concern if the hacker shares it with the world or deletes it.... but it could impact some users.

Re: Repositories held for ransom by using valid credentials

#129
post #93

Earlier quoted context omitted.

That is a crazy-ass quote. “We believe that no data has been lost... well, except for the data we keep. But you weren’t actually relying on us to save any data, right?” I know, back up everything at least twice. But still, when somebody loses one of your copies, they don’t get to say “it’s cool, no data was lost, you have other copies, right?”

To provide additional context, on GitLab.com, we maintain two weeks' backups. The last time we restored a single project repo, it was a significant effort that utilized many hours of an SRE's time to complete.

What’s crazy isn’t the loss of data (that happens, and I don’t really expect most cloud services to be save me from explicit deletion requests) but rather framing it as no data being lost because your customers have it elsewhere.

Re: Repositories held for ransom by using valid credentials

#130

Earlier quoted context omitted.

Monetyzer: The world is your oyster, it's about time you start collecting pearls.

Maybe this comment was inspired by WellDeserved, a truly underappreciated but still important app: https://www.youtube.com/watch?v=WoK4_dQbfuU

No, but that was incredible. Thanks for introducing me to Cultivated Wit.
Post reply on HN