Anyone can access deleted and private repository data on GitHub • Hacker News Board

Re: Anyone can access deleted and private repository data on GitHub

#361

Post by Log_out_ » Thu, Jul 25, 2024, 5:16 PM UTC

pcthrowaway wrote: post #218

CGamesPlay wrote: post #200

Earlier quoted context omitted.

According to the screenshot in the documentation, though, new commits made to the fork will not be accessible by hash. So private feature branches in forks may be accessible via the upstream that was changed to public, if those branches existed at the time the upstream's visibility changed, but new feature branches made after that time won't be accessible.

OK but say a company has a private, closed source internal tool, and they want to open-source some part of it. They fork it and start working on cleaning up the history to make it publishable. After some changes which include deleting sensitive information and proprietary code, and squashing all the history to one commit, they change the repo to public. According to this article, any commit on either repo which was m…

Chat gpt given the following repo, create a plausible perfect commit history to create this repository.

Re: Anyone can access deleted and private repository data on GitHub

#362

Post by ethbr1 » Thu, Jul 25, 2024, 5:59 PM UTC

coldtea wrote: post #311

ethbr1 wrote: post #191

Earlier quoted context omitted.

Do some companies intend for their platform to feature remote code execution?

Some might very well do. E.g. a company with a service for training hackers and security researchers. In this case the question is moot, as this doesn't involve remote code execution.

Make a general point, get a general answer.

If the criteria for bug is "not intended", and that's solely judged by the company, then broken auth et al. suddenly become part of their product design.

If it quacks like a bug, it's a bug.

Re: Anyone can access deleted and private repository data on GitHub

#363

Post by reisse » Thu, Jul 25, 2024, 6:25 PM UTC

Perseids wrote: post #354

reisse wrote: post #303

Earlier quoted context omitted.

> After some changes which include deleting sensitive information and proprietary code, and squashing all the history to one commit, they change the repo to public. I know this might look like a valid approach on the first glance but... it is stupid for anyone who knows how git or GitHub API works? Remote (GitHub's) reflog is not GC'd immediately, you can try to get commit hashes from events history via API, and then…

> it is stupid for anyone who knows how git or GitHub API works? You need to know how git works and GitHub's API. I would say I have a pretty good understanding about how (local) git works internally, but was deeply surprised about GitHub's brute-forceable short commit IDs and the existence of a public log of all reflog activity [1]. When the article said "You might think you’re protected by needing to know the commi…

> You need to know how git works and GitHub's API.

No; just knowing how git works is enough to understand that force-pushing squashed commits or removing branches on remote will not necessarily remove the actual data on remote.

GitHub API (or just using the web UI) only makes these features more obvious. For example, you can find and check commit referenced in MR comments even if it was force-pushed away.

> was deeply surprised about GitHub's brute-forceable short commit IDs

Short commit IDs are not GitHub feature, they are git feature.

> If I use GitHub's web interface to click myself a repository with complex access rights, I assume they have concepts in place to thoroughly enforce these access rights.

Have you ever tried to make private GitHub repository public? There is a clear warning that code, logs and activity history will become public. Maybe they should include additional clause about forks there.

Re: Anyone can access deleted and private repository data on GitHub

#364

Post by Dylan16807 » Thu, Jul 25, 2024, 7:12 PM UTC

reisse wrote: post #363

Perseids wrote: post #354

Earlier quoted context omitted.

> it is stupid for anyone who knows how git or GitHub API works? You need to know how git works and GitHub's API. I would say I have a pretty good understanding about how (local) git works internally, but was deeply surprised about GitHub's brute-forceable short commit IDs and the existence of a public log of all reflog activity [1]. When the article said "You might think you’re protected by needing to know the commi…

> You need to know how git works and GitHub's API. No; just knowing how git works is enough to understand that force-pushing squashed commits or removing branches on remote will not necessarily remove the actual data on remote. GitHub API (or just using the web UI) only makes these features more obvious. For example, you can find and check commit referenced in MR comments even if it was force-pushed away. > was deepl…

> Short commit IDs are not GitHub feature, they are git feature.

They're a local feature sure. But you already have a list of local commits, just open the .git directory.

Can you connect to a vanilla git server and enumerate every single hash?

> Maybe they should include additional clause about forks there.

It would help but they need much more than a clause about forks.

Ideally they would purge that extra data when making something public.

Re: Anyone can access deleted and private repository data on GitHub

#365

Post by pcthrowaway » Thu, Jul 25, 2024, 7:36 PM UTC

reisse wrote: post #363

Perseids wrote: post #354

Earlier quoted context omitted.

> it is stupid for anyone who knows how git or GitHub API works? You need to know how git works and GitHub's API. I would say I have a pretty good understanding about how (local) git works internally, but was deeply surprised about GitHub's brute-forceable short commit IDs and the existence of a public log of all reflog activity [1]. When the article said "You might think you’re protected by needing to know the commi…

> You need to know how git works and GitHub's API. No; just knowing how git works is enough to understand that force-pushing squashed commits or removing branches on remote will not necessarily remove the actual data on remote. GitHub API (or just using the web UI) only makes these features more obvious. For example, you can find and check commit referenced in MR comments even if it was force-pushed away. > was deepl…

Dereferenced commits which haven't yet been garbage collected in a remote yet are not available to your local clones via git... I suppose there could be some obscure way to pull them from the remote if you know the hash (though I'm not actually sure), but either way (via web interface or CLI) you'd have to know the hash.

And it's completely reasonable to assume no one external to the org when it was private would have those hashes.

It sounds like github's antipattern here is retaining a log of all events which may leak these hashes, and is really not an assumption I'd expect a git user to make.

Re: Anyone can access deleted and private repository data on GitHub

#366

Post by Dylan16807 » Thu, Jul 25, 2024, 7:38 PM UTC

thrdbndndn wrote: post #219

I cannot access the commit https://github.com/trufflesecurity/trufflehog/commit/7bc0b shown in one of the pictures in the article (right before "Where do you get these hash values") despite this repo is even public. What gives?

It's probably either that they took the picture a while ago, or they were going character by character and accidentally screenshot with one too few. All of these are valid right now, and the first one is the one they show:

https://github.com/trufflesecurity/trufflehog/commit/7bc0b2 https://github.com/trufflesecurity/trufflehog/commit/7bc0b4 https://github.com/trufflesecurity/trufflehog/commit/7bc0b8 https://github.com/trufflesecurity/trufflehog/commit/7bc0b8

Re: Anyone can access deleted and private repository data on GitHub

#367

Post by pcthrowaway » Thu, Jul 25, 2024, 7:38 PM UTC

Dylan16807 wrote: post #364

reisse wrote: post #363

Earlier quoted context omitted.

> You need to know how git works and GitHub's API. No; just knowing how git works is enough to understand that force-pushing squashed commits or removing branches on remote will not necessarily remove the actual data on remote. GitHub API (or just using the web UI) only makes these features more obvious. For example, you can find and check commit referenced in MR comments even if it was force-pushed away. > was deepl…

> Short commit IDs are not GitHub feature, they are git feature. They're a local feature sure. But you already have a list of local commits, just open the .git directory. Can you connect to a vanilla git server and enumerate every single hash? > Maybe they should include additional clause about forks there. It would help but they need much more than a clause about forks. Ideally they would purge that extra data when…

> Can you connect to a vanilla git server and enumerate every single hash?

If you have ssh access yes, but I don't think you can do this with just git (and of course github doesn't provide ssh access to the git repo servers)

The public distribution of commit hashes via their event log seems really irresponsible on github's part to me.

Re: Anyone can access deleted and private repository data on GitHub

#368

Post by cutemonster » Thu, Jul 25, 2024, 8:19 PM UTC

sickblastoise wrote: post #314

pcthrowaway wrote: post #218

Earlier quoted context omitted.

OK but say a company has a private, closed source internal tool, and they want to open-source some part of it. They fork it and start working on cleaning up the history to make it publishable. After some changes which include deleting sensitive information and proprietary code, and squashing all the history to one commit, they change the repo to public. According to this article, any commit on either repo which was m…

Why not just create a new public repo and copy all of the source code that you want to it?

Because they haven't read the article and this HN discussion?

"Why not just...". Once you already know something it can seem obvious.

Re: Anyone can access deleted and private repository data on GitHub

#369

Post by quilnux » Thu, Jul 25, 2024, 9:04 PM UTC

There is a reason that anyone who cares about forks being private forever, (even if you delete it) should never use or trust a third party. I never use Github. I run my own git server and everyone else should to in my opinion. Github has always been a huge security problem.

But that's just me...

Re: Anyone can access deleted and private repository data on GitHub

#370

Post by dwaite » Thu, Jul 25, 2024, 9:17 PM UTC

Dylan16807 wrote: post #245

HeatrayEnjoyer wrote: post #240

Earlier quoted context omitted.

What about Save As bit people?

It's not doing Save As that bit people. Think of a .doc file as a bad database format. It gets lots of in-place overwrites, and fragments of old versions stick around. I can't find a lot that discusses it, but here's one mention: https://news.ycombinator.com/item?id=35252331

Right, OLE documents are comparable to read/write filesystems inside a file. The only mechanism given to make sure it is sparse was to create a new filesystem by having the application walk the existing one, basically a copy-based garbage collection.

Powerpoint files can be megabytes larger due to unused graphic artifacts; Word documents may contain older revisions of the text that contain deleted sections. Other things like the MSI installer file format are also OLE documents.

Microsoft eventually made Save in Office apps always create a new sparse filesystem to prevent these problems.