Just use a fucking blog, man. I'm so sick of threads like this. Edit: I'm sorry, this came off as way more aggressive than I intended. I get why people use twitter to share stuff like this, but it's much harder to archive, find or reference in the future, not to mention it being much less readable than a simple webpage. To anyone reading this, please consider publishing your findings on a blog as well as on twitter.
Or post it on Reddit or Medium or whatever if you can't be bothered with a blog. Twitter "threads" need to die.
What will happen when you commit secrets to a public Git repo?
21–30 of 68 posts
Re: What will happen when you commit secrets to a public Git repo?
#22For starters I recommend reading "How Bad Can It Git" [1] and "Detecting and Mitigating Secret-Key Leaks inSource Code Repositories" [2] papers.
After that you can read "How I made $10K in bug bounties from GitHub secret leaks" [3] and some notable reports on HackerOne Hacktivity [4] [5] and [6]. This last one is interesting - leaking secrets is not only about code repository! Actually it's about entire toolset used for software development, hence secret scanning could (should?) be performed for other places such as CICD logs or even Slack messages [7].
Anyhow, back to code repositories. GitHub and GitLab both recognized secrets as a problem, so they came up with solutions. If you use GitHub you can easily integrate GitGuardian [8] into your workflow ($$$) but even if you don't GitHub provides you with Secret Scanning feature [9] (both are mentioned within the Twitter and HN threads). If you use GitLab you have a Secret Detection feature [10] at your disposal BUT in order to use it you need to setup Auto DevOps (that's why in my experiment GitLab didn't alert me - I just pushed commits to my public repo but didn't setup anything).
Apart from built-in solutions provided by GitHub and GitLab, one can use tooling of their own choice. For this I'd recommend two types of solutions: proactive and reactive. For proactive security, as mentioned in the Twitter thread, you can use Talisman [11] as pre-commit hook. For reactive security you can use GitLeaks [12] (used by GitLab) or similar tools - there are many of them but one stands out, namely truffleHog [13] which can sniff each and every commit across all branches (also used by GitLab).
What if you already commited a secret into the public repository? Start with revoking and continue with this tutorial [14]
gl, hf.
[1] https://www.ndss-symposium.org/ndss-paper/how-bad-can-it-git... [2] https://people.eecs.berkeley.edu/~rohanpadhye/files/key_leak... [3] https://tillsongalloway.com/finding-sensitive-information-on... [4] https://hackerone.com/reports/716292 [5] https://hackerone.com/reports/396467 [6] https://hackerone.com/reports/496937 [7] https://github.com/PaperMtn/slack-watchman [8] https://www.gitguardian.com/ [9] https://developer.github.com/partnerships/secret-scanning/ [10] https://docs.gitlab.com/ee/user/application_security/sast/#s... [11] https://github.com/thoughtworks/talisman [12] https://github.com/zricethezav/gitleaks [13] https://github.com/dxa4481/truffleHog [14] https://docs.github.com/en/free-pro-team@latest/github/authe...
Re: What will happen when you commit secrets to a public Git repo?
#23Earlier quoted context omitted.
Why not refuse to publish a detected secret at all until the repo owner takes an action to allow it?
There are a few considerations on that one, but one very practical reason is the developer experience of dealing with false positives. False positives are one of the big problems in secret scanning. Some partners issue credentials with patterns that make them very hard to distinguish from innocuous strings. For example, a Datadog token looks identical to a commit SHA. We would never block developers from pushing comm…
Re: What will happen when you commit secrets to a public Git repo?
#24Just use a fucking blog, man. I'm so sick of threads like this. Edit: I'm sorry, this came off as way more aggressive than I intended. I get why people use twitter to share stuff like this, but it's much harder to archive, find or reference in the future, not to mention it being much less readable than a simple webpage. To anyone reading this, please consider publishing your findings on a blog as well as on twitter.
Re: What will happen when you commit secrets to a public Git repo?
#25Just use a fucking blog, man. I'm so sick of threads like this. Edit: I'm sorry, this came off as way more aggressive than I intended. I get why people use twitter to share stuff like this, but it's much harder to archive, find or reference in the future, not to mention it being much less readable than a simple webpage. To anyone reading this, please consider publishing your findings on a blog as well as on twitter.
Hey, OP here. I agree that a blog post would be more readable. In this particular case I just didn't expect that it will catch fire. If I would then I would spend more time on the form. I won't make that mistake again (i.e. in the future I will use a blog post as main driver of such twitter thread).
Thanks for writing about your experiment.
Re: What will happen when you commit secrets to a public Git repo?
#26Cool experiment! I PM the secret scanning team at GitHub and wanted to mention what GitHub did behind the scenes here. GitHub scans every commit to a public repo for secrets one of our secret scanning partners may have issued. We forward those candidate secrets to the issuing partner, and they take action. In some cases they auto-revoke the secret (AWS normally does this, I believe), in some cases they notify the use…
I realize that the search space is huge for many tokens types, but it seems viable.
Re: What will happen when you commit secrets to a public Git repo?
#27I remember one time I installed Windows 95/98. I wanted the PC to be on internet but did not have a firewall for Windows. But I knew the internet address where I could get one.
So after installing Windows I took my chances, connected to the internet, downloaded the firewall asap, installed it, and was already too late. The PC was compromised within 10 minutes and I had to reinstall it.
Re: What will happen when you commit secrets to a public Git repo?
#28Cool experiment! I PM the secret scanning team at GitHub and wanted to mention what GitHub did behind the scenes here. GitHub scans every commit to a public repo for secrets one of our secret scanning partners may have issued. We forward those candidate secrets to the issuing partner, and they take action. In some cases they auto-revoke the secret (AWS normally does this, I believe), in some cases they notify the use…
Couldn't auto-revokation be used for a "DOS" attack of sorts by generating a lot of randomized tokens and pushing them to any repo? I realize that the search space is huge for many tokens types, but it seems viable.
Re: What will happen when you commit secrets to a public Git repo?
#29Earlier quoted context omitted.
Couldn't auto-revokation be used for a "DOS" attack of sorts by generating a lot of randomized tokens and pushing them to any repo? I realize that the search space is huge for many tokens types, but it seems viable.
Selecting (reading) data is very fast most of the time. So if no token matches I don't think this will result in a DOS.
Re: What will happen when you commit secrets to a public Git repo?
#30Earlier quoted context omitted.
Couldn't auto-revokation be used for a "DOS" attack of sorts by generating a lot of randomized tokens and pushing them to any repo? I realize that the search space is huge for many tokens types, but it seems viable.
Selecting (reading) data is very fast most of the time. So if no token matches I don't think this will result in a DOS.