Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Deleting Sensitive Information from GitHub Doesn't Save You

jordan-wright.github.io

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#31

Post by Dylan16807 »

ncallaway wrote: post #8
tomphoolery wrote: post #2

It should be noted that GitHub's article on removing sensitive data is still applicable if you haven't pushed anything back to GitHub yet. Remember that a commit is just an entry into your repo, it doesn't synchronize with `origin/master` until you tell it to. So if the user has not pushed to GitHub yet, but has committed in their local Git repo, they should follow GitHub's guide and not worry about changing any keys…

While it's absolutely true that if the credentials haven't been pushed then you are not compromised, I would still encourage people to rotate their credentials regardless. All it takes is a mistake when deleting the sensitive information, or having pushed without realizing it to be compromised. Even if you're absolutely positive there wasn't a breach, it can be a good excuse to drill for a _real_ breach later. It nev…

A credential rotation drill does sound useful, but I'm kind of uncomfortable with assuming like that; it seems like sloppy thinking that can cause trouble down the line.

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#32

Post by jemfinch »

mkal_tsr wrote: post #29
nutanc wrote: post #27

I think this problem is widespread enough and there are enough idiots out there(me included),that there should be a feature request for Github to provide a prompt in case Github detects sensitive information in the code hosted.

> there should be a feature request for Github to provide a prompt in case Github detects sensitive information in the code hosted. Sure, just enumerate any and all possible types of sensitive data, the format they may be in, regex / matching functions to account for them (supported across 20+ programming languages) and I'm sure Github will have that done asap. Alternatively, don't commit passwords/API-keys/sensitive…

> Sure, just enumerate any and all possible types of sensitive data

False dichotomy. It doesn't have to be "everything" or "nothing". An 80% solution here is better than nothing.

I still find it useful that gmail warns me before I send an email without an attachment if I've written "I've attached" in an email. Can gmail detect with 100% accuracy if I intended to send an attachment? Of course not. But the 80% solution here still does me and lots of other people a lot of good.

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#33

Post by nutanc »

mkal_tsr wrote: post #29
nutanc wrote: post #27

I think this problem is widespread enough and there are enough idiots out there(me included),that there should be a feature request for Github to provide a prompt in case Github detects sensitive information in the code hosted.

> there should be a feature request for Github to provide a prompt in case Github detects sensitive information in the code hosted. Sure, just enumerate any and all possible types of sensitive data, the format they may be in, regex / matching functions to account for them (supported across 20+ programming languages) and I'm sure Github will have that done asap. Alternatively, don't commit passwords/API-keys/sensitive…

80:20 rule.I am sure, if we check for api_key,password(variants),key should help 80% of the people from making mistakes

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#34

Post by jemfinch »

sesteel wrote: post #25
TTPrograms wrote: post #17

Earlier quoted context omitted.

I've always wondered the proper way to deal with this, and this makes total sense. How would you typically set such an environment variable? In bash init?

Environment variables can work well for development but I wouldn't put them in .bashrc or .bash_profile; if you are like me, you like to store your dot files somewhere public. I typically leave them in an encrypted file on dev systems, but this only solves the accidental over the shoulder problem. Production systems require another level of security altogether. Typically, I've seen services run in restricted user acc…

> if you are like me, you like to store your dot files somewhere public.

In your public .bashrc, put a line "source .bashrc.secret." Just keep an empty .bashrc.secret in your public repository, and keep your actual secret credential on your machines.

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#35

Post by mkal_tsr »

jemfinch wrote: post #32
mkal_tsr wrote: post #29

Earlier quoted context omitted.

> there should be a feature request for Github to provide a prompt in case Github detects sensitive information in the code hosted. Sure, just enumerate any and all possible types of sensitive data, the format they may be in, regex / matching functions to account for them (supported across 20+ programming languages) and I'm sure Github will have that done asap. Alternatively, don't commit passwords/API-keys/sensitive…

> Sure, just enumerate any and all possible types of sensitive data False dichotomy. It doesn't have to be "everything" or "nothing". An 80% solution here is better than nothing. I still find it useful that gmail warns me before I send an email without an attachment if I've written "I've attached" in an email. Can gmail detect with 100% accuracy if I intended to send an attachment? Of course not. But the 80% solution…

80/20 rule goes a looooooong way, but relying on a 3rd party to handle your infosec is terribly misguided. Offloading your net/infoseq issues to a 3rd party doesn't guarantee you understand the attack surface nor mitigation steps. The advantage of doing it yourself is you learn and integrate knowledge as you go. Yes, more error prone, but imo it is more lasting as you (generally) retain the knowledge as time goes on. The issue with relying on a 3rd party is it becomes a "don't-care" because 'someone' else is taking care of it for you (github, sr. sw engr, sysadmin, etc.).

There will never be a 100%-fool-proof "Did you mean to commit this sensitive unicode string?" but getting in front of that with a, "ok, I've checked my code, ran my tests, pruned the sensitive data, is there anything I'm missing?" will go a long way both in present and future times.

There's an issue with your example. Google looks for the substr 'attach', but does it know that a file with a string 20 chars from the newline 50 lines deep with two single quotes is actually your root password? There's a world of difference between keying off of a word/phrase and understanding the context of a larger document+metadata. Trusting computers to do the latter will result in bad times for all, while learning for yourself can help spread the knowledge to those that are both technically inclined or not (infosec is everyone's issue!).

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#36

Post by infinitone »

guiambros wrote: post #14

> In this post, I’m going to show exactly how hackers instantly harvest information committed to public Github repositories... A few days ago I published my blog to GitHub, with my MailGun API key in the config file (stupid mistake, I know). In less than 12 hours, spammers had harvested the key AND sent a few thousand emails with my account, using my entire monthly limit. Thankfully I was using the free MailGun accou…

I had a similar but less pleasant experience. I had decided to opensource an old side project of mine, that gets a good amount of users daily. And by that, it was just initially to make the repo public. But I had totally forgot about the mail server keys- this was a paid mail server, so you can imagine my disbelief when I get an email of a $1000 bill and a complaint saying that I had sent upwards of 250k emails with what seemed to be a iOS mail app malware email. Luckily it was resolved within a week with support.

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#37

Post by jemfinch »

mkal_tsr wrote: post #35
jemfinch wrote: post #32

Earlier quoted context omitted.

> Sure, just enumerate any and all possible types of sensitive data False dichotomy. It doesn't have to be "everything" or "nothing". An 80% solution here is better than nothing. I still find it useful that gmail warns me before I send an email without an attachment if I've written "I've attached" in an email. Can gmail detect with 100% accuracy if I intended to send an attachment? Of course not. But the 80% solution…

80/20 rule goes a looooooong way, but relying on a 3rd party to handle your infosec is terribly misguided. Offloading your net/infoseq issues to a 3rd party doesn't guarantee you understand the attack surface nor mitigation steps. The advantage of doing it yourself is you learn and integrate knowledge as you go. Yes, more error prone, but imo it is more lasting as you (generally) retain the knowledge as time goes on.…

> but relying on a 3rd party to handle your infosec is terribly misguided.

No one's suggesting anything like that. It's just a basic protection that Github could offer its users, because its users are humans, and humans make mistakes.

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#38

Post by mkal_tsr »

jemfinch wrote: post #37
mkal_tsr wrote: post #35

Earlier quoted context omitted.

80/20 rule goes a looooooong way, but relying on a 3rd party to handle your infosec is terribly misguided. Offloading your net/infoseq issues to a 3rd party doesn't guarantee you understand the attack surface nor mitigation steps. The advantage of doing it yourself is you learn and integrate knowledge as you go. Yes, more error prone, but imo it is more lasting as you (generally) retain the knowledge as time goes on.…

> but relying on a 3rd party to handle your infosec is terribly misguided. No one's suggesting anything like that. It's just a basic protection that Github could offer its users, because its users are humans, and humans make mistakes.

But that's my point. It's not basic because it's so context dependent (hence the comment about the regex/functions).

No matter how clever you get with your pattern matching, you're going to have to always play catch-up with Web Framework N+1's format / weird-ass package manager. The dual approach is the only sensible approach because it expands your coverage. The important thing to this is to internalize the knowledge learned from 3rd parties and integrate that into your native process / tools, but not everyone will do that.

It'd be grand if we could say, "yeah, they should take care of my security for me because I'm paying them" but reality is a bitch. It doesn't matter what they were 'supposed' to do if there was an infosec leak or attack, you can't ctrl-z that (set of) event(s) and that info is out there, so it must be fixed (re-roll credentials, regen keys/certs, etc) and accounted for next time. There really is no 'getting ahead' but 'being less behind' will at least help mitigate getting eaten from the herd ;-)

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#39

Post by gojomo »

nutanc wrote: post #27

I think this problem is widespread enough and there are enough idiots out there(me included),that there should be a feature request for Github to provide a prompt in case Github detects sensitive information in the code hosted.

Similarly, a white hat could watch /events and warn users and/or services when credentials are 'burned'. (A major exploitable service like AWS might even want to do this itself.)

Re: Why Deleting Sensitive Information from GitHub Doesn't Save You

#40

Post by 1stop »

mkal_tsr wrote: post #38
jemfinch wrote: post #37

Earlier quoted context omitted.

> but relying on a 3rd party to handle your infosec is terribly misguided. No one's suggesting anything like that. It's just a basic protection that Github could offer its users, because its users are humans, and humans make mistakes.

But that's my point. It's not basic because it's so context dependent (hence the comment about the regex/functions). No matter how clever you get with your pattern matching, you're going to have to always play catch-up with Web Framework N+1's format / weird-ass package manager. The dual approach is the only sensible approach because it expands your coverage. The important thing to this is to internalize the knowledg…

If only github had a community of developers that could play such a catch up game... oh wait.

It's kind of amusing you are arguing that it's impossible to play this game, even though that's exactly what the perpetrators are doing, they are automatically detecting API keys and harvesting the code... maybe their script is hosted on github?

Post reply on HN