71–80 of 110 posts
Earlier quoted context omitted.
obviously someone is good at detecting it considering 6.5k in usage got racked up with data scraped within 10 minutes of commit
The quick bitcoin mining spin-ups, and the initial email from Amazon about the key going public, are (probably) thanks to the github events firehose. https://github.com/hootware/github-firehose-node http://blog.scalyr.com/2013/10/exploring-the-github-firehose...
> As a senior developer with over 12 years of experience in the financial services industry, I didn't think it was possible that I could be the victim of a data breach. Ok... > Better yet, move access keys to a seperate config file, and exclude this from Git deploys with a .gitignore. No shit?
Even that's a terrible solution. AWS specifically changed their tooling to look for access keys in ~/.aws and make it more difficult to load them from inside a project directory. It's really not that hard to setup MFA and require it when using STS to assume whatever role you need to make changes. For automated changes, you can give instances the necessary an IAM role and only run those from inside EC2. He didn't get…
> He didn't get a $6.5k bill because of a bug in VisualStudio, he got one because he didn't finish setting up his AWS account. Regardless of the public/private mixup, checking in AWS credentials to any repository is just bad practice and creates a huge risk of the exact situation that happened in this story.
I agree
Earlier quoted context omitted.
AWS do do things. They have service limits and routinely waive charges from abuse events like this. I do wonder if GitHub's events firehose has benefits that outweigh all the compromised credentials, though. Not just limited to AWS keys.
> They have service limits and routinely waive charges from abuse events like this. That's almost worse though. Maybe I'm being unfair here, but the fact that they will waive charges immediately if you ask, but don't just block them pre-emptively, tells me that they're hoping at least some customers (with deep pockets and automated billing that no one pays attention to) will just pay the bill and never notice. Maybe…
Earlier quoted context omitted.
I don't know. Even for private repos, it's bad practice to commit private keys to source control.
> Even for private repos, it's bad practice to commit private keys to source control. Is it, though? Committing them to the same repository as one's code lives in, sure, but committing them to a separate production-deploy repo seems okay to me (although I'd much prefer that private repo never to hit a centralised service like GitHub).
Yes, it is. Period. Full Stop. Don't ever check access keys into any repo, public, private or even self-hosted. AWS needs to do a better job of making you realize that access keys are like 100-year-old sticks of dynamite and should be handled with an equal amount of care considering they can cause a similar amount of damage.
To their credit, they basically noted this when they changed the way access keys are handled in the command line tool[1]. Quoting:
An important point is that the default location for the credentials file is a user
directory. It's no longer part of a project file structure, such as an app.config file
(.NET) or .properties file (Java). This can enhance security by allowing you to keep the
credentials in a location that's accessible only to you, and it makes it less likely that
you'll inadvertently upload credentials if you upload a project to a developer sharing
site like GitHub.
This is becoming such a common story that I'm starting to think it's a little irresponsible of both Amazon and GitHub to not react better to it. Clearly Amazon is capable of detecting when this happens as evidenced by their rapid response; when they see AWS keys published to GitHub, they should block the account, or at least prevent it from spinning up any new instances. They can then contact the owner over e-mail to…
From committing the basic design error of using hard-coded credentials at all; to committing them to any kind of repository; to failing to check on the visibility of that repository; to giving those credentials privileges they did not need; to failing to act effectively on warning messages.
And all this, apparently, is laid at the feet of Visual Studio.
I have no love of the Microsoft ecosystem, but the author does not acquit themselves well in this incident. The arrogant claims of seniority in the opening paragraph seem ironic in this light.
1) Seems like an inherent risk of closed source tools. 2) Someone other than the OP should pickup/waive the tab.
Are we really going to run this story every month? "I posted my private AWS keys by mistake on github, a spider got them, I got a huge bill" Yeah, you don't say
Don't be so quick to dismiss this one. This is notable for several reasons, including (1) it wasn't his mistake, but rather that of a Visual Studio bug not following his instructions to make a private repo on GitHub; (2) the speed in which this happened (minutes), and (3) it has useful analysis into some sorely lacking functionality in AWS that lets this continue to happen.
Are we really going to run this story every month? "I posted my private AWS keys by mistake on github, a spider got them, I got a huge bill" Yeah, you don't say
He didn't post it by mistake. He asked Visual Studio to create a private repo and it created a public one. A bug which he reproduced.
Earlier quoted context omitted.
> Even for private repos, it's bad practice to commit private keys to source control. Is it, though? Committing them to the same repository as one's code lives in, sure, but committing them to a separate production-deploy repo seems okay to me (although I'd much prefer that private repo never to hit a centralised service like GitHub).
> Is it, though? Yes, it is. Period. Full Stop. Don't ever check access keys into any repo, public, private or even self-hosted. AWS needs to do a better job of making you realize that access keys are like 100-year-old sticks of dynamite and should be handled with an equal amount of care considering they can cause a similar amount of damage. To their credit, they basically noted this when they changed the way access…
So, how do you propose to handle version control of your exact production configuration? Do you think that recording one's production configuration is a valuable (or even useful) technique? Do you propose to have a single file with production configuration relying only on point-in-time backups for history? That gets us back to the bad old days…
A local or self-hosted repo is 'a local location that's accessible only to youation that's accessible only to you.'