Live data from Hacker News

How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

humankode.com

51–60 of 110 posts

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#51
post #16
post #8

Earlier quoted context omitted.

There's really no financial incentive to do so.

When people run up a huge bill and don't pay it, that's an incentive.

The profit margin on AWS is stupid high. It costs Amazon nothing but power and opportunity cost to refund unintended use.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#52
post #37

Why does somebody pushes AWS Keys into a Git Repository? Why should somebody ever push some Authentications? Currently when I code an app I create a database with name of the app and have new credentials with just name:name so that the only thing that I check into my config if the app needs a application.conf or resource.conf are default username:passwords for development, so that bootstrapping is easier. However EVE…

> Why does somebody pushes AWS Keys into a Git Repository?

Wild speculation, but this is a side effect of "developer" centric operations. Everything is done via code, and developers are instinctively trained to check in code.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#54
post #24

This is becoming such a common story that I'm starting to think it's a little irresponsible of both Amazon and GitHub to not react better to it. Clearly Amazon is capable of detecting when this happens as evidenced by their rapid response; when they see AWS keys published to GitHub, they should block the account, or at least prevent it from spinning up any new instances. They can then contact the owner over e-mail to…

Is it really on the web service companies to make sure you (the developer) don't do anything stupid? The article author could have avoided a lot of headache if he simply checked the repo before walking away. Red flags should have gone up as soon as he received the first weird Amazon messages, but he still didn't think to check his own work. The author calls this a "Data Breach" but in reality it was a "Data Leak".

There's a huge amount of value in a service that insulates you from mistakes like this. You really, really have to know what you're doing with EC2 because it is a loaded gun.

Personally, I don't like using things that are covered in spring loaded booby traps that require eternal hypervigilance in order to avoid nasty financial consequences. Certainly not when deploying something that would be right at home on something like Heroku.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#56

"As a senior developer with over 12 years of experience in the financial services industry, I didn't think it was possible that I could be the victim of a data breach." We're only one sentence in and I'm already very, very nervous about the author's hubris.

Right? Who spends 12 years in the software industry and doesn't actually test their work after doing shit?

Do you test the functionality of your git client or code editor? I personally generally expect them to work, as do most people.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#57

The most important part of this story seems to be glossed over – how did an IAM key that the author insists only had access to the Alexa service get used to spin up EC2 instances? Either the author is just mistaken/lying about that point, or there's another leak he doesn't know about, or there's a serious privilege escalation vulnerability in AWS.

Someone else in a thread above supposes that they actually published amazon master keys, rather than an AWS particular IAM role. The author of the post states they didn't even use AWS prior, so they are most likely not aware of such things.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#58

Earlier quoted context omitted.

Don't be so quick to dismiss this one. This is notable for several reasons, including (1) it wasn't his mistake, but rather that of a Visual Studio bug not following his instructions to make a private repo on GitHub; (2) the speed in which this happened (minutes), and (3) it has useful analysis into some sorely lacking functionality in AWS that lets this continue to happen.

I don't know. Even for private repos, it's bad practice to commit private keys to source control.

> Even for private repos, it's bad practice to commit private keys to source control.

Is it, though? Committing them to the same repository as one's code lives in, sure, but committing them to a separate production-deploy repo seems okay to me (although I'd much prefer that private repo never to hit a centralised service like GitHub).

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#59
post #28
post #19

Does Amazon actually end up charging you when this type of thing happens or do they eventually wipe the bill after you talk to support? If Amazon tried to charge me, I would definitely be in touch with the credit card company I have attached to the account. This is rather straightforward fraud/identity theft so you should be covered by the standard consumer protections most credit card's provide.

I accidentally posted AWS credentials to the AWS EC2 forum, and got > $10k (US) of charges very quickly. They removed all the charges immediately, without me pushing at all. They offered it.

That is good to hear, but it also means the headline is a little on the click-baity side. This is a bug that caused a lot of headaches and cost time to fix, not one that cost $6,500.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#60

This is becoming such a common story that I'm starting to think it's a little irresponsible of both Amazon and GitHub to not react better to it. Clearly Amazon is capable of detecting when this happens as evidenced by their rapid response; when they see AWS keys published to GitHub, they should block the account, or at least prevent it from spinning up any new instances. They can then contact the owner over e-mail to…

It would probably be easy for Amazon to offer an automatic key blacklisting feature....
Post reply on HN