Live data from Hacker News

How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

humankode.com

21–30 of 110 posts

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#21
> As a senior developer with over 12 years of experience in the financial services industry, I didn't think it was possible that I could be the victim of a data breach.

Ok...

> Better yet, move access keys to a seperate config file, and exclude this from Git deploys with a .gitignore.

No shit?

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#22
Hopefully he posts updates about his unanswered questions. Since it sounds like the compromised key should not have been able to create ec2 instances.

And I don't blame him for making that mistake. That kind of bug should have been caught in QA.

On the other hand, I'm paranoid enough about things that I wouldn't push unsecured access keys to any server I didn't control. Even if I was paying for a private repo.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#23

"As a senior developer with over 12 years of experience in the financial services industry, I didn't think it was possible that I could be the victim of a data breach." We're only one sentence in and I'm already very, very nervous about the author's hubris.

Thankfully, the author realizes later what it they really did wrong here:

> What could be done to prevent this?

> Always test new version control GUIs before using them in the wild. There could be a bug that could expose your data.

> Encrypt access keys in config files instead of just leaving them exposed in the config file.

> Better yet, move access keys to a seperate config file, and exclude this from Git deploys with a .gitignore.

The fact this supposedly security experienced and conscious developer thought it was okay to have any kind of access credentials stored in a git repository is glaring.

You should consider all data in any repository public for security reasons. Publishing to github publicly made it quite obvious - but he could have been just as screwed over by people he gave access to the private repositories and people who have broken into github and can directly access the repositories.

Sure the numbers on both those groups are lower than the numbers of people apparently scanning public github for access keys - but the severity of the risk of keeping access keys and other secrets in your revision control are the same.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#24

This is becoming such a common story that I'm starting to think it's a little irresponsible of both Amazon and GitHub to not react better to it. Clearly Amazon is capable of detecting when this happens as evidenced by their rapid response; when they see AWS keys published to GitHub, they should block the account, or at least prevent it from spinning up any new instances. They can then contact the owner over e-mail to…

Is it really on the web service companies to make sure you (the developer) don't do anything stupid?

The article author could have avoided a lot of headache if he simply checked the repo before walking away. Red flags should have gone up as soon as he received the first weird Amazon messages, but he still didn't think to check his own work.

The author calls this a "Data Breach" but in reality it was a "Data Leak".

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#25
post #10

This is becoming such a common story that I'm starting to think it's a little irresponsible of both Amazon and GitHub to not react better to it. Clearly Amazon is capable of detecting when this happens as evidenced by their rapid response; when they see AWS keys published to GitHub, they should block the account, or at least prevent it from spinning up any new instances. They can then contact the owner over e-mail to…

AWS do do things. They have service limits and routinely waive charges from abuse events like this. I do wonder if GitHub's events firehose has benefits that outweigh all the compromised credentials, though. Not just limited to AWS keys.

> They have service limits and routinely waive charges from abuse events like this.

That's almost worse though. Maybe I'm being unfair here, but the fact that they will waive charges immediately if you ask, but don't just block them pre-emptively, tells me that they're hoping at least some customers (with deep pockets and automated billing that no one pays attention to) will just pay the bill and never notice. Maybe they're making more money from that than they're losing by waiving charges on compromised accounts for customers who ask.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#26

"As a senior developer with over 12 years of experience in the financial services industry, I didn't think it was possible that I could be the victim of a data breach." We're only one sentence in and I'm already very, very nervous about the author's hubris.

Why? It sounded like he is very security conscious. Thought he was creating a private repo in VS, immediately after he was notified deleted and recreated his keys. A pretty spot on course of action I think.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#27

"As a senior developer with over 12 years of experience in the financial services industry, I didn't think it was possible that I could be the victim of a data breach." We're only one sentence in and I'm already very, very nervous about the author's hubris.

I took it less as hubris and more of a way to imply to readers that no matter their experience they are not immune to the mistake he is going to tell you about.

I think it's definitely some hubris, because the mistake he made wasn't publishing to github publicly by accident. It was thinking a git repository was a good place to store secrets.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#28
post #19

Does Amazon actually end up charging you when this type of thing happens or do they eventually wipe the bill after you talk to support? If Amazon tried to charge me, I would definitely be in touch with the credit card company I have attached to the account. This is rather straightforward fraud/identity theft so you should be covered by the standard consumer protections most credit card's provide.

I accidentally posted AWS credentials to the AWS EC2 forum, and got > $10k (US) of charges very quickly. They removed all the charges immediately, without me pushing at all. They offered it.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#30
post #26

"As a senior developer with over 12 years of experience in the financial services industry, I didn't think it was possible that I could be the victim of a data breach." We're only one sentence in and I'm already very, very nervous about the author's hubris.

Why? It sounded like he is very security conscious. Thought he was creating a private repo in VS, immediately after he was notified deleted and recreated his keys. A pretty spot on course of action I think.

Because anyone who ever says "I didn't think it was possible that I could be the victim of a data breach" is dangerously delusional, and in case I'm not being clear - wrong.

Also putting keys in (even a private) cloud-hosted repos is at least a step under what I'd describe as "very security conscious" for someone who self describes as working in the financial industry.

Post reply on HN