Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

humankode.com

Post reply on HN
Print view View original

81–90 of 110 posts

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#81

Post by jongalloway2 »

I'm fairly certain this is a bug in the GitHub extension for Visual Studio (https://visualstudio.github.com/), not in Visual Studio itself.

Here's how the sync dialog appears in Visual Studio 2015 without the GitHub extension: http://i.imgur.com/EGjYziR.png

After installing the GitHub extension: http://i.imgur.com/shRBwXM.png

Try it yourself, then complete these fun, but completely optional activities:

1. Re-read all comments on this thread using your best Andy Rooney voice.

2. Wish that the author had comments on his blog.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#82

Post by voltagex_ »

tomschlick wrote: post #13
Analemma_ wrote: post #5

This is becoming such a common story that I'm starting to think it's a little irresponsible of both Amazon and GitHub to not react better to it. Clearly Amazon is capable of detecting when this happens as evidenced by their rapid response; when they see AWS keys published to GitHub, they should block the account, or at least prevent it from spinning up any new instances. They can then contact the owner over e-mail to…

> or at least prevent it from spinning up any new instances This is the answer I would like to see implemented. If they can detect the key they can put a freeze on the account so that only existing resources are available. They should also have a timeline of all created instances in every zone so you can see what has happened in the last x hours and kill all instances at once.

Two things would make controlling these situations easier:

* Allow me to see all instances in all regions from a single screen

* When accounts are first created for AWS, set up some sane default IAM policies and/or hire some UI designers to fix that godawful mess.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#83

Post by voltagex_ »

nailer wrote: post #64
merlincorey wrote: post #23

Earlier quoted context omitted.

Thankfully, the author realizes later what it they really did wrong here: > What could be done to prevent this? > Always test new version control GUIs before using them in the wild. There could be a bug that could expose your data. > Encrypt access keys in config files instead of just leaving them exposed in the config file. > Better yet, move access keys to a seperate config file, and exclude this from Git deploys w…

> Encrypt access keys in config files instead of just leaving them exposed in the config file. How do decrypt them when you app re-launches? How do you decrypt them automatically on every instance?

I've moved to using environment variables. I realise they have their own security challenges but at least I'm not accidentally going to git push the keys.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#84

Post by nathancahill »

zeveb wrote: post #80
curun1r wrote: post #74

Earlier quoted context omitted.

> Is it, though? Yes, it is. Period. Full Stop. Don't ever check access keys into any repo, public, private or even self-hosted. AWS needs to do a better job of making you realize that access keys are like 100-year-old sticks of dynamite and should be handled with an equal amount of care considering they can cause a similar amount of damage. To their credit, they basically noted this when they changed the way access…

> Don't ever check access keys into any repo, public, private or even self-hosted. So, how do you propose to handle version control of your exact production configuration? Do you think that recording one's production configuration is a valuable (or even useful) technique? Do you propose to have a single file with production configuration relying only on point-in-time backups for history? That gets us back to the bad…

I use Ansible Vault if it's imperative to store keys in a repo, and I've been really happy with it. StackExchange has something similar called Blackbox[0]. I remember reading about a Ruby gem too, but I can't find it now.

[0] https://github.com/StackExchange/blackbox

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#85

Post by jongalloway2 »

jongalloway2 wrote: post #81

I'm fairly certain this is a bug in the GitHub extension for Visual Studio ( https://visualstudio.github.com/ ), not in Visual Studio itself. Here's how the sync dialog appears in Visual Studio 2015 without the GitHub extension: http://i.imgur.com/EGjYziR.png After installing the GitHub extension: http://i.imgur.com/shRBwXM.png Try it yourself, then complete these fun, but completely optional activities: 1. Re-read a…

I logged an issue on the GitHub extension in case that's the problem: https://github.com/github/VisualStudio/issues/62

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#86

Post by mercora »

nailer wrote: post #56
tomphoolery wrote: post #36

Earlier quoted context omitted.

Right? Who spends 12 years in the software industry and doesn't actually test their work after doing shit?

Do you test the functionality of your git client or code editor? I personally generally expect them to work, as do most people.

On first use, i guess i have checked these parts one way or another. To assume things work the way you expect them might be the reason the bug was present in VS (if it ever actually was).

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#87

Post by curun1r »

zeveb wrote: post #80
curun1r wrote: post #74

Earlier quoted context omitted.

> Is it, though? Yes, it is. Period. Full Stop. Don't ever check access keys into any repo, public, private or even self-hosted. AWS needs to do a better job of making you realize that access keys are like 100-year-old sticks of dynamite and should be handled with an equal amount of care considering they can cause a similar amount of damage. To their credit, they basically noted this when they changed the way access…

> Don't ever check access keys into any repo, public, private or even self-hosted. So, how do you propose to handle version control of your exact production configuration? Do you think that recording one's production configuration is a valuable (or even useful) technique? Do you propose to have a single file with production configuration relying only on point-in-time backups for history? That gets us back to the bad…

> So, how do you propose to handle version control of your exact production configuration?

Version control is perfectly acceptable for that, but the key point is that access keys aren’t part of your production configuration. I personally don’t allow my team to have long-lived AWS access keys that have access to production. Production is only changed via an assumed role requiring MFA or an IAM instance role.

If this kind of setup is too complex to create on your own, solutions like Vault (https://vaultproject.io) can make it much easier to get right.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#88

Post by 15155 »

onion2k wrote: post #40
Alupis wrote: post #24

Earlier quoted context omitted.

Is it really on the web service companies to make sure you (the developer) don't do anything stupid? The article author could have avoided a lot of headache if he simply checked the repo before walking away. Red flags should have gone up as soon as he received the first weird Amazon messages, but he still didn't think to check his own work. The author calls this a "Data Breach" but in reality it was a "Data Leak".

Is it really on the web service companies to make sure you (the developer) don't do anything stupid? Yes. The web service company should look at it as basic security - if people are using their service to discover private information and use it to defraud their users, even if the users have been stupid enough to upload that information publicly, the company should do what they can to mitigate the problem. If I post a…

> mitigate the problem.

This seems like very minor engineering effort on behalf of GH - simple regexes, even.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#89

Post by balls187 »

curun1r wrote: post #74
zeveb wrote: post #58

Earlier quoted context omitted.

> Even for private repos, it's bad practice to commit private keys to source control. Is it, though? Committing them to the same repository as one's code lives in, sure, but committing them to a separate production-deploy repo seems okay to me (although I'd much prefer that private repo never to hit a centralised service like GitHub).

> Is it, though? Yes, it is. Period. Full Stop. Don't ever check access keys into any repo, public, private or even self-hosted. AWS needs to do a better job of making you realize that access keys are like 100-year-old sticks of dynamite and should be handled with an equal amount of care considering they can cause a similar amount of damage. To their credit, they basically noted this when they changed the way access…

> Yes, it is. Period. Full Stop. Don't ever check access keys into any repo, public, private or even self-hosted.

Generalizing here (and for the record I agree with your point), is that to a developer, source code is as valuable as anything else, and if a private repo is secure enough for source code, it should be secure enough for access credentials.

I speculate that this is a by product to moving towards a developer-centric engineering culture.

Re: How a bug in VS2015 exposed my source code on GitHub and cost me $6,500

#90

Post by ocdtrekkie »

JorgeGT wrote: post #55
ocdtrekkie wrote: post #2

I still find it amazing that AWS doesn't offer any way to hardcap your billing and cut off service.

Side question: is there any similar cloud service with hard billing caps?

I'd really love to know. Because if I could get AWS-like service with the ability to have it cut off if my bill gets to say... three times my usual monthly... That'd be worth switching for the peace of mind.
Post reply on HN