Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ruby deploys temporarily disabled

status.heroku.com

Post reply on HN
Print view View original

41–50 of 98 posts

Re: Ruby deploys temporarily disabled

#41

Post by Corrado »

mildavw wrote: post #16

Survey: Do you depend on RubyGems for every deploy? Or do you have your own gem server? Or cache them at some point earlier in your pipeline? We rely on RubyGems and had a meeting yesterday about changing that when one of the gems we use had a version just disappear.

I was looking for a RubyGems proxy a couple of weeks ago but was unable to find anything suitable. What I would like to find is something similar to Artifactory for Maven. You include the proxy in your Gemfile and if it doesn't have the Gem it downloads it from RubyGems and caches it locally.

This type of proxy wouldn't help in this particular case but it would allow you to keep traffic to RubyGems.org down and also give you the ability to easily host private Gems.

Re: Ruby deploys temporarily disabled

#42

Post by btown »

sergiotapia wrote: post #5

The DAY I manage to convince the big wigs where I work that we should switch from a typical shared environment to Heroku, this happens. Talk about luck. :( Hopefully I can spin this and not leave a bad taste in their mouths. We (engineers) understand what's happening, management doesn't and they don't give a shit.

"Thank goodness we switched to Heroku. Had we stayed on our previous environment, we would have been opened up to a security exploit without even knowing it."

Re: Ruby deploys temporarily disabled

#43

Post by redeemedfadi »

scotje wrote: post #29
mildavw wrote: post #16

Survey: Do you depend on RubyGems for every deploy? Or do you have your own gem server? Or cache them at some point earlier in your pipeline? We rely on RubyGems and had a meeting yesterday about changing that when one of the gems we use had a version just disappear.

If you use bundler, "bundle package" can help reduce or eliminate your dependency on external gem repositories. At least for deployments. I generally try to follow the "vendor everything" philosophy: http://ryan.mcgeary.org/2011/02/09/vendor-everything-still-a...

I couldn't get through this whole article due to the dude in the corner staring at me... :-/

Re: Ruby deploys temporarily disabled

#44

Post by Pewpewarrows »

simonw wrote: post #26
Pewpewarrows wrote: post #6

This should also be a reminder to everyone that you shouldn't be reliant on a single point of failure for your deploys. It's something that we in the Python community have already encountered (and hopefully learned from) due to the historical unreliability of our equivalent package repo, PyPI. Have an internal repo that's accessible by your deploy servers, which in turn locally caches anything that you might have pre…

The way we handle that for our Python deploys is to have a separate "deploy" git repo which includes complete .tar.gz files of all of our dependencies, then have our pip requirements.txt file point to those file paths rather than using external HTTP URLs. To avoid packages sneakily trying to download their own dependencies from the internet we run pip install with a "--proxy http://localhost:9999 argument (where noth…

We do something very similar, but like you said there are the occasional sneaky devils trying to download their own dependencies. Nine times out of ten it seems like it's some version of distribute that they insist on fetching.

The non-existant proxy trick seems useful, I'll have to try that out.

Re: Ruby deploys temporarily disabled

#45

Post by JonnieCache »

Pewpewarrows wrote: post #6

This should also be a reminder to everyone that you shouldn't be reliant on a single point of failure for your deploys. It's something that we in the Python community have already encountered (and hopefully learned from) due to the historical unreliability of our equivalent package repo, PyPI. Have an internal repo that's accessible by your deploy servers, which in turn locally caches anything that you might have pre…

    bundle package
puts all your app's dependencies in vendor/cache. That can then be put into a git submodule.

The problem then becomes the Gemfile and Gemfile.lock, which should really be in that submodule as well. You need to pass flags to bundler commands because it assumes the Gemfile is in the project root.

Re: Ruby deploys temporarily disabled

#46

Post by hadees »

martingordon wrote: post #15

Between this hack and the recent Rails vulnerabilities, it seems like a perfect storm. I wonder if either the hack attempted to tamper with the Rails gems to catch late updaters or to remove the ability to use RubyGems to update to the latest versions and keep vulnerable sites vulnerable.

I think this hack was related to the recent Rails vulnerability. Heroku is blaming this on a "YAML parsing vulnerability" which I think was the problem same issue with rails. I'm not sure if they are using rails or not but its surprising that if it's the same issue they didn't do anything about it before this happened.

Re: Ruby deploys temporarily disabled

#47

Post by steiza »

Pewpewarrows wrote: post #6

This should also be a reminder to everyone that you shouldn't be reliant on a single point of failure for your deploys. It's something that we in the Python community have already encountered (and hopefully learned from) due to the historical unreliability of our equivalent package repo, PyPI. Have an internal repo that's accessible by your deploy servers, which in turn locally caches anything that you might have pre…

I only recently realized how easy it was to run your own PyPI - it just has to handle a few HTTP GET / POSTs.

If you want to run your own PyPI internally, here's a very simple PyPI server (~150 lines of Python) that I wrote: https://github.com/steiza/simplepypi

Re: Ruby deploys temporarily disabled

#48

Post by [deleted] »

martingordon wrote: post #15

Between this hack and the recent Rails vulnerabilities, it seems like a perfect storm. I wonder if either the hack attempted to tamper with the Rails gems to catch late updaters or to remove the ability to use RubyGems to update to the latest versions and keep vulnerable sites vulnerable.

[deleted]

Re: Ruby deploys temporarily disabled

#49

Post by cobrabyte »

pardner wrote: post #24
sergiotapia wrote: post #5

The DAY I manage to convince the big wigs where I work that we should switch from a typical shared environment to Heroku, this happens. Talk about luck. :( Hopefully I can spin this and not leave a bad taste in their mouths. We (engineers) understand what's happening, management doesn't and they don't give a shit.

From a manager perspective, this sort of alert (as well as recent Heroku emails telling you specifically which app(s) needed patching for the recent Rails CVE issues) is a great example of one of the extra benefits of Heroku. A team of engineers 'watching your back' at no extra charge is a good thing.

Agreed. I was surprised when I received an email from Heroku letting me know that a few of my apps needed to be updated after the Rails vulnerabilities were uncovered. They also named the apps that needed to be updated, which makes my job that much simpler.

Re: Ruby deploys temporarily disabled

#50

Post by jim-greer »

themgt wrote: post #35
rmoriz wrote: post #28

Earlier quoted context omitted.

The central architecture of rubygems allows you to publish and yank gems within minutes. CPAN takes some hours (and deletion may not be controlled). Personally I'm a big fan of the CPAN approach as it is fairly simply. Just mirror via FTP. It's a nobrainer to setup and run a mirror. That said, CPAN's master (PAUSE.cpan.org) is a SPOF as well. What I like is that not a single party is responsible for paying server bil…

Agreed. We've looked into running our own mirrors for rubygems and it's there's nothing really supported out there. The addition of git gems in bundler means you'd really need a git mirror tool as well.

At Kongregate we just check the gems into our repo under vendor/cache and then run 'bundler install --local' at deploy time.

Here's the docs that describe it:

"While installing gems, Bundler will check vendor/cache and then your system's gems."

http://gembundler.com/v1.2/bundle_install.html

Post reply on HN