Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thank You Heroku, or "How To Eliminate Sysadminning"

blog.dougpetkanics.com

Post reply on HN
Print view View original

51–60 of 66 posts

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#51

Post by thegoleffect »

njl wrote: post #25
jokull wrote: post #12

Earlier quoted context omitted.

No. There are a number of some pretty serious restrictions. I would _love_ something like Heroku for WSGI applications. Ian Bicking is working on something similar, not as high level with his Silver Lining project. http://bitbucket.org/ianb/silverlining/

A couple of months ago I explored building a Django-friendly Heroku clone. I white-boarded out the architecture, estimated how much work would be needed for all the pieces, and built a spreadsheet revenue model. It's an interesting and attractive business, but I'm only one man and my C skills are rusty (really, really rusty). I couldn't get myself to a point where I could deliver a product for at least a year, and I…

I ended up writing a lot of similar functionality for the higut.com team. It was setup to auto deploy Tornado apps instead of Django but its probably not too much of a stretch to do WSGI in general. Didn't know there'd be interest. I could factor it out and release it?

It did deploy on commit/push, bundled eggs, worked with mysql or sqlite and redis. Dunno if that sounds interesting to others though.

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#52

Post by qeorge »

xal wrote: post #36
freetard wrote: post #10

I tried installing spree the other day, a rails shopping app, and I gave up after entering gem dependency hell and the no local file policy. Went faster on my own server with nginx+passenger.

You cannot be PCI DSS compliant on Heroku unless you only use paypal express or google checkout for your payment backend. Be mindful of that before you commit to something like spree. I recommend Shopify instead :D

This doesn't sound right to me. Do you perhaps mean PCI Level 1 compliance (which requires on site data center audits and thus cannot be provided by any cloud host)?

For instance, what about Heroku/EC2 prohibits me from using Authorize.net as my payment processor, provided I never store card data?

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#53

Post by andrewvc »

bphogan wrote: post #28
bgnm2000 wrote: post #16

Earlier quoted context omitted.

Exact same experience yesterday. Weird.

You will. Heroku has three big limitations that can make your app a no-go there. The first is only hourly cron tasks which means doing sweeping every 15 minutes is not possible. Not terrible, but sometimes a showstopper if you're trying to do background tasks frequently. The second is that it's a read-only filesystem which means you either don't use the filesystem for your work, or you use S3. S3 is great, but is ano…

If you need to perform background tasks more frequently you can always spawn a worker process.

I find that the read only FS is a good thing, decoupling storage from app servers really helps wrt to scaling.

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#54

Post by jackowayed »

hopeless wrote: post #40
blasdel wrote: post #30

Earlier quoted context omitted.

Though they just fucked it up in the betas by depending on rubygems-1.3.6, it's not so hard to get it running on Heroku. You run the rake tasks to copy the public directories from extensions to the root one locally before pushing, you pre-cache the generated CSS (not a problem in the betas, they eliminated sass), and setup asset image uploads to go to S3. The real problem with hosting Spree on Heroku is having to spe…

The SSL thing is really annoying and expensive but I think there's sufficient demand/pressure that Heroku will find a better alternative... possibly with the help of Amazon.

They really need to. If the thing you're doing is important enough that you need SSL (and a custom domain), you have to be pretty big for SSL not to be prohibitively expensive.

SSL alone is as expensive as a 1.5 GB slice from slicehost. I know it's not really their fault, but they need to work with amazon to get several IPs for each of their images, or something.

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#55

Post by njl »

thegoleffect wrote: post #51
njl wrote: post #25

Earlier quoted context omitted.

A couple of months ago I explored building a Django-friendly Heroku clone. I white-boarded out the architecture, estimated how much work would be needed for all the pieces, and built a spreadsheet revenue model. It's an interesting and attractive business, but I'm only one man and my C skills are rusty (really, really rusty). I couldn't get myself to a point where I could deliver a product for at least a year, and I…

I ended up writing a lot of similar functionality for the higut.com team. It was setup to auto deploy Tornado apps instead of Django but its probably not too much of a stretch to do WSGI in general. Didn't know there'd be interest. I could factor it out and release it? It did deploy on commit/push, bundled eggs, worked with mysql or sqlite and redis. Dunno if that sounds interesting to others though.

If you haven't had a chance to mull over Heroku's architecture, I'd suggest giving it a look. http://heroku.com/how/architecture

Basically, nginx fronts a bunch of Varnish caches, which are then in front of a "routing mesh", which queues up requests for individual "dynos". A dyno is a single-threaded instance of your application. When one dies or is migrated, another is deployed in its place. Multiple dynos are automatically spread across multiple machines. It's a very neat solution to automatically scaling hundreds or thousands of apps.

In other words, bundling the apps and pushing them out is just the beginning of a pretty fascinating architecture.

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#56

Post by jhancock »

hopeless wrote: post #42
dasil003 wrote: post #22

I'm intrigued by Heroku (and EY Cloud), but the thing that really freaks me out is how they charge for every little extra thing. I love the idea of a turnkey solution for Rails deployment and cloud scalability, but for me personally I don't think I would do a startup on it because the value proposition is squeezed from both sides. On the low-end I can get a VPS for $10-$20 month and get a Rails app up and running for…

Even though I had a VPS configured and running for almost a year (i.e., all the upfront sysadmin setup done) I was stressing about what I hadn't done, or more precisely what I didn't know I needed to do. Forgetting to update the OS, check if the machine had been compromised, check the load/mem/disk space etc. Just a hundred little worries floating around inside my head. So I moved to Heroku to focus purely on the app…

For me, not owning my own server as a serial entrepreneur is like a graphics design consultant not owning a mac. Or a mechanic without four extra cars in his backyard.

I'm sure Heoku is a great deal if you don't have the sysadmin skills. My skills are just good enough and I manage my own box. I bought it 5 years ago..4 cores, 8 GBs, 73GB RAID-1...for $5000. I pay $105/month to host it.

This is at least the equivalent in horsepower of what you get from Heroku for $500+ a month (that's being generous). So in a few years, I'm ahead of the curve: $11,300 vs $30,000. Add in the fact that I paid down $5000 in advance as a capital investment and I'm still ahead. You could do this today for under $3000 capital investment and have an even more capable box.

Sure there's risk I'll have a hardware failure (I did have 3 years 24 hours on site warranty, but that's still potential downtime and headaches). But there is also risk that over the course of that time you will want to run things on your server you did not anticipate when you started. It certainly was the case for me. e.g. I started my app in rails then switched to merb. Now I run a second site on the same server with mongodb and solr.

Bottom line, if you have moderate linux skills and know your going to be in "some business" that requires you have a server, buy your own. If not, Heorku seems a fine choice for some.

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#57

Post by hachiya »

jhancock wrote: post #56
hopeless wrote: post #42

Earlier quoted context omitted.

Even though I had a VPS configured and running for almost a year (i.e., all the upfront sysadmin setup done) I was stressing about what I hadn't done, or more precisely what I didn't know I needed to do. Forgetting to update the OS, check if the machine had been compromised, check the load/mem/disk space etc. Just a hundred little worries floating around inside my head. So I moved to Heroku to focus purely on the app…

For me, not owning my own server as a serial entrepreneur is like a graphics design consultant not owning a mac. Or a mechanic without four extra cars in his backyard. I'm sure Heoku is a great deal if you don't have the sysadmin skills. My skills are just good enough and I manage my own box. I bought it 5 years ago..4 cores, 8 GBs, 73GB RAID-1...for $5000. I pay $105/month to host it. This is at least the equivalent…

Thanks for this comparison.

Can you recommend the company that you colocate/lease-to-own with?

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#58

Post by jhancock »

hachiya wrote: post #57
jhancock wrote: post #56

Earlier quoted context omitted.

For me, not owning my own server as a serial entrepreneur is like a graphics design consultant not owning a mac. Or a mechanic without four extra cars in his backyard. I'm sure Heoku is a great deal if you don't have the sysadmin skills. My skills are just good enough and I manage my own box. I bought it 5 years ago..4 cores, 8 GBs, 73GB RAID-1...for $5000. I pay $105/month to host it. This is at least the equivalent…

Thanks for this comparison. Can you recommend the company that you colocate/lease-to-own with?

http://m5hosting.com. They're in a solid data center in San Diego. I started with them for two reasons: 1- they knew FreeBSD (I use ubuntu now but wanted someone who could hands on cover me in a pinch with FreeBSD at the time). 2 - I was in Shanghai at the time and needed good bandwidth from the U.S. That data center has better routing than most.

m5hosting is bigger than when I started with them but they are still a "medium-sized" company. They seem to be solid and whenever I have a support request they are on it quickly...about half the time I get a response from the owner.

I generally follow the rule of picking business partners that are "right-sized" for me as a customer. For many services I don't want a company so big that I'm not worth their time after the sale...especially when it comes to server support.

I have never been to the data center and never seen my own server. I bought it on ebay and had it shipped to them ;).

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#59

Post by _pius »

donw wrote: post #43
moe wrote: post #17

Earlier quoted context omitted.

In a perfect world heroku would be the ideal "pay later" option. I.e. you start out with them and once you outgrow their platform you simply hire the sysadmin then , and make him transplant the beautifully clean rails app onto your own infrastructure. Except... reality is harsh and ugly. You want the guy with the sysadmin foo on your team from as early as possible. Because if your thinking goes along the lines you ju…

I'm a sysadmin-turned-startup-founder, so there's my bias right there, plain for all to see. With that, this is really, really good advice. In sysadmin-land, all my pain comes from software that was never written with management in mind. Assumptions like 'all TCP ports are open, all the time, between all servers', 'we can put things wherever we want in the filesystem', and 'the server should be configured just like m…

In sysadmin-land, all my pain comes from software that was never written with management in mind. Assumptions like 'all TCP ports are open, all the time, between all servers', 'we can put things wherever we want in the filesystem' ...

One good thing about Heroku, though, is that you essentially have to do a lot of these things properly from the beginning because they enforce a shared nothing, tiered architecture. For instance, you can't write to the filesystem, you can't set up some poorly secured e-mail server, you can't have arbitrary daemons running. In general, every process that isn't core to the web stack must be executed in a completely different tier.

Obviously you can still shoot yourself in the foot with poorly written SQL and other things, but a lot of the "sys-admin" type mistakes are avoided by enforcing best practices when you start with Heroku.

Re: Thank You Heroku, or "How To Eliminate Sysadminning"

#60

Post by thegoleffect »

njl wrote: post #55
thegoleffect wrote: post #51

Earlier quoted context omitted.

I ended up writing a lot of similar functionality for the higut.com team. It was setup to auto deploy Tornado apps instead of Django but its probably not too much of a stretch to do WSGI in general. Didn't know there'd be interest. I could factor it out and release it? It did deploy on commit/push, bundled eggs, worked with mysql or sqlite and redis. Dunno if that sounds interesting to others though.

If you haven't had a chance to mull over Heroku's architecture, I'd suggest giving it a look. http://heroku.com/how/architecture Basically, nginx fronts a bunch of Varnish caches, which are then in front of a "routing mesh", which queues up requests for individual "dynos". A dyno is a single-threaded instance of your application. When one dies or is migrated, another is deployed in its place. Multiple dynos are autom…

Yeah, I meant those were additional features. We used a similar setup on Rackspace Cloud, though not as refined (or as secure imo). Tornado really lends itself well to a dyno type arrangement. We used nginx for load balancing and automatic frontend resolution, probably in a similar way as Heroku. Thanks for the tip though, interesting read.

Talking to some other Pythonistas, there didn't seem to be much desire for a Heroku-equivalent system though - most ppl enjoy rolling their own deployments.

Post reply on HN