Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How and Why Swiftype Moved from EC2 to Real Hardware

highscalability.com

Post reply on HN
Print view View original

91–100 of 194 posts

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#91

Post by jzawodn »

ChuckMcM wrote: post #79
rodgerd wrote: post #61

Earlier quoted context omitted.

> There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing: I wish it were true, but plenty of companies are gripped by cloud fever. I've seen quite a few going down the route of charging into the cloud not because they've run the numbers and found it stacks up, but because they want to be in the cloud, and Amazon have some great marketing people.

It is interesting coming at it from the other side, I get hit up all the time to move my infrastructure to the cloud, I give them the specs and what I pay per month, tell them that if they can beat it I'll switch. The closest anyone has come has been about 4x the cost I'm currently paying. But I note that I have a unique case that isn't covered by the cloud "architecture" (crawling the web and indexing it). Explainin…

Yeah, it's always fun when I tell someone at we're 100% self-hosted too. They never stop to think that traffic, load, and usage patterns on craigslist are pretty well understood at this point. Plus it's certainly cheaper than paying someone else to run all your hardware. You get to pick your peers, pick your hardware, and allocate as you see fit.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#92

Post by karamazov »

skwirl wrote: post #69
snoopybbt wrote: post #43

There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing: Cloud computing offers you the great and awesome advantages of being able to instantly scale your application, replicate your data and basically just grow according to your business volume, and all this without significant investments, delivery time, setup time, people time, maintenance or anything bu…

>you're also paying Amazon's profits. Are you? Amazon is not known for posting profits, and is quite possibly operating AWS at a loss. They have promised to show financial results for AWS itself with the quarterly report being released next month, so we'll see.

If AWS posts a loss it'll be because Amazon is reinvesting resources into its growth, not because they're subsidizing each user's costs.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#93

Post by Someone1234 »

sz4kerto wrote: post #76
Someone1234 wrote: post #57

Earlier quoted context omitted.

I see no explanation for why suppliers wouldn't match demand. Aside from the HDD shortage which hit everyone, I've seen no issues like you're describing where essentially the market runs out of servers/CPUs/etc. Because that's the only way this theory applies, if they're completely unable to meet the demand due to some specific shortage in the market. The only reason your shares analogy makes sense is because there A…

No, they can't scale demand that quickly. And time matters. Also, there's no situation where demand can't be met (well, extremely rarely it happens, in case of inelastic goods). What happens is you increase prices until the demand lowers to match the supply. You never run out of products. Similarly, oil supplies will last forever, we won't ever run out of fossil fuels. Simply it will not be worth using them because o…

> Really, microeconomics 101, all of us should have studied this in the first semester of any engineering degree :)

That seems like quite a snide remark.

I have in fact studied "economics 101," and while you're using basic economic theory to form your opinions, you're mixing that in with data which you've just created for the sake of supporting your original point.

Essentially you have no supportable reason to assume that supply cannot meet demand OR that Amazon cannot space out their demand/pre-warn the supply chain. Amazon could, for all we know, give them a 12 months lead time.

To be honest this entire conversation reminds me of that scene in Good Will Hunting when the guy in the bar is mouthing off about "market economy in the early colonies" because he just finished studying them last semester. Reading your posts comes across like you're trying to shoehorn in as much eco 101 knowledge as you can. And rather than provide data or any meaningful explanation for why you believe the market would go a certain way, you just shove in more econ 101 theory and hope for the best.

This post in particular lacks any substance, and is just trying to impress upon us how much econ 101 you know. But really I am more interested in why you believe the market wouldn't meet demand, rather than how many buzz words and theory names you can reproduce from your textbook.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#94

Post by chx »

bobofettfett wrote: post #33

My 5cent: If you have lots and lots of money and a high margin business, do yourself a favor and go with Amazon (much less hassle with contract management and low level challenges). If you need to scale month to month and are growing 50% per month, go with Amazon. If you are very small and can live with 10 instances, go with Amazon. If CAPEX doesn't help you and for whatever reasons you need to spend OPEX, go with Am…

> If you need to scale month to month and are growing 50% per month,

Then rent more servers.

> If you are very small and can live with 10 instances,

Then rent a few servers.

I maintain that there are extremely few cases for a typical website to use the cloud. To handle peaks, it is both simpler and cheaper to keep enough capacity just idling around than spinning up and down Amazon instances. The cloud is almost always a useless hype. It can be different if you can architect to use the various services Amazon provides.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#95

Post by deitcher »

josh2600 wrote: post #15
toomuchtodo wrote: post #12

Earlier quoted context omitted.

> In my experience, I've never found an issue with an application on AWS that wasn't caused by either a misunderstanding of what was being offered (e.g. not provisioning enough PIOPS for database volumes), or simply issues with the application code. You haven't been using Amazon long enough then. Amazon is great for proof of concept. No upfront costs, extremely scalable, etc. Unfortunately, its expensive compared to…

You can use abstraction layers to isolate yourself from issues with the underlying metal. For example, I had a good thread the last time the maintenance reboots happened: https://news.ycombinator.com/item?id=9120289 . Solving multi-tenancy issues is hard, but not impossible. I think it's a lot easier with live migration. If a box is giving you problems, just move the load to a new box while maintaining the same IP ad…

That is what I like about the cloud. Running in your own hardware lets you be, well, "lazy" about application architecture. Running in a place where it is shared and instances can disappear forces you to design a lot more robustly and nimbly.

Of course, that design discipline is great wherever you are running....

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#96

Post by vanessa98 »

ori_b wrote: post #86
kovyrin wrote: post #66

Earlier quoted context omitted.

If only it were that simple... But every second relatively young engineer I interview points out that "But Netflix! Look how good the cloud works for them!" and once again I need to explain, that Netflix spends millions in engineering resources cost to handle EC2 issues and we are not (almost nobody is) Netflix (yet?).

Netflix also runs many of their own servers, and puts their own racks into many data centers. https://openconnect.itp.netflix.com/hardware/index.html https://openconnect.itp.netflix.com/peeringLocations/index.h...

Amazon also runs Netflix racks

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#97

Post by toomuchtodo »

deitcher wrote: post #95
josh2600 wrote: post #15

Earlier quoted context omitted.

You can use abstraction layers to isolate yourself from issues with the underlying metal. For example, I had a good thread the last time the maintenance reboots happened: https://news.ycombinator.com/item?id=9120289 . Solving multi-tenancy issues is hard, but not impossible. I think it's a lot easier with live migration. If a box is giving you problems, just move the load to a new box while maintaining the same IP ad…

That is what I like about the cloud. Running in your own hardware lets you be, well, "lazy" about application architecture. Running in a place where it is shared and instances can disappear forces you to design a lot more robustly and nimbly. Of course, that design discipline is great wherever you are running....

Hardware is almost always cheaper than engineering time. Don't optimize prematurely.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#98

Post by kovyrin »

chx wrote: post #89
lucas_alwin wrote: post #87

Here's my anecdotal, one-data-point experience from moving a giant EC2 environment to datacenter: 1. Your operational overhead will increase _a lot_. Be ready to hire on a lot of ops staff if you expect them to do anything but put out fires. And as you grow you'll need experts, people like network engineers. 2. Any weirdness you experienced with AWS infrastructure will be replaced with weirdness in your own environme…

> Your operational overhead will increase _a lot_. Be ready to hire on a lot of ops staff if you expect them to do anything but put out fires. And as you grow you'll need experts, people like network engineers. Why? What are you talking about? You are hiring servers you are no colo'ing them. Networking them is not your problem. Your responsibility still starts from a root prompt just there's no VM layer between that…

Yes, it really does look like @stephen-mw is talking about a migration to an owned/colocated hardware. And that's a very different kind of hairy mess I'd prefer to stay away from as long as humanly possible.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#99

Post by babo »

babo wrote: post #53
Thaxll wrote: post #40

Earlier quoted context omitted.

Netflix uses thousands of instances, they don't share servers.

Using thousands of servers doesn't mean that you don't share physical servers.

Even if you own all the VMs on a physical server it's hard to avoid the noisy neighbour problem. Netflix has its own instance monitoring tool - Vector - to handle these issues: http://techblog.netflix.com/2015/02/a-microscope-on-microser...

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#100

Post by kovyrin »

babo wrote: post #72
kovyrin wrote: post #59

Earlier quoted context omitted.

Oh, there wasn't a handful of servers after we finished the migration (we have migrated a bit late IMO, so we had a lot of traffic even back then). And today, with much larger infrastructure, with hardware clusters specifically tailored to our customers needs, etc I'm pretty sure the same infrastructure on EC2 would cost more than 2x. (Update) Re: failures - with a ~50 servers we see a hardware issue (disk dead in a…

How do you monitor HW and network failures and how do you notify SoftLayer? Is that 1-2 hours replacement time true for each components of your server fleet?

1-2 hours is their new server provisioning time. For HW issues we use nagios (that checks raid health and ECC memory health regularly) and at the moment we just file a ticket with SL about the issue showing them the output from our monitoring. They react within an hour and HW replacement is usually performed within an few hours after that (usually limited by our ability to quickly move our load away from a box to let them work on it).
Post reply on HN