Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

forbes.com

Post reply on HN
Print view View original

91–100 of 128 posts

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#91

Post by tsmith »

3am wrote: post #25

I hate cloud headlines. What's the hypervisor? VMWare is a hypervisor (ESX) and management applications (vCenter/vSphere, vCloud Director, etc). Openstack is only the management applications. And VMWare contributes the most interesting part of OpenStack, anyway, which is the virtual networking based on Nicira's OpenFlow. My personal experience with OpenStack (admittedly late Diablo release timeframe) was that it was…

Almost definitely KVM. The Ubuntu/KVM/OpenStack combo is the bread-and-butter deployment scenario outside of a small number of folks (notably Rackspace, which is OpenStack on XenServer^TM).

Also, Source: knowing these things is my job.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#92

Post by tsmith »

hp50g wrote: post #26

I think this only solves the cost problem. VMware is damn expensive for what it is. I'm not really a fan of virtualization myself. We canned it at our organisation a couple of years ago. We had 30 ESX/vSphere managed hosts across 3 data centers which hosted app servers, web front end servers, virtual load balancers and some other minor infrastructure. The final straw was the upgrade costs. Not only that we had proble…

> I can understand that virtualization is useful when your resource requirement is less than one machine but above that, I doubt there are any real benefits. It's snake oil.

Tell that to Netflix - or any of AWS' customers, really.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#93

Post by hp50g »

tsmith wrote: post #92
hp50g wrote: post #26

I think this only solves the cost problem. VMware is damn expensive for what it is. I'm not really a fan of virtualization myself. We canned it at our organisation a couple of years ago. We had 30 ESX/vSphere managed hosts across 3 data centers which hosted app servers, web front end servers, virtual load balancers and some other minor infrastructure. The final straw was the upgrade costs. Not only that we had proble…

> I can understand that virtualization is useful when your resource requirement is less than one machine but above that, I doubt there are any real benefits. It's snake oil. Tell that to Netflix - or any of AWS' customers, really.

AWS is a prime example. EC2 unreliable, doesn't perform well and is expensive (and fairly weather dependent by the looks too). If you want anything that can actually shift anything, dedicated servers are cheaper (but you might actually have to commit - oh dear if your margin is that tight, you don't have a product worth it).

The only use case I can see for it really is rapid scaling but that seems only viable for content delivery as your data back end architecture is way harder to scale than click the magic cloud button. Back in the old days we used to do content via CDN (Akamai etc) which actually works out cheaper per GiB.

Then we approach things like S3 which is terribly unreliable (some of our clients use it for storage as it's cheaper than a SAN but they suffer for that). Drop outs, unreliable network requests, latency, rubbish throughput and buggy as fuck SDK.

Its like consuming value line products from a supermarket. Sure there's quantity but its lacking in quality.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#94

Post by 3am »

tsmith wrote: post #91
3am wrote: post #25

I hate cloud headlines. What's the hypervisor? VMWare is a hypervisor (ESX) and management applications (vCenter/vSphere, vCloud Director, etc). Openstack is only the management applications. And VMWare contributes the most interesting part of OpenStack, anyway, which is the virtual networking based on Nicira's OpenFlow. My personal experience with OpenStack (admittedly late Diablo release timeframe) was that it was…

Almost definitely KVM. The Ubuntu/KVM/OpenStack combo is the bread-and-butter deployment scenario outside of a small number of folks (notably Rackspace, which is OpenStack on XenServer^TM). Also, Source: knowing these things is my job.

Many thanks for the info. I had a hell of a time trying^H^H^H^H^H^Hfailing to get Diablo working with XenServer, too. No ill will towards the project, and glad to hear it's maturing. It seemed to be in relatively good hands with Rackspace.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#95

Post by riotingpscifis »

hp50g wrote: post #93
tsmith wrote: post #92

Earlier quoted context omitted.

> I can understand that virtualization is useful when your resource requirement is less than one machine but above that, I doubt there are any real benefits. It's snake oil. Tell that to Netflix - or any of AWS' customers, really.

AWS is a prime example. EC2 unreliable, doesn't perform well and is expensive (and fairly weather dependent by the looks too). If you want anything that can actually shift anything, dedicated servers are cheaper (but you might actually have to commit - oh dear if your margin is that tight, you don't have a product worth it). The only use case I can see for it really is rapid scaling but that seems only viable for con…

If you setup you're stack wrong then some of you're points are valid. But many amazon services offer you redundant infrastructure at a reasonable cost.

But the key advantage over traditional hosted solution is that you can run reproducible test and dev stacks for 8/12 hours a day without having to pay for hardware that sits there doing nothing while you sleep. Sure cloud computing has it's faults but the ability to run you a HA test stack in 3/4 datacenters in minutes, and pay for just the time you are testing it, enables people to move forward and develop more and more impressive technology.

OTOH I'm not sure why you would run cloud on your own hardware as you're already paying for the HW, I suppose it simplifies the management significantly for PayPal or they wouldn't be doing it

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#96

Post by eksith »

rektide wrote: post #70
sc68cal wrote: post #38

Earlier quoted context omitted.

You can also use Vagrant+Devstack[1] to get started even quicker. https://github.com/bcwaldon/vagrant_devstack

So, what, now you can virtualize while your virtualize? I thought the point of OpenStack was to be a way to manage VMs, isn't that also much of what Vagrant does? I'd tend to avoid mixing the streams & go for a single node management system rather than attempt two at once- were it me. But I also am not invested in Vagrant, which people are free to enjoy, so if they can happily run both VM managers at once, so be it.…

  >So, what, now you can virtualize while your virtualize?
Yo dawg... yes (in a way). You can have issues with the environment as is the case with most new things, but this way there's nothing you need to keep should you decide to start fresh or decide to do something differently. It simplifies things a whole lot.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#97

Post by acdha »

druiid wrote: post #81
acdha wrote: post #73

Earlier quoted context omitted.

Can't speak for the parent poster's company but the numbers don't match my experience with VMware many years back. It's possible they've had a sharp regression but we were maxing out gigabit ethernet and local RAID arrays in 2006.

Well, don't confuse I/O with throughput here. You can look at performance numbers for just about anything and tweak one direction or the other. For instance it's easy to make a benchmark showing huge throughput to any given storage solution (and many NAS providers sell on this basis), but your I/O might be terrible because to get that throughput you're maxing the CPU (etc.). Likewise, you can change your benchmark an…

The parent very clearly specified I/O, which is what I was commenting on.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#98

Post by acdha »

hp50g wrote: post #77
acdha wrote: post #72

Earlier quoted context omitted.

> The 20-30% hit to IO If you're seeing that, something is configured wrong. VMWare was 95+% of native disk and gigabit ethernet 5 years ago.

Unless you are doing really heavy IO or CPU across multiple VMs on the same host which is likely if you have any load worth mentioning. There is a 20% to 30% difference if you run one process per VM or 8 processes on bare metal in an 8 core machine. We benchmarked it on known good configs optimised to bits. Either the hypervisor scheduler is shit or the abstraction is costly. I reckon its down to the reduction in CPU…

> Unless you are doing really heavy IO or CPU across multiple VMs on the same host which is likely if you have any load worth mentioning. There is a 20% to 30% difference if you run one process per VM or 8 processes on bare metal in an 8 core machine. We benchmarked it on known good configs optimised to bits. > > Either the hypervisor scheduler is shit or the abstraction is costly. I reckon its down to the reduction in CPU cache availability and the IO mux.

We were running heavy loads and there was nothing like a 20-30% hit. I'm not saying you didn't see one but this isn't magic or a black box: we had a few spots where we needed to tune (e.g. configuring our virtual networking to distribute traffic across all of the host NICs) but it performed very similarly to the bare metal benchmarks in equivalent configs.

What precisely was slower, anyway - network, local disk, SAN? Each of those have significant potential confounds for benchmarking.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#99

Post by Peaker »

hp50g wrote: post #26

I think this only solves the cost problem. VMware is damn expensive for what it is. I'm not really a fan of virtualization myself. We canned it at our organisation a couple of years ago. We had 30 ESX/vSphere managed hosts across 3 data centers which hosted app servers, web front end servers, virtual load balancers and some other minor infrastructure. The final straw was the upgrade costs. Not only that we had proble…

What was your SAN?

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#100

Post by [deleted] »

easytiger wrote: post #60
sc68cal wrote: post #37

Earlier quoted context omitted.

>What's the hypervisor? OpenStack supports multiple hypervisors. They are implemented as drivers - and are in this part of the OpenStack codebase: https://github.com/openstack/nova/tree/master/nova/virt/ See my slides from a talk I did yesterday. https://github.com/sc68cal/openstack-theory-and-practice-pre...

Then they should say that. Reading their website tells me their are doing that bullshit thing were they divorce the tech from the marketing completely. > OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web…

[deleted]
Post reply on HN