Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

forbes.com

Post reply on HN
Print view View original

61–70 of 128 posts

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#61

Post by hp50g »

druiid wrote: post #54
hhw wrote: post #52

Earlier quoted context omitted.

Exactly. Virtualization/Cloud are only good for scaling down, not scaling up. The 20-30% hit to IO, additional latency of SAN vs local storage, management overhead, and added complexity to infrastructure more than negate any potential benefits of improved load distribution when you're dealing with applications that use anywhere near the capacity of physical servers.

20-30% hit in IO? It hasn't been at that level for a long time. With new KVM versions and good Intel processors there is a performance hit of as low as 5% these days. That's not to necessarily say your particular workload will see that, but 10% is generally an 'at-worst' level at this point.

Its definitely 20-30% for realistic workloads using VM based tech on top I.e. CLR/JVM or a database engine which is realistic. This is on top of VMware. I can't speak for Xen.

The outcome is pretty grim.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#62

Post by hp50g »

incision wrote: post #59
hp50g wrote: post #26

I think this only solves the cost problem. VMware is damn expensive for what it is. I'm not really a fan of virtualization myself. We canned it at our organisation a couple of years ago. We had 30 ESX/vSphere managed hosts across 3 data centers which hosted app servers, web front end servers, virtual load balancers and some other minor infrastructure. The final straw was the upgrade costs. Not only that we had proble…

>It's snake oil. As opposed to a miracle elixir that cures all ills? It's a tool. Like most tools, it's best used when you thoroughly understand the challenge and how/why to apply that tool to it. The problems you've noted here seem more indicative of poor planning and understanding of the solution than any intrinsic deficiency of virtualization.

Its a shit tool that creates issues and costs fuckwads of cash. That is it.

We understand it. The numbers sold are not the numbers gained.

The entire infrastructure deployment was planned around virtualization, DC failover and resilience using VMware's guidelines and solutions (VMware were even paid to consult on this). It never delivered and doubled our administrative and licensing overhead.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#63

Post by rektide »

3am wrote: post #36
anotherjesse wrote: post #33

Earlier quoted context omitted.

The Diablo version was the first version that really combined the projects. Before Diablo each system had its own auth/user system, api style, ... We've come a long way in 2 years :) Redhat (EPEL/Fedora) and Ubuntu now have built-in packages that are much more solid. That said, the way to think about OpenStack is like you think about the linux kernel. Most people want a distro, not raw source to setup their environme…

I've kept up a little on OpenStack since then and know people that work with the technology. I'm sure the keystone integrated auth system (or whatever it's called now) really improved things, and it seems like people are making really progress with pushbutton deployments using the usual config/orchestration suspects. Congrats! That said I know more people would would be content to manage ESX with ovftool or some home…

  I know more people would would be content to manage ESX with ovftool or some homebrew solution w/ the soap api
I for one am not content relying on someone else to manage all of our management tools. On that grounds, love hearing interesting news about OpenStack, CloudStack, Docker.io, or any project which releases open code purporting to helps us all to manage our many nodes, real and virtual.

  without usual userspace tools
As for the unusual "without usual userspace tools" criteria you added, I literally have no idea what you are talking about.

OpenStack is a bunch of code that runs in an OS, there's nothing nearly so magical or different about an OpenStack node as you imply. Do whatever you feel is best for your host nodes.

I install Debian packages on the host and manage it with many of the same tools used to manage it's images. It looks very much like the same usual "just a boring usual Linux node" node I log in to. http://wiki.debian.org/OpenStack

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#64

Post by hp50g »

IheartApplesDix wrote: post #50
hp50g wrote: post #26

I think this only solves the cost problem. VMware is damn expensive for what it is. I'm not really a fan of virtualization myself. We canned it at our organisation a couple of years ago. We had 30 ESX/vSphere managed hosts across 3 data centers which hosted app servers, web front end servers, virtual load balancers and some other minor infrastructure. The final straw was the upgrade costs. Not only that we had proble…

Virtualization is a requirement for any kind of serious dynamic infrastructure.

Processes are a form of virtualisation are they not?

That's enough abstraction.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#65

Post by superuser2 »

3am wrote: post #25

I hate cloud headlines. What's the hypervisor? VMWare is a hypervisor (ESX) and management applications (vCenter/vSphere, vCloud Director, etc). Openstack is only the management applications. And VMWare contributes the most interesting part of OpenStack, anyway, which is the virtual networking based on Nicira's OpenFlow. My personal experience with OpenStack (admittedly late Diablo release timeframe) was that it was…

Rackspace Cloud Servers runs on OpenStack. It seems to work pretty well for them.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#66

Post by incision »

outside1234 wrote: post #5

Is anyone in the startup world using OpenStack? If so, how is it working for you? I'm always skeptical of abstractions but it'd be great to hear about any real world experience.

Not in the startup world, but I've used it quite a bit.

I'm a big fan.

It's pretty trivial to get a basic system up and running and your basic functions for creating, controlling and destroying instances work flawlessly. In general, core services are quite reliably and reasonably documented.

I've managed to create some sticky problems fooling with Quantum [1] when it was still in incubation, but nothing a little digging couldn't resolve.

1: https://wiki.openstack.org/wiki/Quantum#What_is_Quantum.3F

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#67

Post by rektide »

hp50g wrote: post #61
druiid wrote: post #54

Earlier quoted context omitted.

20-30% hit in IO? It hasn't been at that level for a long time. With new KVM versions and good Intel processors there is a performance hit of as low as 5% these days. That's not to necessarily say your particular workload will see that, but 10% is generally an 'at-worst' level at this point.

Its definitely 20-30% for realistic workloads using VM based tech on top I.e. CLR/JVM or a database engine which is realistic. This is on top of VMware. I can't speak for Xen. The outcome is pretty grim.

Ah, VMware is the problem, that is explains it.

Parties that care muchly about fine performance margins apparently need to be using Xen or KVM or Illuminos then.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#68

Post by wmf »

hp50g wrote: post #64
IheartApplesDix wrote: post #50

Earlier quoted context omitted.

Virtualization is a requirement for any kind of serious dynamic infrastructure.

Processes are a form of virtualisation are they not? That's enough abstraction.

Originally, yes. Unfortunately Unix/Windows has accumulated a ton of global state (libfoo 3.7.1 not parallel installable with libfoo 3.6.4) and enterprise software is now so fragile that any alternative to VM sprawl is unthinkable.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#69

Post by rektide »

babarock wrote: post #29

Slightly relevant: If you're interested in discovering OpenStack, you should check out devstack[1]. It's a bash script that (git) pulls different projects of openstack into a machine and install them. The script itself is very readable, thoroughly commented, on purpose. You're encouraged to read it while the whole thing installs (which can take several minutes). Feel free to launch it inside of a VM (preferably runni…

I went with a somewhat more ready-made, native way of getting OpenStack- I installed my OS's packages of it. http://wiki.debian.org/OpenStack

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#70

Post by rektide »

sc68cal wrote: post #38
babarock wrote: post #29

Slightly relevant: If you're interested in discovering OpenStack, you should check out devstack[1]. It's a bash script that (git) pulls different projects of openstack into a machine and install them. The script itself is very readable, thoroughly commented, on purpose. You're encouraged to read it while the whole thing installs (which can take several minutes). Feel free to launch it inside of a VM (preferably runni…

You can also use Vagrant+Devstack[1] to get started even quicker. https://github.com/bcwaldon/vagrant_devstack

So, what, now you can virtualize while your virtualize? I thought the point of OpenStack was to be a way to manage VMs, isn't that also much of what Vagrant does?

I'd tend to avoid mixing the streams & go for a single node management system rather than attempt two at once- were it me. But I also am not invested in Vagrant, which people are free to enjoy, so if they can happily run both VM managers at once, so be it. Cool, I guess.

I would like some clarity on what exactly this hybrid achieves though- what is it's goal?

Post reply on HN