Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

forbes.com

Post reply on HN
Print view View original

121–128 of 128 posts

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#121

Post by hp50g »

simplexion wrote: post #102
hp50g wrote: post #26

I think this only solves the cost problem. VMware is damn expensive for what it is. I'm not really a fan of virtualization myself. We canned it at our organisation a couple of years ago. We had 30 ESX/vSphere managed hosts across 3 data centers which hosted app servers, web front end servers, virtual load balancers and some other minor infrastructure. The final straw was the upgrade costs. Not only that we had proble…

Volume size limits on your SAN... Is 16TB too restrictive for your volume size limit?

2TiB per LUN on ESX until recently actually. We have a single filesystem that is 19TiB.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#122

Post by incision »

simplexion wrote: post #104
druiid wrote: post #79

Earlier quoted context omitted.

VMWare sales from what I have heard, has a history of over-selling, over-promising and ultimately getting people to spend a crap-ton of money on their products. Honestly I have to say, it sounds a bit like you guys got taken for a ride. I'd suggest sitting back, relaxing and taking a look at what other VM options there are out there, some of which don't have license fees attached to them. When I first deployed virtua…

I see people being WAY-oversold on the SAN more than the software.

I echo this observation.

By now, I'd actually be more surprised not to find a VMAX or Shark in an "Enterprise" datacenter.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#123

Post by justincormack »

hp50g wrote: post #61
druiid wrote: post #54

Earlier quoted context omitted.

20-30% hit in IO? It hasn't been at that level for a long time. With new KVM versions and good Intel processors there is a performance hit of as low as 5% these days. That's not to necessarily say your particular workload will see that, but 10% is generally an 'at-worst' level at this point.

Its definitely 20-30% for realistic workloads using VM based tech on top I.e. CLR/JVM or a database engine which is realistic. This is on top of VMware. I can't speak for Xen. The outcome is pretty grim.

Virtualization which cares about iops needs SSD. Hard drives can't be sanely virtualized.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#124

Post by simplexion »

hp50g wrote: post #121
simplexion wrote: post #102

Earlier quoted context omitted.

Volume size limits on your SAN... Is 16TB too restrictive for your volume size limit?

2TiB per LUN on ESX until recently actually. We have a single filesystem that is 19TiB.

So... 19TiB and you're only using iSCSI instead of NFS and you're complaining about volume size restrictions?

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#125

Post by hp50g »

simplexion wrote: post #124
hp50g wrote: post #121

Earlier quoted context omitted.

2TiB per LUN on ESX until recently actually. We have a single filesystem that is 19TiB.

So... 19TiB and you're only using iSCSI instead of NFS and you're complaining about volume size restrictions?

We have to use iSCSI. Not all our kit talks NFS plus we use SAN block replication between DCs which really doesn't play nice with NFS (we did test it and discovered that not all NFS implementations are good).

This whole thing has to play nice with a windows DFS cluster as well.

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#126

Post by simplexion »

hp50g wrote: post #125
simplexion wrote: post #124

Earlier quoted context omitted.

So... 19TiB and you're only using iSCSI instead of NFS and you're complaining about volume size restrictions?

We have to use iSCSI. Not all our kit talks NFS plus we use SAN block replication between DCs which really doesn't play nice with NFS (we did test it and discovered that not all NFS implementations are good). This whole thing has to play nice with a windows DFS cluster as well.

You said you're using ESXi? The Windows virtual servers won't care whether they are sitting on a LUN or NFS. Why do you have to use iSCSI? Or SAN block replication for that matter? If you think you need to use iSCSI, why don't you use it for some of your storage and NFS for parts that require storage beyond 2TB?

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#127

Post by Swannie »

IheartApplesDix wrote: post #50
hp50g wrote: post #26

I think this only solves the cost problem. VMware is damn expensive for what it is. I'm not really a fan of virtualization myself. We canned it at our organisation a couple of years ago. We had 30 ESX/vSphere managed hosts across 3 data centers which hosted app servers, web front end servers, virtual load balancers and some other minor infrastructure. The final straw was the upgrade costs. Not only that we had proble…

Virtualization is a requirement for any kind of serious dynamic infrastructure.

Except that Google, Facebook, etc. etc. successfully use baremetal deployments without a VM in sight...

Re: Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack

#128

Post by ternaryoperator »

mindjiver wrote: post #118
ternaryoperator wrote: post #114

Earlier quoted context omitted.

Curious how you've set this up. What role does the MQ play? Does it ship results to the mother ship or a console? Or are you sending notices of changes to the test machine? etc. TIA

We have custom plugin to send message to the MQ. Other Jenkins-instances running the same plugin will receive the message and can act on them according to the configuration. Currently we mostly use this to send internal release notifications around, e.g. "foobar-1.0.0 released, find it here ". Kind of neat with a n * 1000 developer organization spread over the entire world. Currently the plugin is not open source but…

Very cool. Thanks for the explanation!
Post reply on HN