Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparing Filesystem Performance in Virtual Machines

mitchellh.com

Post reply on HN
Print view View original

21–30 of 46 posts

Re: Comparing Filesystem Performance in Virtual Machines

#21

Post by noselasd »

buster wrote: post #12
icebraining wrote: post #4

Earlier quoted context omitted.

From the article: It is interesting that sometimes the native filesystem within the virtual machine outperforms the native filesystem on the host machine. This test uses raw read system calls with zero user-space buffering. It is very likely that the hypervisors do buffering for reads from their virtual machines, so they’re seeing better performance from not context switching to the native kernel as much. This theory…

It doesn't explain a thing on why. He just measured the performance of memory access and different caching strategies. From my point of view the "benchmarks" say nothing at all about actual I/O disk performance in virtual and native environments.

That says exactly why.

You read from a file and tell the OS to not buffer it(or you tell the OS to flush the caches before you start) on a native system, and it does exactly that.

You read from a file and tell the OS to not buffer it (or you tell the OS to flush the caches before you start) on a VM, and the OS thinks it does that, but the hypervisor living below the VM buffers some of the data anyhow, so you're really just reading from memory.

Re: Comparing Filesystem Performance in Virtual Machines

#22

Post by bradleyland »

Nux wrote: post #20

Would love to see in there KVM and Xenserver; you know, stuff that actual clouds run on.

This is pretty clearly a test of developer related tools, not production cloud server infrastructure. I'm not even sure there's an equivalent of VirtualBox/VMWare shared folders in KVM or Xen, because guests and hosts don't usually share folders in the same way that you do with these workstation virtualization tools.

...

Spoke too soon. A Google search shows there are some methods [1], but their use cases are different.

[1]: http://www.linux-kvm.org/page/9p_virtio

Re: Comparing Filesystem Performance in Virtual Machines

#24

Post by mitchellh »

weddpros wrote: post #10

3GB/s writes on a single SSD should raise more eyebrows. I dunno what was actually benchmarked, but there's a problem somewhere...

You're assuming the writes are actually going to a physical disk. As I mentioned in the post, the hypervisors are very likely just writing to RAM and not ever committing it to disk. Even when you `fsync()` from a VM, there is no guarantee the hypervisor puts that to disk.

If you look at the graphs, they corroborate this. The "native" disk never really exceeds 500 to 600 MB/s, which is about as fast as my SSD goes. The hypervisors, however, are exceeding multiple GB/s. It must be RAM.

Also, re: "I'm not sure what was actually benchmarked" The method of benchmarking is covered in the bottom of the post. I realize it isn't extremely detailed. If you have any questions, I'd be happy to answer.

Re: Comparing Filesystem Performance in Virtual Machines

#25

Post by mitchellh »

Nux wrote: post #20

Would love to see in there KVM and Xenserver; you know, stuff that actual clouds run on.

bradleyland is correct: This test was focused primarily on using VMs for development tools. This test was done on a local machine with desktop virtualization software. The opening paragraph mentions I was investigating performance for development environments. This post should not be used for any production applications, since it would make no sense.

Re: Comparing Filesystem Performance in Virtual Machines

#26

Post by weddpros »

mitchellh wrote: post #24
weddpros wrote: post #10

3GB/s writes on a single SSD should raise more eyebrows. I dunno what was actually benchmarked, but there's a problem somewhere...

You're assuming the writes are actually going to a physical disk. As I mentioned in the post, the hypervisors are very likely just writing to RAM and not ever committing it to disk. Even when you `fsync()` from a VM, there is no guarantee the hypervisor puts that to disk. If you look at the graphs, they corroborate this. The "native" disk never really exceeds 500 to 600 MB/s, which is about as fast as my SSD goes. Th…

I meant to say you've not benchmarked disk access, and we have no idea what each filesystem is actually doing. Caching performance is "nice", but it says nothing about the actual performance we'd get in real use. Maybe the "slow" fs just exhibits less aggressive caching, which might prove just as efficient depending on the workload. It's definitely interesting to note the huge differences, but I'd really like to see how it goes in "real" conditions...

Re: Comparing Filesystem Performance in Virtual Machines

#27

Post by contingencies »

In the past, industry threw hardware at things. Virtualization reduced this wastefulness somewhat, but now developers are fighting back against unreliable performance. If you are developing a performance-sensitive system, executing similar tests routinely but with real workloads should be part of your test process... and certainly occur before deployment. Third party tests on some hardware with some version of some code on some kernel, such as what we see here, are really neither here nor there.

Re: Comparing Filesystem Performance in Virtual Machines

#28

Post by liuw »

Nux wrote: post #20

Would love to see in there KVM and Xenserver; you know, stuff that actual clouds run on.

I think you mean KVM and Xen. Xen hypervisor is open source project just like KVM while XenServer is a product that uses Xen hypervisor.

Just think of Linux kernel and Linux distributions.

Re: Comparing Filesystem Performance in Virtual Machines

#29

Post by rbanffy »

newman314 wrote: post #13

It would have been interesting to see a comparison with Xen etc. too.

It's primarily a development environment test, where the host runs OSX. It would be interesting extending the test to Parallels on Macs and adding a Linux host where KVM and LXC could be used.
Post reply on HN