Tuning JVM for a VM - Lessons Learned, Directly from VMware • Hacker News Board

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#11

Post by guelo » Fri, Apr 13, 2012, 6:25 AM UTC

rickette wrote: post #10

guelo wrote: post #9

Or you could just run the JVMs on the base OS and use good old file system security if you really need sandboxing. The supposed advantages of adding a OS VM layer seem iffy if you're just running one app per.

It always amazed me that the JVM doesn't have any options to restrict CPU resources (something VMware does have). The only alternative I can think of is to use the unix taskset utility to limit JVMs to specific CPU cores. Anybody got experiece with a setup like this? Limiting memory usage and file permissions with the JVM is indeed easy.

Let the OS manage the CPUs, you can use nice if you need to prioritize.

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#12

Post by HackR » Fri, Apr 13, 2012, 6:31 AM UTC

rrdharan wrote: post #3

sciurus wrote: post #2

Efficiently using memory when you have three levels (Hypervisor->OS->JVM) that are largely opaque to each other is an real challenge. Doing ballooning at the JVM level [0] instead of the OS level makes sense for VMs primarily running Java applications. VMware often seems to be the first to get a new technology out, but their competition quickly follows (e.g. KSM for transparent page sharing and compcache for memory c…

[Disclosure: I worked at VMware from 2002 through 2011.] I take issue with the claim that the competition "quickly followed" on for (at the least) the transparent page sharing feature. VMware was shipping transparent page sharing in their type 1 hypervisor since ESX 1.5, which was released before I joined the company. KSM was first proposed in 2008, and I believe it didn't actually ship until 2009 (note: the delay wa…

Out of curiosity what made you leave VMware?

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#13

Post by JoachimSchipper » Fri, Apr 13, 2012, 7:30 AM UTC

Yeroc wrote: post #6

sciurus wrote: post #2

Efficiently using memory when you have three levels (Hypervisor->OS->JVM) that are largely opaque to each other is an real challenge. Doing ballooning at the JVM level [0] instead of the OS level makes sense for VMs primarily running Java applications. VMware often seems to be the first to get a new technology out, but their competition quickly follows (e.g. KSM for transparent page sharing and compcache for memory c…

Azul Systems has been marketing a different solution to this same problem in the Zing VMs for some time now. I believe their solution is based on adding extra APIs to the underlying kernel etc. so the JVM's heap isn't so opaque to the OS/hypervisor. The VMWare solution sounds like a simpler, pragmatic "hack" to solve the problem. It would be interesting to hear from someone that has used both technologies to see how…

As I understand it, Azul rather solves this problem by giving control of the memory-mapping unit to the VM. Which allows some really neat tricks (I'd expect it also allows VM->kernel privilege escalation, but their machines are Java appliances anyway.)

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#14

Post by ntoshev » Fri, Apr 13, 2012, 7:53 AM UTC

guelo wrote: post #11

rickette wrote: post #10

Earlier quoted context omitted.

It always amazed me that the JVM doesn't have any options to restrict CPU resources (something VMware does have). The only alternative I can think of is to use the unix taskset utility to limit JVMs to specific CPU cores. Anybody got experiece with a setup like this? Limiting memory usage and file permissions with the JVM is indeed easy.

Let the OS manage the CPUs, you can use nice if you need to prioritize.

There is also cpulimit if you want to set hard limits.

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#15

Post by zokier » Fri, Apr 13, 2012, 8:42 AM UTC

derleth wrote: post #4

So, how reasonable is it right now to run a JVM directly on a VM?

http://labs.oracle.com/projects/guestvm/

edit: see also LiquidVM, possibly aka JRockit VE

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#16

Post by gaius » Fri, Apr 13, 2012, 8:45 AM UTC

Is it just me or is running a VM in a VM just crazy? The application stack now looks like

OS -> Hypervisor -> VM -> OS -> VM -> App

Where "App" might be a service so you might have this stack duplicated dozens of times for an actual app that a user can use. And you don't save anything either, rather than "processes" your sysadmins manage "VMs". What happened to

OS -> App

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#17

Post by gaius » Fri, Apr 13, 2012, 8:47 AM UTC

derleth wrote: post #4

So, how reasonable is it right now to run a JVM directly on a VM?

Oracle are running JRockit directly on the hypervisor, no OS.

http://docs.oracle.com/cd/E23009_01/doc.200/e22518/introduct...

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#18

Post by sciurus » Fri, Apr 13, 2012, 5:15 PM UTC

rrdharan wrote: post #3

sciurus wrote: post #2

Efficiently using memory when you have three levels (Hypervisor->OS->JVM) that are largely opaque to each other is an real challenge. Doing ballooning at the JVM level [0] instead of the OS level makes sense for VMs primarily running Java applications. VMware often seems to be the first to get a new technology out, but their competition quickly follows (e.g. KSM for transparent page sharing and compcache for memory c…

[Disclosure: I worked at VMware from 2002 through 2011.] I take issue with the claim that the competition "quickly followed" on for (at the least) the transparent page sharing feature. VMware was shipping transparent page sharing in their type 1 hypervisor since ESX 1.5, which was released before I joined the company. KSM was first proposed in 2008, and I believe it didn't actually ship until 2009 (note: the delay wa…

Thanks for the correction!

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#19

Post by sciurus » Fri, Apr 13, 2012, 5:16 PM UTC

kristianp wrote: post #8

sciurus wrote: post #2

Efficiently using memory when you have three levels (Hypervisor->OS->JVM) that are largely opaque to each other is an real challenge. Doing ballooning at the JVM level [0] instead of the OS level makes sense for VMs primarily running Java applications. VMware often seems to be the first to get a new technology out, but their competition quickly follows (e.g. KSM for transparent page sharing and compcache for memory c…

That link is broken, can you fix it please?

Sorry, try http://pubs.vmware.com/vfabric5/topic/com.vmware.ICbase/PDF/...

Re: Tuning JVM for a VM - Lessons Learned, Directly from VMware

#20

Post by sciurus » Fri, Apr 13, 2012, 5:26 PM UTC

rickette wrote: post #10

guelo wrote: post #9

Or you could just run the JVMs on the base OS and use good old file system security if you really need sandboxing. The supposed advantages of adding a OS VM layer seem iffy if you're just running one app per.

It always amazed me that the JVM doesn't have any options to restrict CPU resources (something VMware does have). The only alternative I can think of is to use the unix taskset utility to limit JVMs to specific CPU cores. Anybody got experiece with a setup like this? Limiting memory usage and file permissions with the JVM is indeed easy.

Create a cgroup hierarchy with the CPU subsystem attached, then put each JVM in its own cgroup in that hierarchy. By default, each JVM will get an equal amount of CPU time. A JVM can be given relatively more CPU time by increasing cpu.shares for its cgroup. You can similarly manage memory and i/o with the memory and blkio subsystems.

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6...