Earlier quoted context omitted.
Solaris is not free software and the free software forks never gained much traction. Plus it is now also associated with Oracle which brings along a lot of extra distrust, particularly given how litigious they are.
FreeBSD jails it is then.
Another reason why Docker containers may be slow
31–40 of 72 posts
Re: Another reason why Docker containers may be slow
#32Better title: “nother reason why my code is slow and I’m logging too much”
The point was not to complain about how bad the Docker is but rather to highlight that a lot of unexpected things may come from the fact that the kernel is shared. "Logging too much" was just a reason for the posix_fadvise being called too often but this becomes a problem ONLY when the kernel is shared. In case of virtualization, everyone gets its own version of fadvise (the kernel) and the conflict doesn't happen.
Re: Another reason why Docker containers may be slow
#33Earlier quoted context omitted.
The point was not to complain about how bad the Docker is but rather to highlight that a lot of unexpected things may come from the fact that the kernel is shared. "Logging too much" was just a reason for the posix_fadvise being called too often but this becomes a problem ONLY when the kernel is shared. In case of virtualization, everyone gets its own version of fadvise (the kernel) and the conflict doesn't happen.
If your VM call to `fadvise` is not calling the underlying host kernel operation, is it even working?
Re: Another reason why Docker containers may be slow
#34Earlier quoted context omitted.
If your VM call to `fadvise` is not calling the underlying host kernel operation, is it even working?
Hmm. I'm definitely not an expert in hypervisor implementations but I would guess that it should not proxy any calls to the host kernel...
Re: Another reason why Docker containers may be slow
#35Re: Another reason why Docker containers may be slow
#36I don't understand why more people don't use Solaris Zones, they seem to me to be the superior solution by far, and with work done by Joyent you now have modern LX-branded zones also. Is the lack of adoption mainly due to the fact that it's Solaris, and not Linux? (Solaris lives on in Illumos et al)
Last I used Illumos it had pretty poor hardware support (at least at entry to midlevel hardware which for obvious reasons must be a less important priority). I also ran into issues with their KVM port; mainly panics with various linux kernels (kvm clock, virt*, ..). Good experience in general though, especially the zones/zfs/kvm combo.
If you are itching for some slides, http://bhyvecon.org/bhyvecon2018-Gwydir.pdf
We have some work to do before we are ready for widespread use. As those pieces fall in place, we'll announce on smartos-discuss@lists.smartos.org.
Re: Another reason why Docker containers may be slow
#37Earlier quoted context omitted.
FreeBSD jails it is then.
I don't know anything about Solaris Zones, but I'm guessing that the FreeBSD jails wouldn't help with the problem in the article. It's still the same kernel running in the jail, so the fadvise calls are going to bottleneck in the same place.
Re: Another reason why Docker containers may be slow
#38Earlier quoted context omitted.
Solaris is not free software and the free software forks never gained much traction. Plus it is now also associated with Oracle which brings along a lot of extra distrust, particularly given how litigious they are.
Illumos/SmartOS is FOSS
Re: Another reason why Docker containers may be slow
#39I don't understand why more people don't use Solaris Zones, they seem to me to be the superior solution by far, and with work done by Joyent you now have modern LX-branded zones also. Is the lack of adoption mainly due to the fact that it's Solaris, and not Linux? (Solaris lives on in Illumos et al)
For instance, the directory name lookup cache can greatly impact the performance of file operations. This is what makes it so that when you open /a/b/c/d/e/blah, the OS has a good chance of knowing how to open "blah" directly without first searching /a, /a/b, /a/b/c, /a/b/c/d, and /a/b/c/d/e. I don't have performance numbers handy (left at $job - 1 and $job - 2), but the default size is not great for a system that handles hundreds of thousands of files. If that sounds like a lot of files, count how many files are read as part of a reasonably large build. Then imagine there are dozens of them running concurrently. Or imagine that you are hosting git repos or a bunch of static web content.
The obvious answer is to just increase the size of the DNLC. The problem is that there are other parts of the system that behave poorly with a large DNLC. For instance, whenever anyone tries to unmount a file system, dnlc_purge_vfsp() is called. This walks the cache, looking for entries that are associated with the file system. Who cares, right? We hardly ever unmount file systems. Well, if you use the automounter (by default Solaris uses it for at least /home/*), every few minutes it is trying to unmount all of the automounted file systems. The purge happens before EBUSY can be detected so hot cached entries may be purged while adding contention on dnlc-related locks. What's worse, the automounter doesn't reset its inactivity timer when it hits EBUSY, causing more frequent attempts to unmount than you would otherwise expect. There are other DNLC bottlenecks on a Solaris NFS server when a client removes a file.
And then as you get to larger systems with more NUMA effects, these type of operations become even more expensive.
Operating systems are hard. Making them scale to an infinite number of processes and processors is impossible. At a certain point, it becomes beneficial to use smaller hardware and/or add VMs into the mix.
Re: Another reason why Docker containers may be slow
#40Earlier quoted context omitted.
I don't know anything about Solaris Zones, but I'm guessing that the FreeBSD jails wouldn't help with the problem in the article. It's still the same kernel running in the jail, so the fadvise calls are going to bottleneck in the same place.
Why would an entirely different piece of software have the same performance bottlenecks?
cgroups, Jails, and Zones all suffer from having to cover an immense surface area. Contrast with VMs, which only require managing a few, significantly simpler interfaces. There are definitely differences in quality, but they all use a similar approach.