41–50 of 82 posts
Intel compilers aren't available for bsd. The improved optimisation versus gcc is worthwhile.
I wonder if this is another validation of the "worse is better" philosophy as described recently in an HN article http://minnie.tuhs.org/pipermail/tuhs/2017-May/009935.html and also discussed at https://www.jwz.org/doc/worse-is-better.html
JWZ seems to dislike Hacker News for some reason, judging by the redirect.
HPC may look like COTS gear but it's not. BSD doesn't have drivers for Infiniband and other HPC interconnects. Nor does it have client drivers (let alone server implementation) for Lustre which is the distributed filesystem used by most super computers. I imagine MPI support on BSD is also likely non-existent. Then there is the matter of accelerator support, i.e NVidia GPUs and Intel Xeon Phi. It's not to say that so…
It most certainly did a decade ago. That was what the network in the mental health institution I contracted with used. Infiniband fabric and a BSD kernel, it was my first professional run-in with BSD.
Many of the vendors for HPC (I'm looking at you Mellanox) primarily develop and certify their products on Linux. While they might work on BSD ok, you're not going to get the full performance and all of the features on a BSD system. If you paid for Mellanox EDR 100G Infiniband switches and all of the fancy VPI network cards, you want to use them to the fullest performance capable. The vendor tells you to use Linux for that, you use Linux.
TL;DNR: Linux is what the hardware manufacturers overwhelmingly target and work with. HPC users use what vendors support best.
Scientific high energy physicist here with regional HPC center on the same floor. My observation is that administrators tend to enterprise distributions such as Scientific linux, Suse linux enterprise server (SLES), together with commercial MPI implementations such as IBM MPI and Intel MPI. On the other hand, people are used to Linux, in my environment literally everybody has Ubuntu on their notebook and workstation.…
Modules looks really interesting. Makes me wonder why Continuum is out there trying to reinvent the wheel with Anaconda. Glad to have something I can use at work to replace Conda environments. Now all it needs is Powershell/CMD support so I don't have to use it inside Cygwin...
It also helps with versioning. It is not uncommon to see various versions of gcc and intel compilers. In essence the user should be able to load their environment with a few module loads.
Here's some more info, if you're interested
[1]http://www.admin-magazine.com/HPC/Articles/Managing-the-Buil...
[2]http://www.admin-magazine.com/HPC/Articles/Managing-Cluster-...
[3]https://uisapp2.iu.edu/confluence-prd/pages/viewpage.action?...
Earlier quoted context omitted.
I think netflix use BSD because they wanted to use BSD. Sure some flavours of BSD have ZFS built in, but thats a pretty rare corner case. Linux has two things that are extremely useful compared to BSD: 1) commercial backing (should one choose it) 2) first class support for inifinband, top end ethernet (should they use it) and storage controllers
Netflix uses BSD for OpenConnect because asynchronous disk I/O-which is critical for a CDN-remains a tire fire on Linux after more than 20 years. On Linux you basically have to use blocking threads to emulate async disk I/O, which means tons of threads and overhead when you’re handling 10k-100k concurrent connections per box.
Some will say better hardware support. While Linux has better hardware support, i usually find this to be in the more exotic direction. I think it's simply down to Linux being where the money is. The big players (IBM, Dell, etc) are all actively promoting Linux, and trained personel is also somewhat easy to find. So Linux is "the beast you know". As for FreeBSD, it might be a technically better platform, but it is li…
Linux has ZFS.
Earlier quoted context omitted.
This is the correct answer (especially Infiniband - and Aries on Crays) Also NUMA is very important on supercomputers, and it works well on Linux. The other thing worth noting is the much better support IBM has for Linux on PowerPC (2 in the top 10). I think Sunway (most powerful in the world) is a Linux shop too.
Is it egg and chicken matter? Vendors don't write driver for BSD and BSDs lack of users because lacks of drivers. Honestly I hope I can run an OpenBSD and install whatever driver for my plugged in devices, both for my personal and production servers.
As others have mentioned there was a Mellanox stack in Free circa 2005 that I worked with. It was used at Isilon (BSD based) in production.
There really isn't a technical discussion here at all, when an overwhelmingly large part of your userbase uses X, it would be pretty stupid to only support Y, and probably not defensible to support X and Y
Earlier quoted context omitted.
Netflix uses BSD for OpenConnect because asynchronous disk I/O-which is critical for a CDN-remains a tire fire on Linux after more than 20 years. On Linux you basically have to use blocking threads to emulate async disk I/O, which means tons of threads and overhead when you’re handling 10k-100k concurrent connections per box.
This is incorrect. Linux has had proper direct async disk I/O for a decade or more, used ubiquitously in database engines (among other things). It is not emulated with threads.
Earlier quoted context omitted.
I think netflix use BSD because they wanted to use BSD. Sure some flavours of BSD have ZFS built in, but thats a pretty rare corner case. Linux has two things that are extremely useful compared to BSD: 1) commercial backing (should one choose it) 2) first class support for inifinband, top end ethernet (should they use it) and storage controllers
Netflix uses BSD for OpenConnect because asynchronous disk I/O-which is critical for a CDN-remains a tire fire on Linux after more than 20 years. On Linux you basically have to use blocking threads to emulate async disk I/O, which means tons of threads and overhead when you’re handling 10k-100k concurrent connections per box.
On high performance async IO, this works quite well in Linux, and there are no blocking threads that I am aware of in that stack. The kernel uses bio dispatches to perform the actual block io. If you are complaining about using bio to perform the actual IO, and that linux includes this in its load calculation, sure, that is a conscious decision as I understand it on the part of the block layer folks. Is it wrong or bad? I don't think so, though others have different opinions.
FWIW ... I work at a place now using SmartOS as its primary OS. There are many people I know preferring BSD. Many people preferring linux. I have a different view, one that is not as popular as I hope it would be.
Specifically, I look at operating systems now, largely, as an implementation detail for your stack. You have a mission in many cases, unless you are an OS developer, that consumes the OS services layers to help you perform your mission. In many cases, specifics of the OS don't matter, as long as they don't get in your way. Sometimes the specifics of the OS help you.
From my view as an HPC guy, a hardware guy, a storage/compute/ML/GPU guy, I generally can work in Linux and BSD without pain. Minor config difference, but I am comfortable in both.
I am not, and have not been comfortable in AIX, HP/UX, and UnicOS. I used to enjoy IRIX until I started playing with Linux. I used Solaris and SunOS in the past, and SmartOS/illumos today.
As long as the OS has the tools I need, the libraries I need, or a way for me to build them, and doesn't constrain me or force me to contort to vagaries of the OS itself, I am fine with it.
A problem arises when people get caught up in "my OS > your OS", which, this overall question at least brings in under the covers. This usually comes around from various esoteric aspects of little relevance for the vast unwashed masses of users (like me). On the OS dev side, when this happens, it is usually defensive because something needed is missing, or some OS dev/manager (mis)believes that users don't actually need the features they are requesting.
That is actually a major problem, and it tends to drive people from your platform. Users aren't dumb, and there are many sophisticated people who have a deeper appreciation for the issues, than "my OS > your OS".
Why *BSD isn't used might be for historical reasons, momentum, etc. It is perfectly fine as an OS, and quite usable for HPC. Similar for illumos/SmartOS (not simply saying that as I work for a company using SmartOS). There are missing things in both of these, and I am working (on the side) to try to help SmartOS get some of these things (user space stuff). FreeBSD in particular has most of what is needed.
Basically pick the system that works for you and your users. The OS, as I noted, can be viewed as a detail of the implementation. Or not.
But its not a reason to create friction/tension between groups claiming OS1 > OS2 ...
The VI/Emacs wars are so 80s/90s ...