JavaScript in Parallel: Web Workers and SharedArrayBuffer

It is really how disappointing the options to share data with workers are in the browser.

Combine the limited options options to transfer data quickly, and the limited API available in the worker itself (no version of a DOM, even a gutted one for doing measurements), it is no surprise how few opportunities to use them effectively there are.

When it comes to transferring objects, it is so bad people are resorting to JSON.stringifying messages. [0]

Seems like it would be easy to just add an Immutable Array and Map to the standard library, and let people use that on workers without these silly limitations. What am I missing?

[0] https://nolanlawson.com/2016/02/29/high-performance-web-work...

I used web workers extensively in a JS project a little over a year ago, and it was a nightmare. Granted what I was attempting was crazy, but it shouldn't have been as painful as it was.

Basically the "gotchas" are in the browser implementations. Each browser's web worker implementation is a little different, and those little differences (bugs) have huge implications in terms of what you can and can't do. For example: one browser may relay direct worker-to-worker messages via the main thread, so if your main thread is blocked your workers can't talk to each other, and that largely defeats the entire purpose of direct worker-to-worker communication.

Fortunately some of these issues have been fixed since (including that one, if memory serves), but it's slow going. I believe what happened was workers were introduced 5+ years ago, met with little interest initially, and the APIs have rotted for years until recently when interest picked up again, especially since SharedArrayBuffer.

My qualm with SharedArrayBuffer is that it kind of sucks to use in pure JS projects, because you have to serialize/deserialize everything to and from the buffer. With the Emscripten toolchain's pthreads support, as far as I'm aware you just compile your code and the heap lives inside SharedArrayBuffer. You don't have to write boilerplate serialization code, so compared to plain JavaScript it's seamless experience in that regard.

My advice to anyone using web workers in a pure JavaScript project is to use them as their name implies: offloading long-running calculations. If you try to treat them as true threads, you're going to have a bad time. Especially if your inter-worker messaging volume is high and you have frequent interdependent (blocking) calculations.

That said, the work I did was prior to SharedArrayBuffer. For very high-performance projects, it will likely be prudent to use SharedArrayBuffer itself as a messaging medium between workers.

Opera Mini doesn't support it, the rest of them do

I kept finding myself needing to toss stuff in a background thread, and ended up making this: https://github.com/icodeforlove/task.js

It makes wrangling multiple workers much easier, and doesn't require you to have external JS files. (also works in node.js)

demo: http://s.codepen.io/icodeforlove/debug/ZOjBBB/NjrYzwzWdzLA