That's all well and good ( increasing readability ) but the problem remains that each step has to finish before the next step can begin... sometimes the entire dataset won't fit into memory/machine/whatever...
More useful, IMHO, would be a way to EASILY compose a true pipeline:
const
_pipe = (a, b) => (arg) => b(a(arg)),
pipe = (...ops) => ops.reduce(_pipe)
...but have the behavior work like unix pipes ( a stream ), nodeJS supports this concept at it's most basic level using the pipe() abstraction, although you have to supply methods which handle being pipe'd to, and from... an example:
const crypto = require('crypto');
// ...
fs.createReadStream(file)
.pipe(zlib.createGzip())
.pipe(crypto.createCipher('aes192', 'a_secret'))
.pipe(reportProgress)
.pipe(fs.createWriteStream(file + '.zz'))
.on('finish', () => console.log('Done'));
*ripped from: [source](
https://medium.freecodecamp.org/node-js-streams-everything-y...)
Imagine reading a 100gb json file line-by-line via ajax on the client, and feeding into the pipeline of transformative methods -- iteratively introduce data in one end of the pipe, and gathering the results at the other end, and creating some visualization like a graph or whatever... without ever having to have the entire thing in memory at once...