Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I want to write Node: Stream all the things

caolanmcmahon.com

Post reply on HN
Print view View original

41–50 of 118 posts

Re: How I want to write Node: Stream all the things

#41

Post by qubyte »

caolanmcmahon wrote: post #5
gfosco wrote: post #2

This sounds intriguing. I wish there were some more complex examples. Part of the greatness of Promises is taking a big chunk of pyramid code and turning it into a set of simple steps... I'd like to see how this would handle that.

Good idea, I'll definitely post a follow-up with some real code done using async/callbacks and highland/streams. The comparisons usually start to look more favourable with longer examples, due to the Highland API being so composable.

Nice. I need to compare code written with highland to code with async.series/.each etc. (but not waterfall, I don't like that dude ;)).

Re: How I want to write Node: Stream all the things

#42

Post by ilaksh »

I use async.map when I need to do a bunch of file operations or something asynchronously and wait for them to finish. I have largely avoided using stream specific syntax because the event model was more familiar to me. I have used ToffeeScript instead of async.waterfall or async. series because ToffeeScript is cleaner.

With these improvements to streams in Highland making things more convenient and broadly applicable I expect to be using Highland streams for certain things.

Re: How I want to write Node: Stream all the things

#43

Post by nailer »

spion wrote: post #40
nailer wrote: post #31

Earlier quoted context omitted.

From Great British Node Conf, maybe two months ago: a speaker talking about promises asked what people use for flow control: - promises: about 20% of the room. - async: about 80% of the room For me async.waterfall([list of functions]) is little nicer than 10 chained .thens(). And people advocating promises still keep saying it keeps things flat. No it doesn't, we're already flat because we're all using async. Stop pr…

var Promise = require('bluebird'); var fs = Promise.promisifyAll(require('fs')); Promise.spawn(function* () { var data = yield fs.readFileAsync(somefile); }); Documentation: https://github.com/petkaantonov/bluebird/blob/master/API.md My favorite example is doing a diff using an async diff service which provides a function `svc.diff(string1, string2)`. But also imagine that you need to preprocesses the files using the…

- Why BlueBird? What's wrong with Q?

- Isn't data a scope down?

- https://github.com/petkaantonov/bluebird/blob/master/API.md is API based, not task based. async is API based too, but the API has names like 'waterfall' and 'parallel'. I can click them because I know what they mean.

Promises just makes me feel like I'm reading about and endless series of abstractions.

Re: How I want to write Node: Stream all the things

#44

Post by jashkenas »

hamburglar wrote: post #32
jashkenas wrote: post #28

Earlier quoted context omitted.

Ha ha. You're describing the state of affairs before Underscore existed, back when functional-ish programming in JavaScript was ruled by Prototype.js: http://prototypejs.org/ ... which added a lot of useful methods to native prototypes. While handy in controlled and limited environments, mucking about with native prototypes quickly becomes extremely dangerous and difficult — once you have two third-party modules on t…

Whereas with _.map, you always know exactly which implementation you're getting, right? :)

Actually, you do. You've loaded it, and you can lock it down privately to your library or app with _.noConflict().

You can have ten different versions of Underscore loaded on the page, living in peace and harmony, in ten different third-party modules. Not that you should. But that you could.

Re: How I want to write Node: Stream all the things

#45

Post by nailer »

jkrems wrote: post #37
nailer wrote: post #31

Earlier quoted context omitted.

From Great British Node Conf, maybe two months ago: a speaker talking about promises asked what people use for flow control: - promises: about 20% of the room. - async: about 80% of the room For me async.waterfall([list of functions]) is little nicer than 10 chained .thens(). And people advocating promises still keep saying it keeps things flat. No it doesn't, we're already flat because we're all using async. Stop pr…

async: async.waterfall([ fn1, fn2, fn3 ], function(err, result) { }); Q: resultP = [ fn1, fn2, fn3 ].reduce(Q.when, void 0); resultP .then(function(result) {}); .catch(function(err) {}); If you chain thens in Q, you are not doing it right (imho).

Q docs show both. Either way: .waterfall() is still simpler.

Re: How I want to write Node: Stream all the things

#46

Post by nailer »

phleet wrote: post #38
nailer wrote: post #31

Earlier quoted context omitted.

From Great British Node Conf, maybe two months ago: a speaker talking about promises asked what people use for flow control: - promises: about 20% of the room. - async: about 80% of the room For me async.waterfall([list of functions]) is little nicer than 10 chained .thens(). And people advocating promises still keep saying it keeps things flat. No it doesn't, we're already flat because we're all using async. Stop pr…

Promises aren't just about keeping things flat. The biggest value to me is being able to avoid the passing around of callbacks and them relying on varying conventions (some async are function(args, ..., callback(err, data)), some are function(args, ..., callback(data), errback(err)), some are function({success: callback, error: errback})). Promises solve this problem by not passing around callbacks _at all_. Instead…

I can install pretty much anything from npm at this point and put money on it being a single callback, err first.

Re: How I want to write Node: Stream all the things

#47

Post by spion »

nailer wrote: post #43
spion wrote: post #40

Earlier quoted context omitted.

var Promise = require('bluebird'); var fs = Promise.promisifyAll(require('fs')); Promise.spawn(function* () { var data = yield fs.readFileAsync(somefile); }); Documentation: https://github.com/petkaantonov/bluebird/blob/master/API.md My favorite example is doing a diff using an async diff service which provides a function `svc.diff(string1, string2)`. But also imagine that you need to preprocesses the files using the…

- Why BlueBird? What's wrong with Q? - Isn't data a scope down? - https://github.com/petkaantonov/bluebird/blob/master/API.md is API based, not task based. async is API based too, but the API has names like 'waterfall' and 'parallel'. I can click them because I know what they mean. Promises just makes me feel like I'm reading about and endless series of abstractions.

There is nothing wrong with Q, but Bluebird is a bit more node-oriented and also has really, really low CPU/memory overhead (lower than caolan's async). Also it provides the best debugging experience, period - because of its long stack traces spanning multiple previous async events.

I didn't quite understand the comment about data being a scope down. What do you mean?

Yes, promises do have quite a steep learning curve :/ However they're a lot more flexible than a utility grab-bag of functions that never quite fit the problem you're having. By that I mean I often had to massage my functions (by creating new closures or using bind etc) to make them fit the signature that async requires.

I wrote a bunch of examples for common tasks here - http://promise-nuggets.github.io/

Re: How I want to write Node: Stream all the things

#48

Post by nailer »

spion wrote: post #47
nailer wrote: post #43

Earlier quoted context omitted.

- Why BlueBird? What's wrong with Q? - Isn't data a scope down? - https://github.com/petkaantonov/bluebird/blob/master/API.md is API based, not task based. async is API based too, but the API has names like 'waterfall' and 'parallel'. I can click them because I know what they mean. Promises just makes me feel like I'm reading about and endless series of abstractions.

There is nothing wrong with Q, but Bluebird is a bit more node-oriented and also has really, really low CPU/memory overhead (lower than caolan's async). Also it provides the best debugging experience, period - because of its long stack traces spanning multiple previous async events. I didn't quite understand the comment about data being a scope down. What do you mean? Yes, promises do have quite a steep learning curv…

+1 examples.

For me, nearly everything .waterfall(), .each(), or .parallel(), and has been for two years now.

Re: How I want to write Node: Stream all the things

#49

Post by [deleted] »

ryanatkn wrote: post #14
hamburglar wrote: post #10

This is highly intriguing, but I must ask why the JS community has this fascination with obscure identifiers like "_". It decreases readability when what should be a logically-chosen descriptive identifier for your class is replaced with a single character that visually recedes into the language syntax. Edit: I've grudgingly given jquery a pass on this because of its ubiquity, but come on, a stream library? Not to me…

JS devs rely heavily on libraries to do things that most programmers would expect the language to handle. $ and _ have been adopted as toolkit identifiers and are littered throughout most JS codebases. Highland does much of what one would expect from _ via Underscore/Lodash. I'd bet most people think they improve readability.

[deleted]

Re: How I want to write Node: Stream all the things

#50

Post by [deleted] »

gruseom wrote: post #29
hamburglar wrote: post #22

Earlier quoted context omitted.

Naming is important for code readability. I think a lot of people recognize that. Which is why I'm so baffled by this trend of "screw it, I won't even bother with a name! Call everything underscore!"

Please don't bikeshed in technical threads. One comment on something like this is more than enough; zero is probably better. When a new thing comes out, the discussion should focus on what's significant about it.

[deleted]
Post reply on HN