Live data from Hacker News

To boldly go where Node man has gone before

blog.jgc.org

81–90 of 124 posts

Re: To boldly go where Node man has gone before

#81
post #47

Earlier quoted context omitted.

I wish Leiningen would also handle bleeding edge dependencies as easily as rebar eg: {deps, [ {quoted, "1.0.3", {git, "git://git.corp.smarkets.com/quoted.erl.git", {tag, "1.0.3"}}}, {proper, ".*", {git, "git://git.corp.smarkets.com/proper.git", {branch, "master"}}} ]}.

This plugin[1] looks like what you're looking for. [1] https://github.com/tobyhede/lein-git-deps

"You will also need to manually add the checked-out project's dependencies as your own (the plugin simply checks out the code, it doesn't recursively resolve dependencies)."

I've seen this plugin before but its not particularly useful without dependency checking. It's a start though. I'll have to look at leiningen to see if it exposes hooks for that sort of thing.

Re: To boldly go where Node man has gone before

#82
post #71
post #64

Earlier quoted context omitted.

So what exactly do you think is awkward about Go's way of working with JSON? Here's a code example from one of my current projects: type UploadProgress struct { Progress int `json:"progress"` } //... // variable progress is of type Progress if json_data, err := json.Marshal(progress); err == nil { w.Header().Set("Content-Type", "application/json") w.Write(json_data) } json.Unmarshal works the same way. IMHO, not exac…

Unless there's something I missed, json.Unmarshal works the same way iff you have a struct that matches the JSON data very closely. If the JSON data is a little more free-form, you're stuck with a map of interface{}. Having to muck about with types is just a little tedious compared to JavaScript where there aren't really any types and anything can concisely be converted to a string. (I did a little project involving…

If your json doesn't represent something that already follows some known structure, you have an opaque data structure that you have to hand write a parser for in any language.

Re: To boldly go where Node man has gone before

#84

Earlier quoted context omitted.

> one of the main draws to Node is the huge community and wealth of awesome modules. Socket.io, Now.js, cradle, redis Really? In my experience, all but the most mature modules are hardly beta-quality and in constant flux. Find a good module for your job? Too bad it only runs on 0.4. Find another to get around that; oops, too bad your version of gcc needs to be patched and re-built. The language and its entire package…

I'm using cluster, crypto, express, memcached, mysql, and mysql-pool in production and they have worked flawlessly for a node deployment that serves a pretty busy JSON API (200-300 reqs a sec on average).

As a counterpoint, I built a relatively large system using Node (this was a couple months ago, to be clear) and had issues with several modules. mysql: cannot handle binary blob columns (this is just now being fixed in an alpha version of the library). mysql: very slow parsing of large responses. aws*: many half built / half broken libraries -- nothing that met our modest needs. request: (http request library) found several issues.

Node.JS has a great community that is writing many great modules, no doubt. But the community is very young and almost by definition many of the modules are immature.

This can be very enjoyable from an engineering perspective (you get to write and hack on things that you would not otherwise), but can also slow down the process of building things since you do end up having to reinvent the wheel at times.

Re: To boldly go where Node man has gone before

#85
post #57

I can personally vouch for the maturity of Node especially Socket.io. We're building a web-based email client that relies heavily on real-time communication with a Node-based email importer. See http://philterit.com . I've found Node to be faster and more lightweight than Rails 3, which we initially used for our prototype. With JavaScript being the lingua franca on the client-side, specializing in it as a team has pe…

Nobody is comparing Node with Assembly. sigh

Re: To boldly go where Node man has gone before

#86
post #2

So does Go's net/http module really provide basically exactly the same programming model as Node? If so, why use Node? One thing the author didn't mention is that Go will use multiple cores in this example, whereas Node is single-threaded. Right?

No, the model is completely different. In Node, you pass in a callback to establish what should be done following the return of a longrunning I/O operation. In Go, you use goroutines to manage the concurrency.

Basically if you say this:

    fn()
The Go runtime will execute the function in its entirety and wait for a return value.

If you say this:

    go fn()
The Go runtime will execute that function in a new goroutine (the return value is discarded) and then immediately proceed to the next line in the current goroutine.

So if you have some callDB() function that performs some long-running i/o, in node, you might do something like this to perform that operation without blocking the surrounding code:

    ...
    callDB(someparam, otherparam, function (result) {
        // do something with result here
    })
    ...
while in Go, you would do something more like this:

    ...
    go func() {
        result := callDB(someparam, otherparam)
        // do something with result here
    }()
    ...
which winds up being more like this

    func superGreat() {
        result := callDB(someparam, otherparam)
        // do something with result here
    }

    go superGreat()
With the benefit being that if you want superGreat to run concurrently, you use the go keyword, and if you want it to be run in a synchronous style, you just call it normally.

The problem with Ruby/Python/etc is that it is awkward to take something that is blocking and make it nonblocking. The problem with node.js is that node.js says "blocking is bad; therefore, never block". Go takes a different approach, in that it makes it obvious to know how to write something so that it does or does not block, so that the developer is free to use whatever I/O paradigm fits the problem at hand.

The net/http package allows you to register a handler, which is a function that takes and http request and writes a response. There is a main request loop that, when receiving a request, will create a new goroutine for each request and execute its handler in its own context. So within your handlers, you mostly just write blocking code, because you're already in your own isolated goroutine, and the only thing you'd be blocking is the processing of the current request. If you want to do something in the background, you run it in a new goroutine. goroutines are very cheap, so you can be quite cavalier about their usage.

If you want to use a callback-passing style, you can do that in Go (because it has function literals and closures and first-class functions and all that), but that's not idiomatic by a longshot.

The absolutist position that node.js takes by saying that "all i/o must be nonblocking" is no better than our previous options of "all i/o is blocking". Sometimes the simplest and most readable solution is simply to block. There is a yin to the yang of blocking.

Re: To boldly go where Node man has gone before

#87
I think that actually the Node example was significantly simpler.. there were no pointers, and no types needed to be specified. Also I can use CoffeeScript with Node. The only way Go is easier to write is if you know Go but don't know Node. I say its easier to learn and use Node/Javascript.

I am surprised that Node performed so well.. I would like to see that same Go benchmark with 4 CPUs vs. Node and also vs. Node with a clustered web server and the code to see how much more code that involves for clustering with Node.

Also I would like to see some Go code and benchmark that reads a file and/or database in the request and works efficiently.

Actually the code for a Node cluster that would use all of the available CPUs for http in CoffeeScript would look like this:

    cluster = require 'cluster'
    http = require 'http'

    numCPUs = require('os').cpus().length
    if cluster.isMaster
      cluster.fork() for i in [1..numCPUs]
    else
      app = http.createServer()
      app.on 'request', (req, res) ->
        res.end "hello world\n"
      app.listen 8000
Maybe someone could take V8, change it so it compiles (immediately) to static code and uses CoffeeScript, and add a way to optionally specify types (in an unambiguous and readable way) and pointers only when you have to.

Re: To boldly go where Node man has gone before

#88
This test shows Node handling 100 requests simultaneously just as easily as handling them separately.

However Go's memory usage increases by about 5X.

So what happens when you reach 1000 simultaneous requests? Which will perform better then?

I think this is called "selection bias."

Re: To boldly go where Node man has gone before

#89

Earlier quoted context omitted.

How's that any faster than npm install express Not only do npm handle versioning for you, you also don't have to remember the host or username. Obviously Go is a younger community, and could make a package manager some day, but I'm puzzled by how you're comparing this positively with npm.

It is faster because you don't have to install npm, the case I was explaining involves someone with no developing skills. And I'm not saying this feature of Go is better than npm, I also use npm and I like it very much. What I like is the fact that Go comes already with this feature and I don't need anything extra.

I believe that npm is now included as part of all node distributions.

Re: To boldly go where Node man has gone before

#90

Earlier quoted context omitted.

I'm using cluster, crypto, express, memcached, mysql, and mysql-pool in production and they have worked flawlessly for a node deployment that serves a pretty busy JSON API (200-300 reqs a sec on average).

As a counterpoint, I built a relatively large system using Node (this was a couple months ago, to be clear) and had issues with several modules. mysql: cannot handle binary blob columns (this is just now being fixed in an alpha version of the library). mysql: very slow parsing of large responses. aws*: many half built / half broken libraries -- nothing that met our modest needs. request: (http request library) found…

In our case, it isn't a large system. Our node production environment serves up a smallish set of API/Ajax requests that are heavily used. With that, we've been able to eliminate more than a few Web servers that otherwise had to load the entire Apache/PHP stack just to serve a small 20-100 byte request.

With your response in mind - I don't see Node being mature enough (yet) to solely support an end-to-end large scale Web property. But, for serving up API content from Memcache/MySQL - it is blazing fast with minimal footprint. On our stack - we run Node right on our existing Apache Web servers (behind haproxy).

Post reply on HN