Live data from Hacker News

The Way of the Gopher: Making the Switch from Node.js to Golang

medium.com

71–80 of 189 posts

Re: The Way of the Gopher: Making the Switch from Node.js to Golang

#71
post #42

Earlier quoted context omitted.

what about just using the built in cluster functionality already in nodejs? I've used it many times with great results and it is really easy to implement.

The clustering module on NodeJs cannot compete with speed that threading, or STM, or Actors offer. The thing with NodeJs is, it is not suited for CPU bound workloads. It's a great fit for I/O bound workloads (where most of the hardwork is done by, say, a DB). Process to process communication has tremendous cost, although shared-memory does make that faster, but at the cost of complicating the interaction. Message-que…

I would say the most natural is STM - you don't need to change much, just wrap your concurrent, shared memory accessing code in a STM transaction, be careful with IO (or use a language which prohibits IO in a STM block like Haskell) and you are done.

Actors are pretty nice when it comes to distributed services, and Erlang/OTP is pretty much uncontested here.

Threads with manual locking are only useful when speed is really important, but this is somewhat of a moot point when it comes to concurrency. Both STM and Actors perform slower (starvation, message copying overhead).

Here [0] is a nice, in-depth discussion about some of the different approaches to concurrency.

[0] https://plus.google.com/109566665911385859313/posts/FAmNTExS...

Re: The Way of the Gopher: Making the Switch from Node.js to Golang

#73

I'm disappointed by the overall tone here. It's as if this were some kind of failure that everybody is piling up on. - She made three passes to fix the existing code base. - After that didn't work, with the CTO's involvement, this Go solution was devised. - It took only 2 weeks to ramp up with Go and build a replacement that solved the problem. - The number of instances needed dropped from 4 to 2 This sounds like a r…

[deleted]

Re: The Way of the Gopher: Making the Switch from Node.js to Golang

#74
> for each request, Octo typically fetches somewhere between 10–100 keys from S3

I'm really curious as to why this is neccessary - what are the 10-100 keys you are fetching per request ? Just reading the high level article it sounds like there is some issue with what this service is doing, irrespective of the language it's doing it in.

Re: The Way of the Gopher: Making the Switch from Node.js to Golang

#75
post #47

Earlier quoted context omitted.

> Node.js on the other hand doesn't need any marketing/propaganda; it just sells itself. If Node.js used language X,Y,Z instead of Javascript, it would certainly not have been the success it was. "Javascript on the server" was the marketing/propaganda . Nodejs came at the right moment, Js was exploding, websockets were exploding, coffeescript and co were exploding in popularity. What Go does better than Nodejs is a b…

I agree with almost all of your post except this: > With Go it doesn't matter if an operation is blocking or non blocking, that fact can totally be abstracted from the client code. No, it can't, and pretending that it can is misleading in a way that allows large teams of developers to cause themselves real problems. The only sense in which this is true is that you can write a Go function with a simple signature like…

Can you please explain why, exactly, I would want to take a simple function like "DoSomeStuff" and make it non-blocking with futures in Go?

Are you sure you're not just explaining how to write a Node program in Go? Write Node programs in Node.

Re: The Way of the Gopher: Making the Switch from Node.js to Golang

#78
post #41

Earlier quoted context omitted.

Totally agree! I would say the same thing if someone had a whole app written on Python and were having performance issues (I definitely wouldn't advise them to switch to Node.js or Golang) - If your ecosystem offers the tools to solve the problem, it's better to use these.

Why on earth are both of you getting downvoted so hard? I am so tired of reading hacker news articles with comments that are so pushed down because of subjective opinionated voting that is so obviously skewed towards one personality type. I have so many thoughts and opinions I want to contribute to these discussions but I have zero confidence it will fit within this type of environment that's been created. It really…

pg believes downvoting is an appropriate expression of disagreement: https://news.ycombinator.com/item?id=117171

Re: The Way of the Gopher: Making the Switch from Node.js to Golang

#79
post #58
post #57

Earlier quoted context omitted.

> The caller can make function "non-blocking" by wrapping the call in a goroutine themselves. Sure, but if they want the return value then either they need to construct the Future-y wrapper I just described or they need to assemble it together in a collection of other function calls wrapped inside a function that itself is either Future-y or uses a long-lived channel to communicate results. It is not novel to build u…

Go isn't just a threaded concurrency model, it uses an M:N greenthreads pattern. Also, when you say that Go I/O operations are blocking, it is true that they'll logically block a goroutine. However, under the hood, it uses the same libuv-style async IO (or IOCP on Windows) that Node does. An operating system thread doesn't get blocked; the goroutine is "shelved" and woken up again when the I/O is complete. It accompl…

I believe that depending on the system call the thread handling the call could block, but it's not the same thread developer Goroutines are running on. Yeah though, same as nodejs.

Re: The Way of the Gopher: Making the Switch from Node.js to Golang

#80
post #69
post #67

I lol'd at the bit about blocking Node's event loop. If Unix weren't so committed to doing I/O the stupid way around, maybe Node wouldn't have to provide fragile, ersatz asynchronicity the way that it does?

Care to explain?

Two iron-clad rules of data processing:

* All data has a type

* All I/O is asynchronous and interrupt (event) driven

At its inception Unix doubled down on stupid, first by encouraging data to be stored and migrated in the form of untyped text streams, which require ad-hoc parsing and unparsing logic at the endpoints; and secondly by failing to provide asynchronous I/O primitives to user space. POSIX later provided an AIO standard, but it was cumbersome to use, bolted on rather than integrated into the OS, and virtually nobody used it. What you want to be able to do is submit I/O requests to the kernel, and have the kernel notify you when they are completed while you wait or do other processing; you could even put the process to sleep until pending I/O is completed and have it take up no CPU during this time. Instead what people do is burn CPU cycles in select/poll loops, which are the I/O equivalent of asking "Are we there yet? Are we there yet? Are we there yet?" over and over. You can fake asynchronicity by doing a bit of processing before asking "are we there yet?" again, but you have to write your processing code in such a manner as to be done piecewise in a loop, and the lower latency you want the more frequently you have to poll (and the more CPU you have to burn polling). This is how Node does "asynchronous" I/O; the VM stops every few instructions to poll for ready FDs, dispatches to callbacks as necessary, and performs parts of pending large I/O operations which can be done non-blocking. (That's the other sucky bit: you can't just call read(2) and write(2) for large chunks of data on an fd that's been opened O_NONBLOCK and expect it all to work; you have to keep reading or writing in a loop when the fd is ready, subtracting the number of bytes successfully written until the entire buffer is processed.)

But if something -- a call into a C library for instance -- stops the VM from doing the poll and I/O bits of its main loop, all I/O simply... stops. And your throughput goes into the toilet. Whereas if the runtime had been based on an OS that natively supports interrupt-driven AIO -- like VMS, Windows, or AmigaOS -- it would be extremely difficult to stall the I/O pipeline completely this way. You might be able to stall further I/O calls with a really long operation, but any pending calls already initiated would complete, and the runtime would be properly notified of their completion.

And all this stems from the fact that Unix was the Node.js of its day -- an environment designed to make things easy for casual programmers, not to do things correctly.

Post reply on HN