I was curious about the memory usage for Elixir. It’s the second time I’ve read (without details).
Ditching Go for Node.js
41–50 of 185 posts
Re: Ditching Go for Node.js
#42Why two (or three) go routines per connection? Why not one net socket (for reads) and two channels (one for writes, other for pub/sub) and select() between them? It seems like the OP is trying too hard to avoid event loops.
Re: Ditching Go for Node.js
#43This is a very interesting direction to take. I've built a lot of my personal stuff on JS, and TBH, the one thing I really wish I had right now was a statically typed codebase. I spend a lot of time thinking about why I'm creating a certain data model, whether I might need to change something in future, etc. About 60% of my productive time is spent thinking about how and why, so I hardly refactor. However, when the n…
You have two excellent options:
TypeScript and PureScript.
Even better, they play fairly well together. You can work in Purescript but still provide well typed integration points for contributors that can't.
And uh, no one here is talking about how fantastically slow Go channels are. But I just saw the code last night, and it's not hard for 20 year old techniques to beat out a "futex for literally every message send" technique.
Re: Ditching Go for Node.js
#44OP is complaining about Goroutine stack size at 4kb per connection, his test shows that Node8 is taking up to 150MB of memory with 5k users, 4kb*5k = 20MB for Go memory, I don't understand how Nodejs can take less memory than Go, and without real numbers / test I'm pretty sure he's doing something wrong somewhere. From my experience on some large prod deployment, Nodejs app takes much more memory and CPU vs Go app fo…
It's 2 per connection, so that's 40MB of RAM. The OP also said that was minor compared to the channel overhead -- it seems the number of channels was exponential to the number of users in a room.
A fully connected graph of n users contains n*(n-1) links if users don't connect to themselves (I would describe this as polynomial growth in the number of channels, a lot better than exponential). A chat broker that acts as a switchboard between users could, I suppose, reduce this to a linear relationship between users and channels.
Re: Ditching Go for Node.js
#45So, I am not a big fan of Javascript. I do not despise it or anything, I just never got to like it. I guess.
I did really love POE, though, so when I heard of Node.JS, I thought I will probably like this very much.
I am not sure what happened. I think it was the tutorials being always out of date. Node.js seem so be such a fast-moving target. I do not mind asynchronous, callback-driven code. But when a tutorial that was written three months ago fails to run because some library had a breaking API change in between, that tends to drive me away.
Think of Go what you want, but its policy towards backward compatibility is a big plus.
Re: Ditching Go for Node.js
#46Please look into typescript. Makes js development sane.
Re: Ditching Go for Node.js
#47One thing to note is that they were using boltdb, which is an in process K/V store designed for high read loads, and doing a lot of writes to it. boltdb also tends to use a lot of memory, as it's using a mmap'd file. The switch also moved them to sqlite, which I would say is a much better fit for what they are doing, but means a lot of this is an apples and oranges comparison.
Re: Ditching Go for Node.js
#48Some random thoughts as somebody who codes a lot of Node.js at work and a lot Go in my freetime (and sometimes at work): Did you try using pprof and the other tooling go provides to better understand your performance limitations? Tooling is a lot better in go ecosystem for understanding CPU and memory consumption, so if/when you run into into issues with Node you're going to be in a world of pain (this is basically a…
There are quite a few things I like about Go, but it's just so hard to leave the safety and reliability of TypeScript. Go's limited type system is its fatal handicap IMO.
Re: Ditching Go for Node.js
#49One thing to note is that they were using boltdb, which is an in process K/V store designed for high read loads, and doing a lot of writes to it. boltdb also tends to use a lot of memory, as it's using a mmap'd file. The switch also moved them to sqlite, which I would say is a much better fit for what they are doing, but means a lot of this is an apples and oranges comparison.
And now to get ranty:
Boltdb was being handled badly: https://github.com/maxpert/raspchat/blob/79315d861968c126670... You should be using bolt's helper funcs, so you don't forget to close the transaction
https://github.com/maxpert/raspchat/blob/79315d861968c126670... At least this is actually deferred, but still.
https://github.com/maxpert/raspchat/blob/79315d861968c126670... These messages are horrifying, would be much better switching back to json and implementing a strategy like http://eagain.net/articles/go-dynamic-json/
It copies in an unlicensed "snowflake generation" thing? https://github.com/maxpert/raspchat/blob/79315d861968c126670... and then essentially relicenses it? That's illegal...?
They do have a better json pattern for some stuff: https://github.com/maxpert/raspchat/blob/79315d861968c126670... but I'd still steer away from reflection based, there's so much better ways of doing this stuff, like so: http://eagain.net/articles/go-json-kind/
Re: Ditching Go for Node.js
#50So, I am not a big fan of Javascript. I do not despise it or anything, I just never got to like it. I guess. I did really love POE, though, so when I heard of Node.JS, I thought I will probably like this very much. I am not sure what happened. I think it was the tutorials being always out of date. Node.js seem so be such a fast-moving target. I do not mind asynchronous, callback-driven code. But when a tutorial that…
You're comparing core libraries of Go with non-core libraries of Node.js. Since Go libraries are not versioned without a tool like dep, you're putting trust that master doesn't have breaking changes vs hoping the maintainer follows SemVer correctly. In fact, even the core Go team has problem with this and has to revert changes to things under golang.org/x/ when they break backwards compatibility.