We had pretty good number of sim.connections on 512MB instance. Unfortunately there is no enough details on methodology you used to compare, so I can't compare the number of clients it can support. I appreciate if you can do it yourself.
Ditching Go for Node.js
31–40 of 185 posts
Re: Ditching Go for Node.js
#32This is a very interesting direction to take. I've built a lot of my personal stuff on JS, and TBH, the one thing I really wish I had right now was a statically typed codebase. I spend a lot of time thinking about why I'm creating a certain data model, whether I might need to change something in future, etc. About 60% of my productive time is spent thinking about how and why, so I hardly refactor. However, when the n…
I've refactored (or should I say, annotated) a >5K LOC node.js app to Typescript. It's an incredibly powerful system. Structural typing gives you 90% of the flexibility of dynamic typing with 90% of the security of classical static typing (of course this is just a feeling - it's not like I'm presenting a scientific result here).
I had more than 90% test coverage, with meaningful tests so I didn't really find too many bugs but I could delete a lot of tests and run-time checks which were making sure stupid input don't cause unexpected behavior.
When I check my commits & logs, LOC/hour didn't change significantly but bugs/month reduced to nearly half if my SQL skills aren't failing me.
Re: Ditching Go for Node.js
#33The reverse @tjholowaychuk ;) (If you're a 10x engineer, check his new tool for 1e10x engineers: https://news.ycombinator.com/item?id=15731936 )
He went meta recently. Made node_modules cleaner in golang https://github.com/tj/node-prune :D
Re: Ditching Go for Node.js
#34Would Matrix be a good fit for this use case? There are a bunch of different server implementations and the (web based) protocol is relatively simple and very well documented.
Last I looked at the reference implementation ("synapse"), the authors hadn't started to optimize its memory usage yet, so it was pretty thrashy on the 512MB ram droplet I ran it on.
Re: Ditching Go for Node.js
#35Earlier quoted context omitted.
It's incredibly easy to throw stuff together in Node.js, it's much harder to maintain over time. TS helps with this but you still run into really hard to diagnose memory leaks (mostly in 3rd party packages written in C++, but sometime in Node core itself). The tooling for these type of issues is really poor in my experience.
My main concern with node is how fast everything moves. Try to maintain a project that started with es5, then migrated to es6 (but some legacy code is still es5), then added some es2017 and now is adding slowly typescript to the monster, and all of this in just 3 years! Things become a mess in no time, tooling changes constantly, and maintaining legacy code while developing new parts with current best practices helps…
I don't get this.
No one forces you to upgrade your perfectly functional es5 code to es2015, es2017, or typescript.
Yes, node moves fast and new features are introduced, but everything is largely backwards-compatible. You don't have to incorporate every shiny new feature, framework, or tooling.
If you do so by choice, this rant is rendered meaningless.
Re: Ditching Go for Node.js
#36Earlier quoted context omitted.
It's incredibly easy to throw stuff together in Node.js, it's much harder to maintain over time. TS helps with this but you still run into really hard to diagnose memory leaks (mostly in 3rd party packages written in C++, but sometime in Node core itself). The tooling for these type of issues is really poor in my experience.
My main concern with node is how fast everything moves. Try to maintain a project that started with es5, then migrated to es6 (but some legacy code is still es5), then added some es2017 and now is adding slowly typescript to the monster, and all of this in just 3 years! Things become a mess in no time, tooling changes constantly, and maintaining legacy code while developing new parts with current best practices helps…
> (...) developing new parts with current best practices (...)
I sincerely suggest that you stop doing this. If JS folks had 10% of the "don't fix what isn't broken" philosophy of the Python developers who were presented with v3 as the future, we wouldn't have any of those "js fatigue" posts.
Re: Ditching Go for Node.js
#37OP is complaining about Goroutine stack size at 4kb per connection, his test shows that Node8 is taking up to 150MB of memory with 5k users, 4kb*5k = 20MB for Go memory, I don't understand how Nodejs can take less memory than Go, and without real numbers / test I'm pretty sure he's doing something wrong somewhere. From my experience on some large prod deployment, Nodejs app takes much more memory and CPU vs Go app fo…
I count three goroutines per connection: One for ChatHandler.Loop (when invoked from ChatService), one for ChatHandler.socketReaderLoop and one for socketWriterLoop.
Each ChatHandler has three (buffered) channels: outgoingInfo.channel, sockChannel, readErrorChannel (not buffered)
There are some things that could cause bottle necks that are not related to goroutines and channels: Every published message calls GroupInfoManager.GetUsers to get a list of all the users in a group. That can be expensive (I don't know if it is or isn't). And then for every user in a channel, their userOutGoingInfo is retrieved.
I would suggest profiling before switching languages.
EDIT: changed "benchmarking" to "profiling" EDIT2: changed "are likely to" to "could" (... cause bottle necks) since I don't know
Re: Ditching Go for Node.js
#38I know very little about both Node and Go (currently learning the latter but haven't done anything really interesting so far :) - but, really it's hard to believe that Elixir/Phoenix would be disregarded so quickly if the crux of the problem is to have good pub/sub support.
Re: Ditching Go for Node.js
#39Some random thoughts as somebody who codes a lot of Node.js at work and a lot Go in my freetime (and sometimes at work): Did you try using pprof and the other tooling go provides to better understand your performance limitations? Tooling is a lot better in go ecosystem for understanding CPU and memory consumption, so if/when you run into into issues with Node you're going to be in a world of pain (this is basically a…
I've found typescript an invaluable tool.
Having worked on large backend codebases in both TS and JS, the difference is stark. The TS API in my case had something like 90% less errors, and those were all subtle bugs in business logic.
On the other hand, the large JS API over time had all sorts of unexpected type errors and undefined behavior. I'm aware that this is anecdotal evidence, but TS is pretty much a no-brainer now on backend.
Re: Ditching Go for Node.js
#40Would Matrix be a good fit for this use case? There are a bunch of different server implementations and the (web based) protocol is relatively simple and very well documented.
Last I looked at the reference implementation ("synapse"), the authors hadn't started to optimize its memory usage yet, so it was pretty thrashy on the 512MB ram droplet I ran it on.
It's still the case, and it requires quite a lot of CPU time too.
There is no way Synapse is a good fit for something like RPi. Maybe when Dendrite is ready, it will make this possible.