Id like to see memory consumption charts for this. It seems you miss this on all your posts. Not a criticism (and thank you for what you have done), its just something I (and others) would like to see, and if you are running the tests its just another metric to log :D Also, any update on your previous article? https://news.ycombinator.com/item?id=19094233
Achieving 100k connections per second with Elixir
111–117 of 117 posts
Re: Achieving 100k connections per second with Elixir
#112Earlier quoted context omitted.
> I'd love to see data on the average on-call incidents Don't have any hard data to compare but having been involved in debugging running Erlang systems. It's very nice having the ability to restart separate supervisors while the rest of the processes handle requests. Being able to do hot code loading to say fix bugs or add extra logging. And my all time favorite -- live tracing after connecting to a VM's remote shel…
Is there a good write up of how to do that somewhere?
Overall I would say this book is a good start https://www.erlang-in-anger.com/
Supervisors are just a general pattern in Erlang. Any book will have something about it. I like this one: https://learnyousomeerlang.com/supervisors
Restarting frequency and limits are just one of the parameters you specify. So don't need to do anything fancy or special there.
Hot code loading might not be as obvious: http://erlang.org/doc/reference_manual/code_loading.html but is essentially just compiling the module on the same VM version (or close by, no more than 2 version away), copying it to the server in the same path as the original. The original could be save to a backup file. The do `l(modulename)` to load it.
For tracing I recommend http://ferd.github.io/recon/. Erlang in Anger book will also have example of tracing. http://erlang.org/doc/man/dbg.html has some nice shortcuts too, but be careful using it in production is it doesn't have any overload protection. So if you accidentally trace all the messages on all the processes, you might crash your service :-)
Re: Achieving 100k connections per second with Elixir
#113Earlier quoted context omitted.
Is there a good write up of how to do that somewhere?
Which part or just in general ops with Erlang? Overall I would say this book is a good start https://www.erlang-in-anger.com/ Supervisors are just a general pattern in Erlang. Any book will have something about it. I like this one: https://learnyousomeerlang.com/supervisors Restarting frequency and limits are just one of the parameters you specify. So don't need to do anything fancy or special there. Hot code loading…
Thanks!
Re: Achieving 100k connections per second with Elixir
#114I can handle 120k connections per second with my custom made, highly optimized multiprocess C++ server. But the main problem is business logic. Just make 2 SQL queries to MySQL on each HTTP request and look at how it will degrade.
There are simple tricks to make those queries not kill performance. Here is a dumb proof-of-concept I made a few months ago: https://github.com/MatthewSteel/carpool The general idea is combining queries from different HTTP requests into a single database query/transaction, amortising the (significant) per-query cost over those requests. For simple use-cases it doesn't add a whole lot of complexity, can reduce both lo…
Re: Achieving 100k connections per second with Elixir
#115im a simple man. i see Elixir, i upvote. that being said, this article was pretty informative. The bit about the proposed SO_REUSEPORT socket option was really interesting. Really fun to read about performance bottleneck detection and improvement. edit: wow, downvoting for making a simple joke about liking elixir. Cool.
I've found that humor in comments on HN is usually not well received. Not sure why, just an observation.
Maybe we should add something about this to https://news.ycombinator.com/newsfaq.html.