21–30 of 81 posts
They didn't mention Java as a possible solution, even though its GC's are far better than anything else out there. I have nothing against Rust but if I was at a startup I would save my innovation points for where they're mandatory
Not sure why they'd consider Java, given that concern.
Great article and thanks for sharing! There are a couple of things that stand out at me as possible architecture smells (hopefully this comes across as positive constructive criticism :)). As someone who has been developing on the BEAM for long time now, it usually sticks out like a sore thumb any time I see Elixir/Erlang paired with Redis. Not that there is anything wrong with Redis, but most of the time you can sav…
If you're load balancing on IP hash you can set each instance to limit at full rate and not worry about it.
Shared state in rate limiting becomes a bottleneck very quickly. If you're trying to mitigate spam/DDOS you could easily get 100,000 requests a second. You're going to max out your shared state db way faster than 10gig lines
Did you try using the kinesis REST API directly: https://docs.aws.amazon.com/kinesis/latest/APIReference/API_...
Great article and thanks for sharing! There are a couple of things that stand out at me as possible architecture smells (hopefully this comes across as positive constructive criticism :)). As someone who has been developing on the BEAM for long time now, it usually sticks out like a sore thumb any time I see Elixir/Erlang paired with Redis. Not that there is anything wrong with Redis, but most of the time you can sav…
I would push rate limiting to the load balancer, HAProxy or Nginx, but that's just me. If you have a round-robin LB in front you just set each instance to limit at 1/nodes rate, that way you don't have to share any state. If you're load balancing on IP hash you can set each instance to limit at full rate and not worry about it. Shared state in rate limiting becomes a bottleneck very quickly. If you're trying to mitig…
I have never used this product so just speculation. But I imagine there is some sort of auth token that valid agents send to tell Bearer that this is a valid/invalid request so that things can be trivially rejected to mitigate a DoS/DDoS to an extent.
They didn't mention Java as a possible solution, even though its GC's are far better than anything else out there. I have nothing against Rust but if I was at a startup I would save my innovation points for where they're mandatory
They ruled out a language because it had stop the world GC, and even if it removed the bottleneck, it would likely become one later. Java has the same issue. Rust does not. Not sure why they'd consider Java, given that concern.
The other option is to avoid allocating memory at all, which you could do in C/Rust but also in Java. The vast majority of shops given the choice for low/no allocation performance use Java anyways (HFT)
They didn't mention Java as a possible solution, even though its GC's are far better than anything else out there. I have nothing against Rust but if I was at a startup I would save my innovation points for where they're mandatory
It's not like Elixir doesn't benefit from a battle hardened VM since it's older and has been used in these kind of high volume scenarios before Java was.
Sorry, I must be missing something in this blog post because the requirements here sound incredibly minimal. You just needed an HTTP service (sitting behind an Envoy proxy) to process a mere 500 requests/second (up to 1MB payload) and pipe them to Kinesis? How much data preparation is happening in Rust? It sounds like all the permission/rate-limiting/etc happens between Envoy/Redis before it ever reaches Rust? I know…
That sounds like a decent amount of work for a service, and without more detail it's very hard to say whether or not a given level is efficient or inefficient (we don't know exactly what was being done; we can assume that they're using pretty small Fargate instances though since the Node one came in at 1.5G). They also give some number; 4k RPM was their scaleout point for Node (that's not necessarily the maximum, but the point they felt load was sufficiently high to warrant a scaleout; certainly, their graph shows an average latency > 1 second). Rewriting in Rust, that number was raised to 30k RPM; 100 mb of memory, Given all that, it sounds like, yes, GC was the issue (both high memory and CPU pressure), and with the Rust implementation (no GC) they're nowhere near any CPU or memory limit, and so the 30k is likely a network bottleneck.
That said, while I agree that sounds like a terrible metric on the face of it, with what data they've provided (and without anything else), it also sounds like it may be due to they're just operationally dealing with very large amounts of traffic. They may want to consider optimizing the network pipe; not familiar enough with Fargate, but if it's like EC2, there may be a sizing of cpu/memory that also gives you a better network connection (EC2 goes from 1 GBPS to a 10 GBPS network card at one instance type)
Having never dealt with issues relating to garbage collection before, how do you go about diagnosing GC issues in a language where that’s all handled for you?
There are some general tricks that are language-agnostic, like allocating a huge "buffer" object when the app starts, the size of which is some significant portion of the memory you allow the process to use, which always has a reference, then storing references to other objects you need in that big object. In other words, circumvent the garbage collector. Of course, this has its own issues, but I've seen it done in e…
I've only seen this done in Go :)
Earlier quoted context omitted.
They ruled out a language because it had stop the world GC, and even if it removed the bottleneck, it would likely become one later. Java has the same issue. Rust does not. Not sure why they'd consider Java, given that concern.
Shenandoah and ZGC collectors have worse case pauses of ~10ms and average pauses of 0.5ms . The average pause is faster than malloc() sometimes in C, so you won't really be introducing more latency than C does. The other option is to avoid allocating memory at all, which you could do in C/Rust but also in Java. The vast majority of shops given the choice for low/no allocation performance use Java anyways (HFT)
"The vast majority of shops" is an interesting metric given the vast majority had to pick a language before Rust existed. Java and trying to minimize allocations, vs C/C++, I'd probably choose Java too. Java trying to minimize allocations (no way to guarantee you've done it right), vs Rust (which does guarantee no GC)...I'd probably pick Rust.
I'm one of the engineers that worked on this. It was the first Rust production app code I've written so it was a really fun project.