51–60 of 81 posts
Sorry, I must be missing something in this blog post because the requirements here sound incredibly minimal. You just needed an HTTP service (sitting behind an Envoy proxy) to process a mere 500 requests/second (up to 1MB payload) and pipe them to Kinesis? How much data preparation is happening in Rust? It sounds like all the permission/rate-limiting/etc happens between Envoy/Redis before it ever reaches Rust? I know…
Earlier quoted context omitted.
> totally unknown in real industry Microsoft, Apple, Amazon, Oxide, Mozilla, Dropbox and CloudFlare would like a word...
Engineering by press release? A fun fact for you: Rust is forbidden at Dropbox for new development.
There is a couple things I see in this post that I wouldn’t do at all, and I maintain a couple services with orders of magnitude higher QPS. I feel that replacing Node.js with any compiled language would have had the same positive effect.
Earlier quoted context omitted.
There are some general tricks that are language-agnostic, like allocating a huge "buffer" object when the app starts, the size of which is some significant portion of the memory you allow the process to use, which always has a reference, then storing references to other objects you need in that big object. In other words, circumvent the garbage collector. Of course, this has its own issues, but I've seen it done in e…
> There are some general tricks that are language-agnostic, like allocating a huge "buffer" object when the app starts I've only seen this done in Go :)
Earlier quoted context omitted.
Ah yes, a 236 word "article", that says to choose "boring old technology" and also to use Rust in the same breath. This article should mention that rust isn't close to ready when it comes to web backends. As much as I love Rust, if I were running a startup or even a decently sized company I would always choose Rails. Now -that- is boring, old... and mature technology. Certain components could get re-written in Rust,…
> This article should mention that rust isn't close to ready when it comes to web backends. Actix-web works just fine. They got a new maintainer team involved that has been spending some time getting rid of all the insane unsafety that was in the code before.
Rails is mature, robust, and has a huge ecosystem with rubygems. Rust (when it comes to web stuff) is not. "it works" does not pass my litmus test. With Actix/Warp I have to implement stuff by hand that either comes by default with rails or already exists in a gem.
I like Rust but I'm not a zealot. People way overestimate performance when they barely have any traffic to begin with as a small startup or even a medium sized company.
You could even use rails, and use Rust to write ruby gems instead of going with actix/warp/etc.
> insane unsafety
This was overblown. Yes, the author didn't respond appropriately, but unsafe isn't inherently dangerous. This is a stupid misconception within the Rust community and caused a lot of unnecessary drama around Actix.
Earlier quoted context omitted.
> That sounds like a decent amount of work for a service 5+ years ago I wrote a real-time transcoding, muxing streaming radio service that did 5000 simultaneous connections with inline, per-client ad spot injection (every 30 seconds in my benchmark). Using C and Lua. On 2 Xeon E3 cores--1 core for all the stream transcoding, muxing, and HTTP/RTSP setup, 1 core for the Lua controller (which was mostly idle). The ceili…
All I'm saying is that without some example of the payloads they're managing, and the logic they're performing, it's hard to say "this is inefficient". And, as I mentioned, if their CPU and memory are both very low, it's likely they're hitting a network (or, yes, OS) limit. I've seen places hit ulimit limits...I've also seen places hit port assignment issues, where they're calling out to a downstream that can handle…
Here's what they're doing:
> Now, when the Bearer Agent in a user's application sends log data to Bearer, it goes into the Envoy proxy. Envoy looks at the request and communicates with Redis to check things like rate limits, authorization details, and usage quotas. Next, the Rust application running alongside Envoy prepares the log data and passes it through Kinesis into an s3 bucket for storage. S3 then triggers our worker to fetch and process the data so Elastic Search can index it. At this point, our users can access the data in our dashboard.
Given their goal and their problems with GC I can tell you right off the bat probably what's the problem with their various architectures from day 1--too much simplistic string munging. If your idea of log ingestion is using in-language regex constructs to chop up strings into pieces, possibly wrapping them in abstract objects, then its predictable you're going to have GC issues, and memory bandwidth issues in general, and poor cache locality in data and code. But 99% of the time this is how people approach the issue.
What a problem like this cries out for is a streaming DFA architecture, using something like Ragel so you can operate on streams and output flat data structures. You could probably implement most of the application logic and I/O in your scripting language of choice, unoptimized GC and all, so long as you're not chopping up a gazillion log lines into a gazillion^2 strings. The latter approach will cause you grief in any language, whether it's JavaScript, Java, Go, Rust or C. The number of objects per connection should be and can be a small N. For example, at 10 distinct objects (incoming connection object, log line, data structure with decomposed metadata, output connection object, etc) per connection times 500 connections, that's 5000 objects per second. Even Python's and Ruby's GC wouldn't break a sweat handling that, even though internally it'd be closer to 10 * (2 or 3) objects.
Here's a big problem today: nobody writes their own HTTP library or JSON library; everybody uses the most popular ones. So right off the bat every ingestion call is going to generate hundreds or thousands of objects because popular third-party libraries generally suck in each request and explode it into huge, deeply nested data structures. Even in Rust. You can't optimize that inefficiency away. No amount of fearless concurrency, transactional memory, fastest-in-the-world hashing library, or coolest regular expression engine can even begin to compensate. You have to avoid it from day 1. But if your expectations about what's possible are wrong (including how tractable it is with some experience), it won't even occur to you that you can do better. Instead, you'll just recapitulate the same architectural sins in the next fastest language.
Earlier quoted context omitted.
> totally unknown in real industry Microsoft, Apple, Amazon, Oxide, Mozilla, Dropbox and CloudFlare would like a word...
Engineering by press release? A fun fact for you: Rust is forbidden at Dropbox for new development.
Sorry, I must be missing something in this blog post because the requirements here sound incredibly minimal. You just needed an HTTP service (sitting behind an Envoy proxy) to process a mere 500 requests/second (up to 1MB payload) and pipe them to Kinesis? How much data preparation is happening in Rust? It sounds like all the permission/rate-limiting/etc happens between Envoy/Redis before it ever reaches Rust? I know…
It took your comment to make me notice it wasn't 30k requests/second but minute instead. 500 requests per second is what I would expect of a default PHP + Apache installation on a small Ubuntu server. I too have a hard time grasping whats special here. For example I saw cached Wordpress setups handle 400 to 500 requests per second. And Wordpress isn't known for performance even with caching plugins.