Serving a high-performance blog solely from memory, using Rust • Hacker News Board

Re: Serving a high-performance blog solely from memory, using Rust

#11

Post by dec0dedab0de » Mon, Sep 12, 2022, 1:51 PM UTC

Thaxll wrote: post #2

How can it be faster than a static page that is already in memory, the bytes are there you just send them over a socket? Transforming some template to rust code back to string buffer is somehow faster?

How can it be faster than a static page that is already in memory, the bytes are there you just send them over a socket? Transforming some template to rust code back to string buffer is somehow faster?

I don't think the author is claiming it is faster than a static site stored in memory, they're saying it is faster than a traditional static site that loads files from the disk. At least that's how I read it.

Re: Serving a high-performance blog solely from memory, using Rust

#12

Post by jmillikin » Mon, Sep 12, 2022, 1:54 PM UTC

There may be further opportunities for improvement.

Chrome and Curl both report it takes about 1100ms to load the linked page's HTML, split about 50/50 between establishing a connection and fetching content. I'm not sure how the implementation works internally but that seems like a long time for a site served from memory and aiming to be "high-performance". The images bring the total time up to around 5.7s.

As a point of comparison, my site (nginx serving static content, on the 0.25 CPU GCP instance) serves the index page in 250ms. Of that, ~140ms is connection setup (DNS, TCP, TLS). The whole page loads in https://i.imgur.com/X4LDbWj.png

https://i.imgur.com/Ccwzmgz.png

One thing to remember is that when a server like nginx serves static content, it's often serving it from the page cache (memory). The author of Varnish has written at some length about the benefits of using the OS page cache, for example https://varnish-cache.org/docs/trunk/phk/notes.html>. Some of the same principles can be applied even for servers that render dynamically (by caching expensive fragments).

Re: Serving a high-performance blog solely from memory, using Rust

#13

Post by trh0awayman » Mon, Sep 12, 2022, 1:54 PM UTC

I want to see this taken to the logical extreme. A real OS with actual drivers (no unikernel, no virtio) for a small set of hardware that only serves static pages. No need for virtual memory. Just hardcode the blog posts right into the OS and use the most minimal TCP stack you can make.

Re: Serving a high-performance blog solely from memory, using Rust

#14

Post by xena » Mon, Sep 12, 2022, 1:55 PM UTC

jmillikin wrote: post #12

There may be further opportunities for improvement. Chrome and Curl both report it takes about 1100ms to load the linked page's HTML, split about 50/50 between establishing a connection and fetching content. I'm not sure how the implementation works internally but that seems like a long time for a site served from memory and aiming to be "high-performance". The images bring the total time up to around 5.7s. As a poin…

Author here. I wrote that post before I axed the CDN for my blog site itself. It was true at the time of writing, but it is not true anymore because I need to redo the CDN for the blog itself. All the images are CDNed with XeDN though.

Re: Serving a high-performance blog solely from memory, using Rust

#15

Post by jmillikin » Mon, Sep 12, 2022, 1:56 PM UTC

xena wrote: post #9

awercliyuna wrote: post #8

Earlier quoted context omitted.

Just like the Rust executable is still on disk. Sure, if it is running, it is memory mapped, but it can still be paged out. This is not theoretical. In practice, upon request, the probability of finding the static page in cache should be similar to the probability of the executable not being paged out. (That's true as long as the actual data is the same, and the differing factors, like the size of the web server exec…

Author here. That server doesn't have swap enabled. It can't be paged out.

If the executable is on disk, it can be paged out unless you've used mlock() to tell the kernel to keep it resident.

Re: Serving a high-performance blog solely from memory, using Rust

#16

Post by shrubble » Mon, Sep 12, 2022, 1:56 PM UTC

trh0awayman wrote: post #13

I want to see this taken to the logical extreme. A real OS with actual drivers (no unikernel, no virtio) for a small set of hardware that only serves static pages. No need for virtual memory. Just hardcode the blog posts right into the OS and use the most minimal TCP stack you can make.

I think you could do that with redbean, or very nearly so.

Re: Serving a high-performance blog solely from memory, using Rust

#17

Post by filleokus » Mon, Sep 12, 2022, 2:00 PM UTC

trh0awayman wrote: post #13

I want to see this taken to the logical extreme. A real OS with actual drivers (no unikernel, no virtio) for a small set of hardware that only serves static pages. No need for virtual memory. Just hardcode the blog posts right into the OS and use the most minimal TCP stack you can make.

I guess Unikraft is that, kinda? https://github.com/unikraft/app-nginx

Re: Serving a high-performance blog solely from memory, using Rust

#18

Post by xena » Mon, Sep 12, 2022, 2:00 PM UTC

jmillikin wrote: post #15

xena wrote: post #9

Earlier quoted context omitted.

Author here. That server doesn't have swap enabled. It can't be paged out.

If the executable is on disk, it can be paged out unless you've used mlock() to tell the kernel to keep it resident.

Interesting. I've never heard of that happening before. Can you link a reference to where I can find out more about that aspect of the linux memory subsystem?

Re: Serving a high-performance blog solely from memory, using Rust

#19

Post by staticassertion » Mon, Sep 12, 2022, 2:01 PM UTC

trh0awayman wrote: post #13

I want to see this taken to the logical extreme. A real OS with actual drivers (no unikernel, no virtio) for a small set of hardware that only serves static pages. No need for virtual memory. Just hardcode the blog posts right into the OS and use the most minimal TCP stack you can make.

Seems like a case for a unikernel running on bare metal. No copying bytes across kernel/user, no context switching at all.

Re: Serving a high-performance blog solely from memory, using Rust

#20

Post by codeflo » Mon, Sep 12, 2022, 2:03 PM UTC

dec0dedab0de wrote: post #11

Thaxll wrote: post #2

How can it be faster than a static page that is already in memory, the bytes are there you just send them over a socket? Transforming some template to rust code back to string buffer is somehow faster?

How can it be faster than a static page that is already in memory, the bytes are there you just send them over a socket? Transforming some template to rust code back to string buffer is somehow faster? I don't think the author is claiming it is faster than a static site stored in memory, they're saying it is faster than a traditional static site that loads files from the disk. At least that's how I read it.

That “traditional” site doesn’t actually load the data from disk, in practice. It does once, after a reboot, but that’s true for this solution’s executable file as well.