Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My £4 a month server can handle 4.2M requests a day

mark.mcnally.je

Post reply on HN
Print view View original

81–90 of 479 posts

Re: My £4 a month server can handle 4.2M requests a day

#81

Post by reilly3000 »

That estimate can be verified with load testing systems like Artillery. My theory is that things would break far sooner than estimated along the following lines:

- Too many WSGI connections if the timeouts aren’t tweaked

- Too many database connections, especially without caching and tuning

- on the Apache side if MaxRequestWorkers isn’t set there will be memory issues with 1GB RAM

- the disk could easily hit IOPS limits, especially if there is a noisy neighbor

It’s not likely all or any of these things will hit IRL, but that all depends on traffic and usage patterns. It matters not, if you were getting 4.2 M requests each day you’d be in the Alexa Top 1000 and could probably shell out for the $8 server :)

Re: My £4 a month server can handle 4.2M requests a day

#82

Post by fabian2k »

fbrchps wrote: post #69
fabian2k wrote: post #52

Earlier quoted context omitted.

The important message there is that if you can change your problem from serving slow dynamic content to serving static content you can gain enormous performance benefits. Whether that means actually using static sites for stuff that can be static or just properly caching expensive things. Even dynamic content doesn't have to be slow, but many CMS are seriously inefficient without a cache. I'm not really blaming the C…

We're using Next.js at my current company with a custom MongoDB based CMS. Next has a thing called Incremental Static Regeneration[0] which allows us to grab the top ~100 pages from the CMS at build time, generate the pages, then cache them for however long we want. The rest of the pages are grabbed when requested, then are cached for the same amount of time. After the time, they're re-grabbed from the DB, then re-ca…

Even database queries aren't that slow on reasonable hardware, as long as the queries are simple. The problem appears once you have dozens of DB queries per page. It's really not a fair comparison to the site this topic is about, but for trivial queries you can easily get a few thousand requests per second out of Postgres on desktop hardware without any real tuning as long as the DB fits into memory.

But static content is of course still much faster and also much simpler.

Re: My £4 a month server can handle 4.2M requests a day

#83

Post by nostrademons »

fizwhiz wrote: post #46
nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

Would you mind describing your stack in more detail? Did you use gRPC with Go?

Sure, startup is defunct now and I think arbitrage & data on centralized exchanges is a dead market now. Wall Street HFTs got into the arbitrage game, and the data sites laypeople actually visit are the ones started in 2014.

Codebase was pure server-side Kotlin running on the JVM. Jackson for JSON parsing, when the exchange didn't provide their own client library (I used the native client libraries when they did). Think I used Undertow for exchange websockets, and Jetty for webserving & client websockets. Postgres for DB.

The threading model was actually the biggest bottleneck, and took a few tries to get right. I did JSON parsing and conversion to a common representation on the incoming IO thread. Then everything would get dumped into a big producer/consumer queue, and picked up by a per-CPU threadpool. Main thread handled price normalization (many crypto assets don't trade in USD, so you have to convert through BTC/ETH/USDT to get dollar prices), order book update, volume computations, opportunity detection, and other business logic. It also compared timestamps on incoming messages, and each new second, it'd aggregate the messages for that second (I only cared about historical data on a 1s basis) and hand them off to a separate DB thread. DB would do a big bulk insert every second; this is how I kept database writes below Postgres's QPS limit. Client websocket connections were handled internally within Jetty, which I think uses a threadpool and NIO.

Key architectural principles were 1) do everything in RAM - the RDS machine was the only one that touched disk, and writes to it were strictly throttled 2) throw away data as soon as you're done with it - I had a bunch of OOM issues by trying to put unparsed messages in the main producer/consumer queue rather than parsing and discarding them 3) aggregate & compute early - keep final requirements in mind and don't save raw data you don't need 4) separate blocking and non-blocking activities on different threads, preferring non-blocking whenever possible and 5) limit threads to only those activities that are actively doing work.

Re: My £4 a month server can handle 4.2M requests a day

#84

Post by jfrunyon »

ijidak wrote: post #72
markandrewj wrote: post #38

Normally benchmarks for things like this are measured in how many concurrent requests can be handled, i.e the C10K problem, not by how many requests you are able to serve in a day. It's also well known that you can serve a large amount of requests on limited hardware. https://en.wikipedia.org/wiki/C10k_problem "By the early 2010s millions of connections on a single commodity 1U rackmount server became possible: over…

How does a single server run millions of active connections? Wouldn't you run out of TCP sockets? What am I missing?

I would guess they may be muxed over fewer sockets, by their LBs, but that's not strictly necessary.

I'm not sure exactly what you mean by "run out of TCP sockets", but theoretically speaking, the only limitation is how much memory is available to store the necessary info about the socket (like address/protocol info and process info).

In practice, OS's do have a "max socket" or "max FD" limit, but that's usually configurable and (with enough RAM) could easily be set to "millions".

Re: My £4 a month server can handle 4.2M requests a day

#85

Post by [deleted] »

great-potential wrote: post #79
polote wrote: post #41

What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app? I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page. 50req/sec for a Html file is super low which shows that a $4 month server cant do much actually…

Exactly, I dont see the point bragging about this nevertheless posting about it on HN ...

[deleted]

Re: My £4 a month server can handle 4.2M requests a day

#86

Post by TruthWillHurt »

polote wrote: post #41

What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app? I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page. 50req/sec for a Html file is super low which shows that a $4 month server cant do much actually…

+1. I remember modest VPS/Parrallels serving PHP at 350r/s

Re: My £4 a month server can handle 4.2M requests a day

#87

Post by [deleted] »

dreyfan wrote: post #76
ijidak wrote: post #72

Earlier quoted context omitted.

How does a single server run millions of active connections? Wouldn't you run out of TCP sockets? What am I missing?

A connection isn't just a dest_port, it's the unique combination of 4 components: source_ip:source_port:dest_ip:dest_port

[deleted]

Re: My £4 a month server can handle 4.2M requests a day

#88

Post by napworth »

polote wrote: post #41

What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app? I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page. 50req/sec for a Html file is super low which shows that a $4 month server cant do much actually…

It's called boasting

Re: My £4 a month server can handle 4.2M requests a day

#89

Post by habibur »

markandrewj wrote: post #38

Normally benchmarks for things like this are measured in how many concurrent requests can be handled, i.e the C10K problem, not by how many requests you are able to serve in a day. It's also well known that you can serve a large amount of requests on limited hardware. https://en.wikipedia.org/wiki/C10k_problem "By the early 2010s millions of connections on a single commodity 1U rackmount server became possible: over…

Right. I calculated what 5m/day converts into. And it's like 60 req/sec. Considering non even distribution and spikes, I would assume its like 200req/sec.

Re: My £4 a month server can handle 4.2M requests a day

#90

Post by danudey »

nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

A lot of that is due to absolutely lousy code.

We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating.

Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did:

    get list of users
    
    foreach user:
        get list of all permissions
        filter down to the ones assigned directly to the user
    
    foreach user:
        get list of all groups
        foreach group:
            get list of all permissions
            filter down to the ones assigned to the group
        filter down to the ones the user has
I'm no algorithm genius, but I'm pretty sure O(n^2+n^3) is not an efficient one.

I replaced it with

    get list of all users
    get list of all groups
    get list of all permissions
Suffice to say, it was a lot more responsive.

Also worth noting was that fetching the user list required shelling out to a command (a python script) which shelled out to a command (ldapsearch), and the whole system was a nightmare. There were also dozens of pages where almost no processing was done in the view, but a bunch of objects with lazy-loaded properties were passed into the template and always used, so when benchmarking you'd get 0.01 seconds for the entire function and then 233 seconds for "return render(...)' because for every single row in the database (dozens or hundreds) the template would access a property that would trigger another SQL call to the backend, rather than just doing one giant "SELECT ALL THE THINGS" and hammering it out that way.

Note that we also weren't using Django's foreign keys support, so we couldn't even tell Django to "fetch everything non-lazily" because it had no idea.

If that app were written right it could have run on a Raspberry Pi 2, but instead there was no amount of cores that could have sped it up.

Post reply on HN