My £4 a month server can handle 4.2M requests a day • Hacker News Board

Re: My £4 a month server can handle 4.2M requests a day

#161

Post by Karrot_Kream » Tue, Sep 07, 2021, 11:23 PM UTC

nostrademons wrote: post #108

foobarbazetc wrote: post #78

Earlier quoted context omitted.

Yeah. Now that CPUs are insanely powerful and you have NVMe SSDs etc the bottleneck is always memory.

It's also amazing how much you can fit in RAM if you're careful. I remember ~2007 people were aghast at Facebook's 4T memcached deployment that stored basically everyone's social network posts; now you can get single servers for ~$4K with 4T of RAM. The trick is basically that you have to eschew the last 15 years of "productivity" enhancements. Pretty much any dynamic language is out; if you must use the JVM or .NET,…

Yeah I run mirrors for certain sites in different regions of the world and a trick I often do is scrape the target site and then hold the entire site in memory. Even the top 500 posts of HN and all of the associated comments can fit in < 16 MB of RAM. If you want to serve that up over a socket, it's really fast to grab a pointer to the memory and just write it to the socket. You can get response times which are dwarfed by network latency that way.

Re: My £4 a month server can handle 4.2M requests a day

#162

Post by Foomf » Tue, Sep 07, 2021, 11:25 PM UTC

blacklion wrote: post #101

nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

Crypto markets are very small :-) I'm working and company which process "real" exchanges, like NASDAQ, LSE, and, especially, OPRA feed. We've added 20+ crypto exchanges in our portfolio this year, and all of them are processed on one old server which is unable to process NASDAQ Total View in real-time anymore. On the other hand, whole OPRA feed (more than 5Gbit/s or 65B/day, yes, it is billions, messages of very opti…

I've read your few posts a few times and I'm still not sure why you made your post. You're telling the person that you handle more data than them and thus need more resources than them. Was your goal to smugly belittle them? It's not like they said any problem can be solved on their specific resources.

Re: My £4 a month server can handle 4.2M requests a day

#163

Post by Karrot_Kream » Tue, Sep 07, 2021, 11:26 PM UTC

danudey wrote: post #94

Gepsens wrote: post #49

Earlier quoted context omitted.

I'm running a crypto trading platform I'm developing on 30$ on DigitalOcean. I coded exclusively in Rust and recently added a dynamic interface to python. Today during the BTC crash it spiked at 20k events/s, and that's only incoming data.

> I coded exclusively in Rust This reminds me of back in 2003, a friend of mine worked for an online casino vendor; basically, if you wanted to run an online casino, you'd buy the software from a company and customize it to fit your theme. They were often written in Java, ASP.NET, and so on. They were extremely heavyweight. They'd need 8-10 servers for 10k users. They hogged huge amounts of RAM. My friend wrote the o…

OpenResty [1] is a good mix of these concepts. It serves requests through nginx (which is at its core just a lightweight event loop) and then serves pages through LuaJIT. If you need more speed you could always write an nginx module in C (or in some other language viz the C ABI).

[1]: https://openresty.org/en/

Re: My £4 a month server can handle 4.2M requests a day

#164

Post by onlyrealcuzzo » Tue, Sep 07, 2021, 11:27 PM UTC

danudey wrote: post #90

nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

A lot of that is due to absolutely lousy code. We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating. Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did…

Yeah - you likely want to do this a single simple query - which you can optimize if necessary. an O(N+1) query is bad. An O(N^2) query is something I have rarely seen. Congrats!

Re: My £4 a month server can handle 4.2M requests a day

#165

Post by benhoyt » Tue, Sep 07, 2021, 11:28 PM UTC

danudey wrote: post #90

nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

A lot of that is due to absolutely lousy code. We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating. Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did…

Yeah, I see this a lot. I think it's especially easy to introduce this kind of "accidentally quadratic" behaviour using magical ORMs like Django's, where an innocent-looking attribute access like user.groups can trigger a database query ... access user.groups inside a loop and things get bad quickly.

In the case of groups and permissions there's probably only a few of each, so fetching all of them is probably fine. But depending on your data -- say you're fetching comments written by a subset of users, you can tweak the above to use IN filtering, something like this Python-ish code:

  users = select('SELECT id, name FROM users WHERE id IN $1', user_ids)
  comments = select('SELECT user_id, text FROM comments WHERE user_id IN $1', user_ids)
  comments_by_user_id = defaultdict(list)
  for c in comments:
    comments_by_user_id[c.user_id].append(c)
  for u in users:
    u.comments = comments_by_user_id[u.id]

Only two queries, and O(users + comments).

For development, we had a ?queries=1 query parameter you could add to the URL to show the number of SQL queries and their total time at the bottom of the page. Very helpful when trying to optimize this stuff. "Why is this page doing 350 queries totalling 5 seconds? Oops, I must have an N+1 query issue!"

Re: My £4 a month server can handle 4.2M requests a day

#166

Post by VintageCool » Tue, Sep 07, 2021, 11:29 PM UTC

danudey wrote: post #90

nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

A lot of that is due to absolutely lousy code. We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating. Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did…

I did some work to improve performance on a dashboard several years ago. The way the statistics were queried was generally terrible, so I spent some time setting up aggregations and cleaning that up, but then... the performance was still terrible.

It turned out that the dashboard had been built on top of Wordpress. The way that it checked if the user had permission to access the dashboard was to query all users, join the meta table which held the permission as a serialized object, run a full text search to check which users had permission to access this page, and return the list of all users with permission to access the page. Then, it checked if the current user was in that list.

I switched it to only check permissions for the current user, and the page loaded instantaneously.

Re: My £4 a month server can handle 4.2M requests a day

#167

Post by Aeolun » Tue, Sep 07, 2021, 11:30 PM UTC

Foomf wrote: post #162

blacklion wrote: post #101

Earlier quoted context omitted.

Crypto markets are very small :-) I'm working and company which process "real" exchanges, like NASDAQ, LSE, and, especially, OPRA feed. We've added 20+ crypto exchanges in our portfolio this year, and all of them are processed on one old server which is unable to process NASDAQ Total View in real-time anymore. On the other hand, whole OPRA feed (more than 5Gbit/s or 65B/day, yes, it is billions, messages of very opti…

I've read your few posts a few times and I'm still not sure why you made your post. You're telling the person that you handle more data than them and thus need more resources than them. Was your goal to smugly belittle them? It's not like they said any problem can be solved on their specific resources.

I feel like he’s saying two things.

One, that he’s surprised by how small crypto markets are.

Two, that this one server (or very few server processing) thing scales quite well to billions of messages a day.

I didn’t find any element of smugness here, but maybe I misread the tone.

Re: My £4 a month server can handle 4.2M requests a day

#168

Post by Thaxll » Tue, Sep 07, 2021, 11:35 PM UTC

danudey wrote: post #90

nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

A lot of that is due to absolutely lousy code. We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating. Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did…

Filter in app is rarely the right solution, your data should be organized in a way that you can get what you need in a single query. Reasons:

- it's memory efficient

- it's atomic

- it's faster

Also doesn't LDAP support filtering in query?

Re: My £4 a month server can handle 4.2M requests a day

#169

Post by remram » Tue, Sep 07, 2021, 11:35 PM UTC

ijidak wrote: post #72

markandrewj wrote: post #38

Normally benchmarks for things like this are measured in how many concurrent requests can be handled, i.e the C10K problem, not by how many requests you are able to serve in a day. It's also well known that you can serve a large amount of requests on limited hardware. https://en.wikipedia.org/wiki/C10k_problem "By the early 2010s millions of connections on a single commodity 1U rackmount server became possible: over…

How does a single server run millions of active connections? Wouldn't you run out of TCP sockets? What am I missing?

On my default Ubuntu install, the hard limit for open files of a process is 1048576 (ulimit -Hn). So you you have to run a handful of processes.

Re: My £4 a month server can handle 4.2M requests a day

#170

Post by wyager » Tue, Sep 07, 2021, 11:36 PM UTC

polote wrote: post #41

What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app? I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page. 50req/sec for a Html file is super low which shows that a $4 month server cant do much actually…

My $5/mo server can handle several thousand requests per second. It’s mostly a question of what server software you use. If you use some node, python, ruby thing, it’s going to be slow as shit and need a reverse proxy in front of it. If you use a fast compiled language with a good framework, you can rip through requests no problem.

I tried a bunch of different stuff and ended up using Haskell - all of its popular web libraries are fast as hell. Go was fast but its standard library leaked sockets or I was not cleaning up connections properly or something, and it would tank whenever something went viral. All the popular interpreted language backend I tried were absurdly slow, like tens of RPS.

Source for my current thing is at http://yager.io/Server.hs. It also does all my RSS stuff, image processing for my photo gallery, etc.