Live data from Hacker News

My £4 a month server can handle 4.2M requests a day

mark.mcnally.je

211–220 of 479 posts

Re: My £4 a month server can handle 4.2M requests a day

#211

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

What Andy giveth, Bill taketh away.[0] I'm more than a little annoyed that so much data engineering is still done in Scala Spark or PySpark. Both suffer from pretty high memory overhead, which leads to suboptimal resource utilization. I've worked with a few different systems that compile their queries into C/C++ (which is transparent to the developer). Those tend to be significantly faster or can use fewer nodes to p…

I get a kick out of stuff like this - I’m mostly an exec these days, but I recently prototyped a small database system to feed a business process in SQLite on my laptop.

To my amusement, my little SQLite prototype smoked the “enterprise” database. Turns out that a MacBook Pro SSD performs better than the SAN, and the query planner needs more tlc. We ended up running the queries off my laptop for a few days while the DBAs did their thing.

Re: My £4 a month server can handle 4.2M requests a day

#212
post #41

What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app? I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page. 50req/sec for a Html file is super low which shows that a $4 month server cant do much actually…

>There is not even mention of a dB query. Did you read the post?

> These benchmarks show that a very cheap server can easily handle 50 requests a minute to a "full stack" website.

I did, and all I see is someone spinning some numbers idly, like, hey, if I can lay 1 brick every second, then with 20000 people we can build a house in one second! So good!

a) entirely and totally lacking in experience running a heavy load website.

b) 50 requests a minute is so atrociously bad, it’s not even worth talking about.

c) there isnt any db load going on here, this is a full page single table query. See https://docs.djangoproject.com/en/3.2/ref/contrib/flatpages/

Sure maybe a db exists, but it’s not relevant when you compare this to the complexity of doing write operations.

Ie. this is some hiiiigh level arm chair commentary right here.

Sure, they’re just talking about their website, but anyone going “oh yeah, look at this, those mangadex guys should learn a thing or two and run it on django”. …has no idea what they’re talking about.

Re: My £4 a month server can handle 4.2M requests a day

#213
post #210

Earlier quoted context omitted.

Thanks. Yeah, I think I used that years ago when I first ran into this problem, and it worked well. Whether one uses an ORM or not, one needs to know how to use one's tools. My problem (not just with Django, but with ORMs in general) is how they make bad code look good. Like the following (I don't know Django well anymore, but something like this): users = User.objects.all() for u in user: print(u.name, len(u.comment…

I agree that this stuff can definitely be handled better. https://github.com/django-query-profiler/django-query-profil... has a neat option for detecting likely N+1 queries. I usually use the Django Debug Toolbar for this. Django's ".only()" method lets you specify just the columns you want to retrieve - with the downside that any additional property access can trigger another SQL query. I thought I'd seen code somew…

Thanks for the thoughtful responses and the link -- Django Query Profiler looks nice!

Re: My £4 a month server can handle 4.2M requests a day

#214
post #107
post #23

Earlier quoted context omitted.

Broadly speaking people on HN have no clue how to setup a performant httpd/app server and are impressed by abysmal performance/cost metrics like this or the MangaDex post. Everything these days is obscured through multiple layers of SaaS offerings and unnecessary bloat like kubernetes. ~10k rps (it was concurrent connections but close enough) was state of the art in 1999. Now 22 years later ~50 rps is somehow impress…

> ~10k rps (it was concurrent connections but close enough) was state of the art in 1999. Now 22 years later ~50 rps is somehow impressive. I honestly don't understand how that can be true. I'm not suggesting you're lying of course, but when you put it this way it's almost like people are actively trying to slow their programs down. I have a few ideas on why that might be the case (switch to slow interpreted language…

It's death by a thousand paper cuts. Lots of things that aren't really that slow in isolation, but in aggregate (or under pressure) they slow down the system and become impossible to measure.

Let's do web development. Since you mentioned payloads: today they're bigger, and often come with redundant fields, or sometimes they're not even paginated! This slows down the database I/O, requires more cache space, slows down the serialisation, slows down compression, requires more memory and bandwidth...

And then you also have the number of requests per page. Ten years ago you'd make one request that would serve you all the data in one go, but today each page calls a bunch of endpoints. Each endpoint has to potentially authenticate/authorise, go to the cache, go to the database, and each payload is probably wasteful too, as in the previous paragraph.

About authentication and authorisation: One specific product I worked on had to perform about 20 database queries for each request just for checking the permissions of the user. We changed the authentication to use a JWT-like token and moved the authorisation part to inside each query (adding "where creator_id = ?" to objects). We no longer needed 20 database queries before the real work.

About 15 years ago I would have done "the optimised way" simply because it was much easier. I would have used SQL Views for complex queries. With ORMs it gets a bit harder, and it takes time to convince the team that SQL views are not just a stupid relic of the past.

Libraries are often an issue that goes unnoticed too. I mentioned serialisation above: this was a bottleneck in a Rails app I worked. Some responses were taking 600ms or more to serialise. We changed to fast_jsonapi and it went to sub-20ms times for the same payload that was 600ms. This app already had responses tailored to each request, but imagine if we were dumping the entire records in the payload...

Another common one is also related to SQL: when I was a beginner dev, our on-premises product was very slow in one customer: some things on the interface were taking upwards of 30 seconds. That wasn't happening in tests or in smaller customers. A veteran sat down by my side and explained query plans, and we brought that number down to milliseconds after improving indexing and removing useless joins.

A few weeks ago an intern tried to put a javascript .sort() inside a .filter() and I caught it. Accidentally quadratic (actually it was more like O(n^4)). He tried to defend himself with a "benchmark" and show it wasn't a problem. A co-worker then ran anonymised production data into it and it choked immediately. Now imagine this happening on hundreds of libraries maintained by voluntaries on Github: https://accidentallyquadratic.tumblr.com

All those things are very simple, and you certainly know all of them. They're the bread and butter of our profession, but honestly somewhere along the way it became difficult to measure and change those things. Why that happened is left as an exercise.

Re: My £4 a month server can handle 4.2M requests a day

#215

I hosted draftsim.com on a $3/month hosting plan for a few years. We served 500GB of data the first month. I imagine that the hosting company lost money on us (but they never called to complain).

Just wanted to thank you for draftsim.com! As someone who got into MTG Arena a few months ago it has been very useful to learn some basics.

Re: My £4 a month server can handle 4.2M requests a day

#216
post #116

Earlier quoted context omitted.

This is just basic use the right tools for the right job 101. You've got what is basically a static website. You want to serve static files. To do that, you use a fast language and/or servers written in those languages. It's something anyone who has done this for any length of time knows, that HN is impressed by this is confusing to some of us. If you were trying to get as little out of your server as possible you'd…

I thought you meant from 10k to 50 rps doing the same work, not that most of the work could be avoided in the first place. > Is this stuff not being learned? I don't know if it is. I recently finished my studies, and most people had no curiosity at all. As in, they learned a framework early, used it everywhere, and got a job using it. I do remember reading a few times on tutorial that you should put a Nginx as revers…

> On the other hand, having the wisdom of knowing what can be static in the first place? I don't think that it's something teached.

I think the trick is realising that reaching for a "programming language" is just one of the tools we have to solve a certain problem, and probably the last one we should reach for! For a stable system, you want less moving parts. A good programmer fights for it.

Can you solve a problem just by storing a JSON file somewhere? Can you solve a problem without a backend? Can you solve a frontend problem with just CSS or just HTML? Can you solve a problem without Javascript? Can you solve a data storage problem with just a database instead of database+Redis? Do you really need a full-fledged web framework where a micro-framework would suffice? Do you need micro services, Kubernetes, containers and whatnot for your site before it gets its first visitor?

I find that a lot of people go for the "more powerful" tool just to cover their asses. They don't want surprises in the future, so they just go for something that will cover all bases. But what you actually want is the things with the least power [1].

Another issue is that intelligent people have an anti-superpower called "rationalisation". They can justify every single decision they make, as misguided as it is. So it doesn't matter if a website could be done with a single HTML file: it is always possible to find a reasonable explanation for why it needed k8s, micro-services and four languages.

[1] Using the least powerful tool also has other advantages, see Tim Berners-Lee "Principle of Least Power" for example - https://blog.codinghorror.com/the-principle-of-least-power/

Re: My £4 a month server can handle 4.2M requests a day

#217

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

How do I get in touch with you? Definitely using more resources than this to process fewer integrations. I’m curious what trade offs you made to enable this.

If you’re anywhere in the US, let me know.

Re: My £4 a month server can handle 4.2M requests a day

#218
post #206

Earlier quoted context omitted.

> now you can get single servers for ~$4K with 4T of RAM Does the $4K include the cost of the RAM? Where can I find these servers? Thanks!

The more I read comments on subjects I am intimately familiar with, the more I realize most people who comment on HN don't really know what they're talking about and mostly make things up. To answer your question, you can't find these servers because they don't exist. A server with 4T of RAM will cost you at a minimum $20,000 and that will be for some really crappy low-grade RAM. Realistically for an actual server th…

> The more I read comments on subjects I am intimately familiar with, the more I realize most people who comment on HN don't really know what they're talking about and mostly make things up.

HN doesn't look exactly like SlashDot, but it's absolutely just like SlashDot.

Re: My £4 a month server can handle 4.2M requests a day

#219

Earlier quoted context omitted.

Could you share some more about that very optimized binary protocol? I know there are ways to be more efficient than JSON but since you call it crappy, your solution must be much much better. Honestly interested to readup more.

It is not "our" protocol, it is protocol designed by exchange and we need to support it, as we can not change it :). Simple binary messages, with binary encoded numbers, etc. No string parsing, no syntax, nothing like this, only bytes and offsets. Think about TCP header, for example. JSON is very inefficient both in bytes (32 bit price is 4 bytes in binary and could be 7+ bytes as string, think "1299.99" for example)…

Thanks for the writeup!

Re: My £4 a month server can handle 4.2M requests a day

#220
All these "X requests per unit time" posts are starting to make me want to break out some of my experimental code... I have some services that can process several million events per second. This includes: compressing the event batch, persisting to disk, validation of business logic, execution of all view updates (state tracked server-side), aggregation and distribution of client update events, etc. These implementations are easily capable of saturating NVMe flash.

If you want to see where the theoretical limits lie, check out some of the fringe work around the LMAX Disruptor and .NET/C#:

https://medium.com/@ocoanet/improving-net-disruptor-performa...

You will find the upper bound of serialized processing to be somewhere around 500 million events per second.

Personally, I have not pushed much beyond 7 million per second, but I also use reference types, non-ideal allocation strategies, etc.

For making this a web-friendly thing: The trick I have found is to establish a websocket with your clients, and then pipe all of their events down with DOM updates coming up the other way. These 2 streams are entirely decoupled by way of the ringbuffer and a novel update/event strategy. This is how you can chew through insane numbers of events per unit time. All client events get thrown into a gigantic bucket which gets dumped into the CPU furnace in perfectly-sized chunks. The latency added by this approach is measured in hundreds of microseconds to maybe a millisecond. The more complex the client interactions (i.e. more events per unit time), the better this works. Blazor was the original inspiration for this. I may share my implementation at some point in the near future.

Post reply on HN