Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My £4 a month server can handle 4.2M requests a day

mark.mcnally.je

Post reply on HN
Print view View original

181–190 of 479 posts

Re: My £4 a month server can handle 4.2M requests a day

#181

Post by unclebucknasty »

danudey wrote: post #90
nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

A lot of that is due to absolutely lousy code. We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating. Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did…

Not sure what the exact use case was (i.e. the output of the filtering) but—from reading the first algo—seems to be something to do with determining group membership and permissions for a user.

In that case, was there a reason joins couldn't be used? As it still seems pretty wasteful (and less performant) to load all of this data in memory and post-process; whereas a well-indexed database could possibly do it faster and with less-memory usage.

Re: My £4 a month server can handle 4.2M requests a day

#182

Post by zetsurin »

Thaxll wrote: post #168
danudey wrote: post #90

Earlier quoted context omitted.

A lot of that is due to absolutely lousy code. We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating. Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did…

Filter in app is rarely the right solution, your data should be organized in a way that you can get what you need in a single query. Reasons: - it's memory efficient - it's atomic - it's faster Also doesn't LDAP support filtering in query?

Given the context, it probably took the poster very little time to implement that fix, without digging into ldapsearch. With massive speedup, for their likely smallish ldap install. Seems like not a bad call at all.

Re: My £4 a month server can handle 4.2M requests a day

#183

Post by wyager »

When my stuff has been shared on HN and Reddit, I’ve seen peaks of around 500 rps (according to google analytics), so if you can hit 1k rps you’re almost certainly ready to weather any kind of viral sharing that a blog might experience. Several krps on cheap VPS hardware is easy with a good compiled language backend (Haskell’s Warp is a good one). If you use a node/python/ruby/other interpreted backend, you will need aggressive caching through a reverse proxy.

Re: My £4 a month server can handle 4.2M requests a day

#184

Post by FridgeSeal »

xfitm3 wrote: post #138
foobarbazetc wrote: post #78

Earlier quoted context omitted.

Yeah. Now that CPUs are insanely powerful and you have NVMe SSDs etc the bottleneck is always memory.

In my experience disk i/o is the biggest bottleneck. It used to be sync()ing writes to disk for strict consistency but that's been pushed down to the DB now. I just looked at my DB systems and CPU is low but disk is nearly pegged. My data sets are far too big to fit into memory/cache. Disk pressure can be alleviated by optimizing queries but it's a game of whack-a-mole. I have exhausted EBS i/o and been forced to res…

Does anyone know if any of the current crop of standard databases make use of things like Direct IO (where available on Linux), I’ve seen some write-ups that indicate you can get eye-wateringly fast performance when combined with an nvme drive.

Re: My £4 a month server can handle 4.2M requests a day

#185

Post by [deleted] »

habibur wrote: post #89
markandrewj wrote: post #38

Normally benchmarks for things like this are measured in how many concurrent requests can be handled, i.e the C10K problem, not by how many requests you are able to serve in a day. It's also well known that you can serve a large amount of requests on limited hardware. https://en.wikipedia.org/wiki/C10k_problem "By the early 2010s millions of connections on a single commodity 1U rackmount server became possible: over…

Right. I calculated what 5m/day converts into. And it's like 60 req/sec. Considering non even distribution and spikes, I would assume its like 200req/sec.

[deleted]

Re: My £4 a month server can handle 4.2M requests a day

#186

Post by planet-and-halo »

tristor wrote: post #150

The comments in this thread surprise me quite a lot. I suppose it shouldn't, but it does. This post + the response calls to the surface how badly basic system operations knowledge is needed in the industry and how much of it is missing from the toolkit of most developers.

Any recommended reading? I've bought a few things in that vein but I'm self-taught and always looking to improve on the ops/perf side.

Re: My £4 a month server can handle 4.2M requests a day

#187

Post by simonw »

benhoyt wrote: post #165
danudey wrote: post #90

Earlier quoted context omitted.

A lot of that is due to absolutely lousy code. We had a system management backend at my last company. Loading the users list was unbearably slow; 10+ seconds on a warm cache. Not too terrible, except that most user management tasks required a page reload, so it was just wildly infuriating. Eventually I took a look at the code for the page, which queried LDAP for user data and the database for permissions data. It did…

Yeah, I see this a lot. I think it's especially easy to introduce this kind of "accidentally quadratic" behaviour using magical ORMs like Django's, where an innocent-looking attribute access like user.groups can trigger a database query ... access user.groups inside a loop and things get bad quickly. In the case of groups and permissions there's probably only a few of each, so fetching all of them is probably fine. B…

Django's prefetch_related mechanism essentially implements the pattern you describe here for you: https://docs.djangoproject.com/en/3.2/ref/models/querysets/#...

Re: My £4 a month server can handle 4.2M requests a day

#188

Post by b9a2cab5 »

mynameisash wrote: post #158
nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

What Andy giveth, Bill taketh away.[0] I'm more than a little annoyed that so much data engineering is still done in Scala Spark or PySpark. Both suffer from pretty high memory overhead, which leads to suboptimal resource utilization. I've worked with a few different systems that compile their queries into C/C++ (which is transparent to the developer). Those tend to be significantly faster or can use fewer nodes to p…

Abstractions almost always end up leaky. Spark SQL, for example, does whole-stage codegen which collapses multiple project/filter stages into a single compiled stage, but your underlying data format still needs to be memory friendly (i.e. linear accesses, low branching, etc.). The codegen is very naive and the JVM JIT can only do so much.

What I've seen is that you need people who deeply understand the system (e.g. Spark) to be able to tune for these edge cases (e.g. see [1] for examples of some of the tradeoffs between different processing schemes). Those people are expensive (think $500k+ annual salaries) and are really only cost effective when your compute spend is in the tens of millions or higher annually. Everyone else is using open source and throwing more compute at the problem or relying on their data scientists/data engineers to figure out what magic knob to turn.

[1]: https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf

Re: My £4 a month server can handle 4.2M requests a day

#190

Post by brokencode »

It’s the database part that gets expensive for web applications. Serving up static web pages is absolutely trivial for modern servers.

The database is also the part that doesn’t easily scale, unless you pick a highly scalable database from the outset, and those have their own complexity and tradeoffs as well.

That’s why I believe every project should start with a bulletproof model of how the database will work first, then fill in the other details from there.

It’s not always as easy as picking Postgres and calling it a day, unfortunately.

Post reply on HN