Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MangaDex infrastructure overview

mangadex.dev

Post reply on HN
Print view View original

221–230 of 241 posts

Re: MangaDex infrastructure overview

#221

Post by krick »

jiggawatts wrote: post #129
tristan9 wrote: post #113

Earlier quoted context omitted.

Hi, we're trying to lower the requests:pageview ratio in general, but for what it's worth this article essentially: - ignores the vast majority of "image serving" (most is handled by DDG and our custom CDN) - the JS fragments thankfully should load only on first visit and then get aggressively cached by DDG/your browser One of the pain points is that there are a lot of settings for users to decide what they should or…

Hi, I'm a performance tuning expert, and this thread piqued my interest. The first thing that I noticed is that even with caching enabled, you're loading "too much data". After loading the main page and then clicking one of the tiles, there are several JSON API calls. Here's an example, 195 kB transferred (528 kB size): https://api.mangadex.org/manga/bbaa17c4-0f36-4bbb-9861-34fc8... Oof. Half a megabyte of JSON! Igno…

> Displaying 1 kilobyte of data should take roughly 1 kilobyte of traffic

Is this to be taken literally? I don't consider myself a performance-tuning expert, but I'm not sure how can I make something useful out of this advice. Of course, "the less you transfer, the better" is an obvious thing to say (a bit too obvious to be useful, in fact), but does it really mean I should aspire to transfer only what I'm actually going to display right now? For example, there is a city autocomplete form on the page (well, a couple of thousand relatively short entries). In that case I would probably consider making 1 request to fetch all these cities (on input focus, most likely), instead of making a request to the server on every couple of characters you type. Is it actually a wrong way of thinking?

Re: MangaDex infrastructure overview

#222

Post by toast0 »

Zababa wrote: post #220
bigiain wrote: post #107

Earlier quoted context omitted.

> you have some method to decide for a given key what database machine will have the data Super simple example, splitting there phone book into two volumes, A-K and L-Z. (Hmmmm, is a "phonebook" a thing that typical HN readers remember?) > you can fit a lot of database in 8 TB of ram, and if you don't keep the whole thing in ram, you can index a ton of data with 8 TB of ram, which means sharding can wait. For almost…

> So maybe 12 months worth of a senior engineer's time. In America. When the salaries are 2/3 times lower, people spend more time to use less hardware.

Sharding does take more time, but it doesn't save that much in hardware costs. Maybe you can save money with two 4TB ram servers vs one 8TB ram server, because the highest density ram tends to cost more per byte, but you also had to buy a whole second system. And that second system has follow on costs, now you're using more power, and twice the switch ports, etc.

There's also a price breakpoint for single socket vs dual socket. Or four vs two, if you really want to spend money. My feeling is currently, single socket Epyc looks nice if you don't use a ton of ram, but dual socket is still decently affordable if you need more cores or more ram and probably for Intel sevees; quad socket adds a lot of expense and probably isn't worth it.

Of course, if time is cheap and hardware isn't, you can spend more time on reducing data size, profiling to find optimizations, etc.

Re: MangaDex infrastructure overview

#223

Post by Zababa »

toast0 wrote: post #222
Zababa wrote: post #220

Earlier quoted context omitted.

> So maybe 12 months worth of a senior engineer's time. In America. When the salaries are 2/3 times lower, people spend more time to use less hardware.

Sharding does take more time, but it doesn't save that much in hardware costs. Maybe you can save money with two 4TB ram servers vs one 8TB ram server, because the highest density ram tends to cost more per byte, but you also had to buy a whole second system. And that second system has follow on costs, now you're using more power, and twice the switch ports, etc. There's also a price breakpoint for single socket vs d…

Fair points, I'm just trying to push back a bit against "optimizing anything is useless since the main cost is engineering and not hardware", since this situation depends on the local salaries and in low-inome countries the opposite can be true.

Re: MangaDex infrastructure overview

#224

Post by vymague »

hsn915 wrote: post #204
thoughtFrame wrote: post #182

Earlier quoted context omitted.

I'm also curious about this, since in many areas of computing (not only webdev) the old-school guys take some stuff as so obvious that they don't even bother writing about it or explaining it beyond "This is obviously possible, dude". They know how to achieve this level of performance, but for everyone else, we have to cobble together fragmented insights. So if anyone out there reads this and thinks like the GP, plea…

What topics specifically are you interested in? And where do all the people like you hang out? I'm not an old-school guy by any means .. but I might have something to contribute.

> What topics specifically are you interested in?

Well, for example, what's the old-school alternative to mangadex's solution?

> And where do all the people like you hang out?

We are here on HN.

Re: MangaDex infrastructure overview

#225

Post by [deleted] »

TekMol wrote: post #28

My cheap $20/month VPS serves tens of thousands a user per day without breaking much of a sweat. Using a good old LAMP stack (Linux, Apache, MariaDB, PHP). I don't know how many requests per second it can handle. Trying a guess via curl: time curl --insecure --header 'Host: www.mysite.com' https://127.0.0.1 > test This gives me 0.03s So it could handle about 30 requests per second? Or 30x the number of CPUs? What do…

[deleted]

Re: MangaDex infrastructure overview

#226

Post by bradknowles »

kiba wrote: post #168
arthur_sav wrote: post #165

Earlier quoted context omitted.

> Simple beats complex. > Fewer moving parts equals less overhead. Took me almost a decade to really comprehend this. I used to include all sorts of libraries, try out all the fancy patterns/architectures etc... After countless of hours debugging production issues... the best code i've ever written is the one with the fewer moving parts. Easier to debug and the issues are predictable.

"The best part is no part." is an engineering quote I heard.

Said in a slightly different way: No part is better than no part.

I know I’m not the first to use that phrasing, but I’m not sure where I picked it up. If someone wants to point out the etymology of that type of phrase, I’d be glad to read up on what I’ve forgotten/missed.

Re: MangaDex infrastructure overview

#227

Post by jiggawatts »

krick wrote: post #221
jiggawatts wrote: post #129

Earlier quoted context omitted.

Hi, I'm a performance tuning expert, and this thread piqued my interest. The first thing that I noticed is that even with caching enabled, you're loading "too much data". After loading the main page and then clicking one of the tiles, there are several JSON API calls. Here's an example, 195 kB transferred (528 kB size): https://api.mangadex.org/manga/bbaa17c4-0f36-4bbb-9861-34fc8... Oof. Half a megabyte of JSON! Igno…

> Displaying 1 kilobyte of data should take roughly 1 kilobyte of traffic Is this to be taken literally? I don't consider myself a performance-tuning expert, but I'm not sure how can I make something useful out of this advice. Of course, "the less you transfer, the better" is an obvious thing to say (a bit too obvious to be useful, in fact), but does it really mean I should aspire to transfer only what I'm actually g…

It's an aspirational goal, not a hard rule.

In your case, you're optimising for round-trips, which is also important. As long as you only send the city names instead of a huge blob that also includes a bunch of metadata, you're probably fine.

The most common example of my rule is that I often see SELECT statements on unindexed columns. This means that behind the scenes, the database engine is forced to do a table scan to find the row. If the query uses a wildcard selector, then it is also forced to return all columns, whether they are used by the application or not.

I commonly see scans over 100 MB tables returning 100 KB to the web tier, which then converts this to 200 KB of JSON to show 100 bytes of text to the end user. Simply adding an index to the table allows the database engine to reduce the data it has to process to 10-30 KB. Selecting specific columns can reduce that to a few kilobytes, and likely also shrink the JSON to match. Eliminating the JSON and directly generating the HTML on the server like in the good old days would cut the Internet network traffic down to minimum 100 bytes required also.

Similarly, you often see performance monitoring, logging, or graphing programs store data in fantastic detail and precision. Meanwhile, the graph needs only 16 bits of data, because screens are typically at most a few thousand pixels across in size! A case in point is Microsoft System Center Operations Manager (SCOM), which has a metric write amplification of something like 300:1, which is why it can't log metrics at a usefully high frequency. Not because that's impossible, but because it's wasting the available computer power to an absurd degree. Azure has inherited this code, and then layered JSON on top. (I guess when you bill by gigabytes ingested, the incentives are all wrong.)

Re: MangaDex infrastructure overview

#228

Post by lossolo »

tpetry wrote: post #191
jiggawatts wrote: post #122

Earlier quoted context omitted.

> SSL which eats from the performance budget. That was a short-lived thing, and has now become a myth perpetuated by companies like Citrix and F5 that sell "SSL offload" appliances for $$$. Have you benchmarked the overhead of TLS? In my experience, a single CPU core can easily put out multiple gigabytes of AES-256 (tens of gigabits). This benchmark shows 3 GB/s (24 Gbps) for recent AMD CPUs, and nearly 40 Gbps per c…

The problem with these benchmarks is they measure the bandwidth you can push through an established tls connection. Try to build 2000 new tls connections a seconds (yes many are still active and dont need to be restarted) that is what is the really slow part. Not sending the data over already established channels.

This exactly. Most CPU time in our Haproxy is spent on crypto/new TLS sessions handling thousands of new connections per second.

Re: MangaDex infrastructure overview

#229

Post by fomine3 »

krageon wrote: post #176
nine_k wrote: post #99

Earlier quoted context omitted.

I frankly don't see where containers could lower the performance. Basically a container is a glorified chroot. It has the same networking unless you asked for isolation, then packets have to follow a local (inside the host) route. It has exactly no CPU or kernel interface penalty. Maybe you wanted to say about container orchestration like k8s, with its custom network fabric, etc.

> I frankly don't see where containers could lower the performance. Have you seen most k8s deployments? It's not the containers, it's the thoughtspace that comes with them. Even just using bare containers invites a level of abstraction and generally comes with a type of developer that just isn't desirable.

So it's not due to container, but nowadays "container" in prod tend to mean k8s or similar clustering.

Re: MangaDex infrastructure overview

#230

Post by emi2k01 »

maxk42 wrote: post #145
maxk42 wrote: post #132

Earlier quoted context omitted.

My approach to what?

(1) Simple beats complex. (2) You can spend weeks building complex infrastructure or caching systems only to find out that some fixed C in your equation was larger than your overhead savings. In other words: Measure everything. In other other words: Premature optimization is the root of all evil. (3) Fewer moving parts equals less overhead. (Again: Simple beats complex.) It also makes things simpler to reason about.…

> 12 web servers with separate MySQL instances local to each sharded on primary key IDs.

I don't understand this part. Hopefully you can clarify this to me.

If you're sharding by primary key, doesn't that mean that there's a high chance that the shard in your local DB instance won't have the data the web server is requesting?

I'm not familiar with DB management.

Post reply on HN