Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MangaDex infrastructure overview

mangadex.dev

Post reply on HN
Print view View original

111–120 of 241 posts

Re: MangaDex infrastructure overview

#111

Post by starfallg »

radicalbyte wrote: post #103
maxk42 wrote: post #97

I run an Alexa top-2000 website. (Mangadex is presently at about 6000.) I spend less than $250 a month. I have loads and loads of thoughts about what they could be doing differently to reduce their costs but I'll just say that the number one thing Mangadex could be doing right now from a cursory glance is to reduce the number of requests. A fresh load of the home page generates over 100 requests. (Mostly images, then…

Do you manage to get as many buzz-words and OSS products into your system as they do? :) In general the less moving parts you have in a system the more reliable, secure, efficient and cheaper the system becomes. In their case they run a site that is probably under constant attack by the "hired goons", so they're going to need to have more moving parts than others. Plus they will want to optimise for minimal developme…

>In their case they run a site that is probably under constant attack by the "hired goons", so they're going to need to have more moving parts than others.

That's taken care of by the DDoS-Guard system they placed fronting their infrastructure. The design of their system has to take this into account, but that is mainly on a IP and DNS level. The design of their stack behind the loadbalancer is mainly driven by their functional and non-functional requirements, rather than by the need to prevent DDoS attacks.

Re: MangaDex infrastructure overview

#112

Post by maxk42 »

radicalbyte wrote: post #103
maxk42 wrote: post #97

I run an Alexa top-2000 website. (Mangadex is presently at about 6000.) I spend less than $250 a month. I have loads and loads of thoughts about what they could be doing differently to reduce their costs but I'll just say that the number one thing Mangadex could be doing right now from a cursory glance is to reduce the number of requests. A fresh load of the home page generates over 100 requests. (Mostly images, then…

Do you manage to get as many buzz-words and OSS products into your system as they do? :) In general the less moving parts you have in a system the more reliable, secure, efficient and cheaper the system becomes. In their case they run a site that is probably under constant attack by the "hired goons", so they're going to need to have more moving parts than others. Plus they will want to optimise for minimal developme…

lol

> In general the less moving parts you have in a system the more reliable, secure, efficient and cheaper the system becomes.

100% agreed. This is not my first high-traffic site, nor even the highest. (I built the analytics system for a an Alexa top-10 site in 2010, reaching some 30 billion writes / day off of a mere 14 small ec2 instances.) I've never seen a k8s implementation in production that was necessary.

I will note that my Alexa-2k site is also a personal site (no revenue) and under constant attack. In fact we frequently suffer DDOSes that we don't even notice until reviewing the logs later because it doesn't suffer any latency under pressure.

Re: MangaDex infrastructure overview

#113

Post by tristan9 »

maxk42 wrote: post #97

I run an Alexa top-2000 website. (Mangadex is presently at about 6000.) I spend less than $250 a month. I have loads and loads of thoughts about what they could be doing differently to reduce their costs but I'll just say that the number one thing Mangadex could be doing right now from a cursory glance is to reduce the number of requests. A fresh load of the home page generates over 100 requests. (Mostly images, then…

Hi, we're trying to lower the requests:pageview ratio in general, but for what it's worth this article essentially:

- ignores the vast majority of "image serving" (most is handled by DDG and our custom CDN)

- the JS fragments thankfully should load only on first visit and then get aggressively cached by DDG/your browser

One of the pain points is that there are a lot of settings for users to decide what they should or shouldn't see (content rating, original language of origin, search tags, etc) and some are already specifically denormarlized (when querying chapter entities, ES indices for those contain some manga-level properties to avoid needing to dereference that first too) -- however this also makes caching substantially less efficient in many places, alas

Thanks!

Re: MangaDex infrastructure overview

#114

Post by tristan9 »

hsn915 wrote: post #89

I don't understand. Why is 2k requests/sec supposed to be massive? Try this yourself: write a simple web server in Go, host it on a cheap VPS provider, let's say at the option that costs $20/mo. Your website will be able to handle more than 1k/s requests with hardly any resource usage. ok, let's assume you're doing some complicated things. So what? You can scale vertically, upgrade to the $120/mo server. Your website…

> This is great in terms of success as a website, but it's underwhelming in terms of describing a technical problem.

A bit of an intro punchline, even though I agree it admittedly doesn't say much on itself :)

Fwiw most of the work is that there's little "static" traffic going on -- images and cacheable responses are not very CPU intensive to serve -- but what isn't static (which is a good chunk of it) is more problematic, but more to come on these

Re: MangaDex infrastructure overview

#115

Post by radicalbyte »

starfallg wrote: post #111
radicalbyte wrote: post #103

Earlier quoted context omitted.

Do you manage to get as many buzz-words and OSS products into your system as they do? :) In general the less moving parts you have in a system the more reliable, secure, efficient and cheaper the system becomes. In their case they run a site that is probably under constant attack by the "hired goons", so they're going to need to have more moving parts than others. Plus they will want to optimise for minimal developme…

>In their case they run a site that is probably under constant attack by the "hired goons", so they're going to need to have more moving parts than others. That's taken care of by the DDoS-Guard system they placed fronting their infrastructure. The design of their system has to take this into account, but that is mainly on a IP and DNS level. The design of their stack behind the loadbalancer is mainly driven by their…

The layering - defence in depth - is very much a security consideration. Especially if you're building a pure request/response/sync system you need that. Or you decouple with a queue for mutations and avoid a lot of issues.

Re: MangaDex infrastructure overview

#116

Post by radicalbyte »

maxk42 wrote: post #112
radicalbyte wrote: post #103

Earlier quoted context omitted.

Do you manage to get as many buzz-words and OSS products into your system as they do? :) In general the less moving parts you have in a system the more reliable, secure, efficient and cheaper the system becomes. In their case they run a site that is probably under constant attack by the "hired goons", so they're going to need to have more moving parts than others. Plus they will want to optimise for minimal developme…

lol > In general the less moving parts you have in a system the more reliable, secure, efficient and cheaper the system becomes. 100% agreed. This is not my first high-traffic site, nor even the highest. (I built the analytics system for a an Alexa top-10 site in 2010, reaching some 30 billion writes / day off of a mere 14 small ec2 instances.) I've never seen a k8s implementation in production that was necessary. I…

Interesting, wouldn't mind having a chat outside of HN if you're interested (see my profile for mail).

I've spent much of my career working on systems with active users from the hundreds to low thousands, but which process a huge number (50k/sec scale) jobs/tasks.

It's a totally different kettle of fish, and if I'm totally honest I'm shocked at how badly "web" scales and how common these naive and super inefficient implementations are (hint: my bare-metal server from 2005 was faster than expensive cloud VMs).

Recently I've worked on two high-usage systems (one of which was "handling" 30k requests/second for the first couple of week).

Re: MangaDex infrastructure overview

#118

Post by bagels »

sigstoat wrote: post #54
bagels wrote: post #47

Earlier quoted context omitted.

For real, 640 kilobits?

K isn't the abbreviation for kilo, so if you're going to rag on the fellow for the 'b', then you should at least be asking what a Kelvin*bit is.

The question was more about the kilo part, even though I didn't clarify. Seems orders of magnitude too small?

Re: MangaDex infrastructure overview

#119

Post by polote »

maxk42 wrote: post #97

I run an Alexa top-2000 website. (Mangadex is presently at about 6000.) I spend less than $250 a month. I have loads and loads of thoughts about what they could be doing differently to reduce their costs but I'll just say that the number one thing Mangadex could be doing right now from a cursory glance is to reduce the number of requests. A fresh load of the home page generates over 100 requests. (Mostly images, then…

I don't know about you, but they have 42 average Page-views per visit (HN has 3) so Alexa rank is going to be biased

Re: MangaDex infrastructure overview

#120

Post by nvarsj »

germanjoey wrote: post #93

Many manga fans have a love/hate relationship with mangadex. On one hand, it's provided hosting for countless hours of entertainment over the years. Their "v3" version of the site was basically perfect from a usability point of view, to the point that the entire community chose to unite itself under its flag. On the other hand, directly because of the above, their hasty self-inflicted take down earlier this year near…

I didn't get that take at all from the why-rebuild link. It seems reasonable to me - legacy code base, hard to maintain, with security problems that led to the massive data leak a while back. They also don't owe anything to anyone, and as a hobbyist project, they wanted to try something new. I'm impressed as they seem to have managed it - and the new site feels a lot more responsive than the old one.
Post reply on HN