31–40 of 241 posts
I had nothing but respect for the whole team. Dedicating their time to build everything from scratch, not to mention that they maintain everything for free.. It's a cool project, not sure if there's a way for anyone to contribute. I"ll join the discord afterwork to see if they need any extra hand. Gee, how do these people find other people online to work on all of the cool projects. I would love to join rather than p…
I've done things at scale (5-10K req/s) on a budget ($1000 USD) and I've done things at much smaller scales that required a much larger budget. _How_ you hit scale on a budget is one part of the equation. The other part is: what you're doing. Off the top of my head, the "how" will often involve the following (just to list a few): 1 - Baremetal 2 - Cache 3 - Denormalize 4 - Append-only 5 - Shard 6 - Performance focuse…
Mike Cvet's talk about Twitter's fan-in/fan-out problem and its solution makes for a fascinating watch: https://www.youtube-nocookie.com/embed/WEgCjwyXvwc
>more than 10 million unique monthly visitors >our ~$1500/month budget I understand not wanting to show ads, but is there no way for the users to contribute to hosting costs?
My cheap $20/month VPS serves tens of thousands a user per day without breaking much of a sweat. Using a good old LAMP stack (Linux, Apache, MariaDB, PHP). I don't know how many requests per second it can handle. Trying a guess via curl: time curl --insecure --header 'Host: www.mysite.com' https://127.0.0.1 > test This gives me 0.03s So it could handle about 30 requests per second? Or 30x the number of CPUs? What do…
Does it serve 20-40 hi resolution images and uploads per user?
But it seems you think I wanted to start a dick measuring contest?
If your question is genuine: I would serve images via a CDN. The above timing is for assembling a page by doing an auth check, a bunch of database queries and templating the result.
Not familiar with the project but it is great to see a counterpart to over-provisioned enterprise infrastructure. $10 in 2021 can do what $100 in 2011 did, what $1000 in 2001 did, and that is not solely due to hardware. Well-designed deployments of K8s, KVM/LXC, Ceph, LBs like this project can handle so much more traffic than poorly configured Wordpress storefronts. They're using battle-tested tech from Redis and Rab…
I loaded the front page of Mangadex and it made 114 web requests including 10 first-party XHR requests, 30(!!!!) Javascript resource requests and somehow 4 font requests, without me interacting with the page. Clicking one of the titles on the front page resulted in nearly 40 additional requests. Perhaps if you are limited by requests per second you could consider how many requests a single user is making per interact…
They're probably more limited by bandwidth than requests per second, but anyway you look the number of requests are still impressive considered the budget. BTW, the site is not just fast: they serve images on high quality (same as the original [1], that can be multiple MBs per page [2]) at an pretty impressive speed too. [1]: before someone asks why they don't optimize the images, this is by design since they want to…
I had nothing but respect for the whole team. Dedicating their time to build everything from scratch, not to mention that they maintain everything for free.. It's a cool project, not sure if there's a way for anyone to contribute. I"ll join the discord afterwork to see if they need any extra hand. Gee, how do these people find other people online to work on all of the cool projects. I would love to join rather than p…
Find cool project. Contribute. :)
My cheap $20/month VPS serves tens of thousands a user per day without breaking much of a sweat. Using a good old LAMP stack (Linux, Apache, MariaDB, PHP). I don't know how many requests per second it can handle. Trying a guess via curl: time curl --insecure --header 'Host: www.mysite.com' https://127.0.0.1 > test This gives me 0.03s So it could handle about 30 requests per second? Or 30x the number of CPUs? What do…
Does it serve 20-40 hi resolution images and uploads per user?
> In practice, we currently see peaks of above 2000 requests every single second during prime time. That is multiple billions of requests per month, or more than 10 million unique monthly visitors. And all of this before actually serving images.
If I am reading that correctly, 2000r/s does not include images, and makes it unclear if $1500/month does.
I've done things at scale (5-10K req/s) on a budget ($1000 USD) and I've done things at much smaller scales that required a much larger budget. _How_ you hit scale on a budget is one part of the equation. The other part is: what you're doing. Off the top of my head, the "how" will often involve the following (just to list a few): 1 - Baremetal 2 - Cache 3 - Denormalize 4 - Append-only 5 - Shard 6 - Performance focuse…
I easily got 3K requests / sec out of my laptop at the same time, and it was not a trivial app!
People's expectations have shifted so much it's absurd. If you look at the TechEmpower benchmarks, ordinary VMs can easily push 100K requests per second, no sweat, even with managed languages.
Trivial stuff like static content being treated as static content (files on the disk!) not as a distributed cache in front of a database can do wonders.
Am I just old and jaded?