Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Slashdot effect

en.wikipedia.org

Post reply on HN
Print view View original

61–70 of 74 posts

Re: Slashdot effect

#61

Post by nailer »

lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

> Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware.

Back then it was Wordpress/PHP with no caching.

But: you're right.

Once upon a time named Maxim made a web page that pointed out that a Pentium III should be able to handle 10K simultaneous requests, then said Apache was shit, and that he had made his own webserver, and because he was Russian he filled the webpage with animated hammers and sickles.

This was how NGINX was created.

Re: Slashdot effect

#62

Post by esseph »

marcosdumay wrote: post #44
lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

At the time Slashdot was a thing, web servers could serve dozens, maybe a hundred requests per second. Near to the end of (real) Slashdot's life, project 10k was happening. People were rewriting even the Linux process management to make servers perform close to their theoretic capacities.

twitches

Muscle memory still takes me there several times a day...

Re: Slashdot effect

#63

Post by esseph »

jiehong wrote: post #58
ricksunny wrote: post #15

Earlier quoted context omitted.

I still check t regularly alongside HN. Need a counterpoint to HN submission coverage and crowd-editorial zeitgeist.

Maybe lobste.rs?

Mostly HN reposts and extremely limited ability to post there if you don't have Bay Area social circles.

Re: Slashdot effect

#64

Post by Sesse__ »

toast0 wrote: post #59
phkahler wrote: post #41

Earlier quoted context omitted.

The Apache web server was popular because it was one of the first MVP in its category. I think it was weird that the solution developed back then was to figure out load balancing among more servers, rather than optimizing the web server itself.

People did both. And you want load balancing for operations/redundancy anyway. Or because once you get the max requests per sec out of a machine, you can often get a second machine to double rps for much cheaper than a machine that can do double the rps. In the 20-30 year old time frame, easy to obtain servers are one to two cores, often one core per socket. SMP scalability was not a given; you could certainly run on…

> A single core webserver serving static files isn't going to be much faster than 1 ms per file, so you're around 1k rps. And that's assuming you files are all cached.

What “is” is this? Already in 2005, I was serving images at >3k qps on a single core of my server, without any tuning at all. Now we're 20 years later, with much faster TCP stacks and much faster cores.

Re: Slashdot effect

#65

Post by esseph »

hnlmorg wrote: post #40
jdlshore wrote: post #35

Earlier quoted context omitted.

It was a lot like HN, but more curated and with a more complex (and arguably better) moderation system. More focused on Linux and open source than HN, but still a technology news site with comments.

To be honest I never liked the community much. But back then I still stubbornly clinging to smaller, more specialised communities (plus OSNews.com haha)

Very happy OSNews is still around, just wish it had more traffic :(

Re: Slashdot effect

#66

Post by yobbo »

0xbadcafebee wrote: post #25
lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

That's what's possible today. It wasn't back then (see: https://en.wikipedia.org/wiki/C10k_problem ) The other reason is, not every web server, web application, database, etc is optimized for large number of connections. Sometimes they take up too much CPU and memory. Sometimes each page request triggers a connection to a database, so the database connection limit is hit. Sometimes the queries are inefficient and slo…

A 2015 smartphone could be eight 2 GHz cores with 2-4 GB memory and 128 GB SSD. By far faster and more powerful than anything but the heaviest 2000-era servers if that.

Re: Slashdot effect

#67

Post by iso1631 »

storyinmemo wrote: post #51
lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

2003: http://www.geology.smu.edu/~dpa-www/attention_span . 22 Requests / second was a big deal. Perhaps the best benchmark for the maximum potential traffic of the community at that time comes from a Slashdot post by Rob Malda (CmdrTaco) on September 14, 2001 ( https://news.slashdot.org/story/01/09/13/154222/handling-the... ), detailing the site's traffic following the 9/11 attacks. This load on Slashdot itself repre…

I remember the Tuesday well - pretty much every website was down due to the load, only Slashdot remained (with static pages, downloading over my 4kbit/second modem connection)

Re: Slashdot effect

#68

Post by grepfru_it »

aleph_minus_one wrote: post #8
lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

> Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. At that time, it was quite common to start a new thread or even fork a new process per request/connection. Apache was prone to this: > https://www.digitalocean.com/commu…

mpm_worker was notoriously bad back then and would frequently deadlock the entire Apache process. So the default was to either use prefork or some sort of monitoring to restart stuck processes
Post reply on HN