Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Slashdot effect

en.wikipedia.org

Post reply on HN
Print view View original

41–50 of 74 posts

Re: Slashdot effect

#41

Post by phkahler »

locknitpicker wrote: post #27
lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

> Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. You're grossly overstating the capacity of a server to meet peak load demands, specially when hit with unexpected load spikes. The slashdot effect was a phenomenon obser…

The Apache web server was popular because it was one of the first MVP in its category. I think it was weird that the solution developed back then was to figure out load balancing among more servers, rather than optimizing the web server itself.

Re: Slashdot effect

#42

Post by standyro »

joshdavham wrote: post #28
virgil_disgr4ce wrote: post #3

Kind of funny to find this on the front page of HN. Makes me wonder what percentage of today's HN readers didn't live through the Slashdot era. (I'm aware it's still around.)

> Makes me wonder what percentage of today's HN readers didn't live through the Slashdot era. I definitely missed out :( What was it like?

Slashdot was somewhere between a well curated tech subreddit and Hacker News.

It had its own in-jokes like: This is finally "the year of the Linux Desktop" and Jonathan "CowboyNeal" Pater, the site's moderator who often posted polls and commented.

In the late 2000s, most migrated their Slashdot reflex to Digg - and then eventually the YC-backed Reddit when there was a disastrous rollout of Digg v4 (in 2010)

Reddit then became less of a technology-focused site as it gained popularity, and HN became the defacto "tech news" aggregator with a well rounded comment section that resembled the early days of Slashdot, Digg, and Reddit.

Re: Slashdot effect

#44

Post by marcosdumay »

lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

At the time Slashdot was a thing, web servers could serve dozens, maybe a hundred requests per second.

Near to the end of (real) Slashdot's life, project 10k was happening. People were rewriting even the Linux process management to make servers perform close to their theoretic capacities.

Re: Slashdot effect

#45

Post by ihaveajob »

joshdavham wrote: post #28
virgil_disgr4ce wrote: post #3

Kind of funny to find this on the front page of HN. Makes me wonder what percentage of today's HN readers didn't live through the Slashdot era. (I'm aware it's still around.)

> Makes me wonder what percentage of today's HN readers didn't live through the Slashdot era. I definitely missed out :( What was it like?

What I really miss was the moderation system. There was a simple 1-5 score and a main trait (insightful, funny, flamebait, underrated, overrated...). To moderate you had to earn points which would you then spend, so careful consideration mattered. The result was that you could filter for 5+Insightful and get the core of the discussion, or 5+Funny and have a good time, etc.

Re: Slashdot effect

#48

Post by babylon5 »

andrewmcwatters wrote: post #39
schmuckonwheels wrote: post #31

No one in 2025 should be allowed to criticize early 2000s web practices as we have come full circle in many ways. They were limited by tools and resources. Now you'll just see some async "API endpoint error -32768 timeout" and 8,000 spinning pinwheels on the page instead of it simply hanging. Because now you need to allocate 650MB RAM so you can output Hello World in some buggy "stack" with "memory safety". The smart…

One of the most popular API frameworks is called “FastAPI” but in any meaningful benchmark it’s one of the slowest commercially used pieces of web software the industry has adopted. There is no hope for large segments of the industry. We actively choose to make bad decisions all the time, and then people rationalize them.

Fast to develop, fast to run under modest to reasonable requirements. Few things need to scale further.

Re: Slashdot effect

#49

Post by jjcm »

lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

You forget that the vast majority of the web in the 2005 era was blogs hosted on wordpress by either service providers or shared hosting providers, neither of which dedicated significant attention to performance. Many of them were hitting quota limits, but many still were truly just going down because of the oversubscribed hardware they were running on. Even an independent server that was properly set up though could hit issues. I had a few of my posts get slashdotted at that time, and the spike was real. Apache hosting wordpress couldn't quite keep up with the spike - I had to use varnish to set up a reverse proxy cache to properly handle the influx.

Re: Slashdot effect

#50

Post by Johnny555 »

lionkor wrote: post #2

Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware. What's going on that's taking down these sites, really, when its not a quota limit?

Oh I remember those days. In our case, it was because we had so much website data that we couldn't keep it all in cache, so when the site became busy enough, performance fell off a cliff as we became disk I/O bound. We had maxed out our main server with 512MB of RAM.

We couldn't afford the new Sun Ultra 2's that would have given us a spacious 2GB of RAM, so we ended up coding a front-end to shard requests across the 2 web servers and used a Sparcstation 5 to run that front-end.

Eventually we rearchitected the entire site to move the static HTML websites into an Oracle database. (by then we had upgraded to a Sun Ultra Enterprise 3000, which could handle up to 6GB of memory, more than we could ever use)

Post reply on HN