Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stack Overflow is a cacheless, 9-server on-prem monolith

twitter.com

Post reply on HN
Print view View original

101–110 of 120 posts

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#101

Post by cntainer »

ElectricalUnion wrote: post #99
cntainer wrote: post #21

Imagine trying to present this kind of architecture to a room full of executives already sold on the "benefits" of kubernetes, big data, serverless, etc.

What would prevent you from running 9 "web server pods" with 64GB ram each? Just implement the whole thing on top of Kubernetes, why not?

did that before: running a single monolithic app inside a kubernetes cluster on a single pod. I still feel dirty after doing it.

My current hobby is to try and run monolithic apps like these on serverless services like cloud run. There's still some pain related to attaching persistent storage to a container but otherwise it feels like a great option.

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#102

Post by winrid »

jameshart wrote: post #74
winrid wrote: post #62

Earlier quoted context omitted.

You don't, really. You can use Django or Perl today and just enable nginx caching for non authenticated users, for many applications. Stack Overflow didn't need these optimizations. They could have just deployed 20 servers instead and still been profitable. People optimized just because they like to.

Yes, Microsoft SQL Server is famous for its ability to get faster just by adding more servers.

The discussion isn't really around the DB/SQL Server. As far as I could tell, we were discussing .NET and optimizations in its ORM.

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#103

Post by ElectricalUnion »

cosmotic wrote: post #55

It's not cacheless. There are countless caches throughout (including what appears to be ~1TB of memory in the database server), just not a dedicated cache machine.

By this definition almost all non-toy applications under non-toy OSes have caches, because of CPU caches and registers.

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#104

Post by mike_hearn »

mwcampbell wrote: post #100
mike_hearn wrote: post #95

It's a useful reality check. Dedicated machines are fast and you can do a lot without much software complexity. People mention the StackOverflow guys optimizing their software, but their CPU utilization is 5% so they have a lot of headroom to be less optimized. Probably they just enjoyed it and could spend time on that, so why not? At KotlinConf in April I'll be giving a talk on two-tier architecture, which is the St…

On the topic of Postgres versus MS SQL Server or Oracle, I wonder if any of the newer Postgres-compatible databases, like Cockroach or Materialize, solve the scalability issue you raise with Postgres, while not having quite the stigma of MS SQL Server or (especially) Oracle.

I'm not sure. Postgres itself has good performance but the issue for two-tier architecture is number of simultaneous connections. Postgres uses a process per connection. Something like pgbouncer in front can help with that but then the complexity starts going up again, as pgbouncer limits to some extent what you can do. Obviously if you have enough RAM to service all simultaneously connected clients it's not a problem, and you can scale RAM by just adding RO replicas. You can also set connections to aggressively time out if clients are idle, and the clients can re-establish them on demand, so there's lots that can be done.

But ultimately, a db like SQL Server or Oracle will just let you use lots of connections without breaking a sweat. They're both threaded and fully async, it's a much more efficient model.

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#106

Post by sebazzz »

PaulKeeble wrote: post #16

Microservices remains mostly an organisational pattern to scale development teams not necessarily the system performance. Microservices add a lot of complexity and overhead.

Besides, microservices don't guarantee horizontal scaling just like a monolith does not imply no ability to do horizontal scaling.

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#107

Post by kkielhofner »

tiffanyh wrote: post #48
kkielhofner wrote: post #45

Earlier quoted context omitted.

Throwing 1.5TB of RAM in the SQL Server (server) has to help too!

> [1.5TB of RAM] that is a third of the entire Q&A dataset. Yes, but maybe not as much as you’d think. https://twitter.com/sahnlam/status/1629713961951330304

It's probably debatable in this case but from what I know of Postgres (as one example) the general thinking seems to be "throw as much (relatively cheap) RAM at it as you can", tune some of the default (conservative) memory consumption params, and let Postgres eat the RAM.

See the various parameters here[0] - it's complicated but from my understanding you can pretty quickly run into performance issues depending on some not-exactly obvious variables in dataset size, specific queries, etc.

Of course Postgres != SQL Server but the concepts are likely similar. That said you won't catch me every researching this because I've never used SQL Server and never will :).

[0] - https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serv...

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#108

Post by OkayPhysicist »

mike_hearn wrote: post #93
jameshart wrote: post #72

Earlier quoted context omitted.

Great, now you have an nginx reverse proxy as a load balancer in front of a few servers. Now sort out log storage, certificate expiry, access controls, patch management, health monitoring, and remote administration, update it whenever you add or remove backend servers for maintenance, and make sure to synch it up to DNS, and you’ve almost got the same capability as an AWS ELB. Except yours doesn’t have high availabil…

> log storage, certificate expiry, access controls, patch management, health monitoring, and remote administration, etc This is how you can satisfy those needs with stock Linux. Install Ubuntu then: apt-get install certbot unattended-upgrades systemd-journal-remote wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh && sh /tmp/netdata-kickstart.sh Remote admin and access controls are already handled…

To add on to the HA comment: A lot of people have distorted ideas of how much availability they actually need. A lot, if not most, applications could probably get away with the absolutely abysmal 99% uptime, depending on how that downtime was distributed. 99% uptime could mean anything from ~3 days of downtime a year, 7 hours a month, 14 minutes a day, to half a second of unavailability a minute.

Like sure, it's not ideal, but real businesses almost never are. And, as you pointed out, most datacenters get dramatically better uptime than that.

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#109

Post by threeseed »

hedora wrote: post #92
threeseed wrote: post #37

Earlier quoted context omitted.

This comment also says something. You've taken the opinion of one engineer and used it to denigrate the mindset of all modern (young ?) software engineers.

From Britannica: modernity, the self-definition of a generation about its own technological innovation, governance, and socioeconomics. To participate in modernity was to conceive of one's society as engaging in organizational and knowledge advances that make one's immediate predecessors appear antiquated or, at least, surpassed In engineering, etc, these days, it usually refers to the idea that a single solution wor…

You have a very odd interpretation of "modern".

Kubernetes was initially released 9 years ago.

And it's not worlds apart from VM managers like vSphere which is 14 years old.

Re: Stack Overflow is a cacheless, 9-server on-prem monolith

#110

Post by Nextgrid »

ElectricalUnion wrote: post #99
cntainer wrote: post #21

Imagine trying to present this kind of architecture to a room full of executives already sold on the "benefits" of kubernetes, big data, serverless, etc.

What would prevent you from running 9 "web server pods" with 64GB ram each? Just implement the whole thing on top of Kubernetes, why not?

A 64GB RAM instance on a cloud (which is what you're most likely using if you have K8S) will set you back a decent amount of money, even more so if you want one matching the specs that Stack Exchange actually uses.

If you need that level of performance you need to go bare-metal, and this is where you'll hit a lot of roadblocks (yet they will be happy to spend 10-100x more money trying to make do with the cloud).

Post reply on HN