Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“You don't need this overengineered goo for your project.”

twitter.com

Post reply on HN
Print view View original

81–90 of 107 posts

Re: “You don't need this overengineered goo for your project.”

#81

Post by manigandham »

sofixa wrote: post #43
codeulike wrote: post #40

Earlier quoted context omitted.

Here's the stats on Stackoverflow https://stackexchange.com/performance 1.3 billion page views per month, 9 web servers, 4 sql servers Stackoverflow is notable because they went down the C#/MVC/SQL Server route from the start, which meant much better performance per server. Thats why they make an interesting counterexample to the usual way...

> Stackoverflow is notable because they went down the C#/MVC/SQL Server route from the start, which meant much better performance per server And also notable because everything is under an expensive license, so big performant servers is the cheaper option. Edit: everything = Windows servers for their .NET app ( apparently in the process of migrating to .NET Core) and SQL Server

License costs are usually per-core, not per-server. It's the same whether you have a single large server or multiple small ones.

Re: “You don't need this overengineered goo for your project.”

#82

Post by manigandham »

sofixa wrote: post #80
pjmlp wrote: post #74

Earlier quoted context omitted.

License costs are peanuts in traditional enterprise bugdgets, when mapped across the overall costs of developer salaries, contracting, designer agencies, travel expenses,....

Is StackOverflow a traditional enterprise for you? I don't think they have huge travel expenses...

Yes, it's a typical large software/SaaS company.

It's not about the travel specifically, it's that servers and licenses are the cheapest cost of any company when compared to salaries, especially for engineering and sales.

Re: “You don't need this overengineered goo for your project.”

#83

Post by Jnr »

gregoriol wrote: post #78
actually_a_dog wrote: post #70

Earlier quoted context omitted.

You left off 10 servers: 2 Redis servers, 3 tag engine servers, 3 Elasticsearch servers, and 2 HAProxy servers. So, that's 23 servers in total, which is not a trivial amount, but also not a huge number, either.

What is really interesting to me is that they just have a very small number of each kind. It's not like they have 50 redis servers, it's just 2. 2 looks more like a smaller project ar first, but it's great proof that a lot can be achieved by doing engineering.

They are probably able to do it with little because they understand what they are doing.

There are too many developers and teams that don't understand the underlying principles and limitations of the technology they use. In a good case those developers get schooled by experienced engineers, but frequently they end up in an important position in some big enterprise software company and makes life miserable for many people.

Re: “You don't need this overengineered goo for your project.”

#84

Post by szszrk »

slooonz wrote: post #32
szszrk wrote: post #27

Earlier quoted context omitted.

Exactly. That twitter thread is just pure rage based on no data. Sum up resources from that page - we are talking around 6500GB* of RAM worth of servers. That is no homelab. * Maybe a bit more/less, because it's not clear to me if DB RAM is per server, or per cluster. Likely server, as on other servers. There is also no data on how big is their haproxy.

And yet the main point stand : they don't need K8s to manage this application running on 23 servers.

No one needs k8s. Bringing up their infrastructure in a k8s troubleshooting how-to was a weird thing to do in the first place. It's comparing apples and chandelier - makes no sense.

They have a typical vertically scaled infrastructure, most services have just two nodes, one active. The biggest ones are databases which in many companies are handled in "the classic way" anyway. Clearly it's not designed as microservices and doesn't need dynamic automation at all. Why on earth would they even bring k8s up in their plans?

Re: “You don't need this overengineered goo for your project.”

#85

Post by manigandham »

chrismorgan wrote: post #19
DaGardner wrote: post #17

Earlier quoted context omitted.

Yes it does: https://stackexchange.com/performance Pretty impressive I think.

No it doesn’t. From your link: • 9 web servers • 4 SQL servers • 2 Redis servers • 3 tag engine servers • 3 Elasticsearch servers • 2 HAProxy servers That comes to 23. I know “a couple” is sometimes used to mean more than two, but… not that much more than two. “A couple” is just flat-out wrong; I’d guess that he’s misinterpreting ancient figures, taking the figures from no later than about 2013 about how many web ser…

Most of that is extra unused capacity. They've shared their load graphs and past anecdotes where it's clear the entire site runs very lean.

Also 23 is very much a couple for a company and application of that size. It's not uncommon to see several hundred or thousands of nodes deployed by similar sites.

Re: “You don't need this overengineered goo for your project.”

#86

Post by manigandham »

sofixa wrote: post #46
chrismorgan wrote: post #29

Earlier quoted context omitted.

Nevertheless, it is true that Stack Overflow has focused on backend performance and scaled vertically a long way, further than is fashionable. Just not so far as only using two servers for everything .

Because they're constrained by Windows and Microsoft licensing, scaling out was never an easy option for them.

What constraints? Windows Server licenses are bought per-core and the company can easily afford plenty more. This is a non-issue.

Re: “You don't need this overengineered goo for your project.”

#87

Post by krageon »

Cthulhu_ wrote: post #34
corobo wrote: post #10

I love these Twitter takes. They say something to get attention but if you look at it for more than a second it's.. it's just nothing data Comparing a troubleshooting guide to running a site on a couple of servers is a bit too different for me. Compare it to a troubleshooting guide for those two servers, let's see how they stack up. No using any "ask {specific person}" either Don't get me wrong, kubernetes is overkil…

It's taking one source - look at how these people solved it! - and trying to apply it to others. SO is relatively simple; it's basically customized forum software which is a solved problem that has been around for decades. A junior dev can build an alternative, and it can be built using tried and true solutions like MySQL + PHP, which are horizontally scalable with database sharding, read replicas, and maybe stuff li…

> a solved problem

Except 99% of forums run terrible software that doesn't perform, is not easily usable and won't work right on phones. That tells me it's not a solved problem at all.

Re: “You don't need this overengineered goo for your project.”

#89

Post by corobo »

piva00 wrote: post #66
tester34 wrote: post #51

Earlier quoted context omitted.

>A junior dev can build an alternative Of course junior dev can do it, the same way junior dev can make Youtube it'll work as long as there's less than 100 concurrent users on SO and less than 50 4K 20min videos on youtube

Sometimes I question myself if people posting these "a junior dev can build this product" haven't really worked with systems at scale and all of the myriad of issues that scale brings or if they are being the usual hyper-optimistic-dev that isn't considering much past the proof-of-concept point of a product. Either way is quite baffling how common this kind of comment is, almost a decade reading Hacker News and it po…

It also doesn't matter if a junior dev could do it if they just don't do it

If it's so trivial, go be rich!

Re: “You don't need this overengineered goo for your project.”

#90

Post by corobo »

littlestymaar wrote: post #11
monus wrote: post #5

‪That’s not an architecture diagram though, so it doesn’t represent the complexity at all. I’m sure a troubleshooting map for bare linux server wouldn’t be less complicated than that.‬

> I’m sure a troubleshooting map for bare linux server wouldn’t be less complicated than that.‬ Except your k8s runs on a Linux server, so this is just an addition. (Unless you're using a fully managed k8s cloud offering, but then you have an even bigger toubleshooting flowchart to navigate the provider's management interface: at least that's my experience with GKE, maybe Amazon and others are better)

> Except your k8s runs on a Linux server,

Wouldn't it be more likely in this case that the server is built from configs? Ansible or whatever

The troubleshooting for the Linux server side is "spin up a new one and delete the old one"

Post reply on HN