Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is everything so scalable?

stavros.io

Post reply on HN
Print view View original

321–330 of 383 posts

Re: Why is everything so scalable?

#321

Post by robotresearcher »

huflungdung wrote: post #94
sgarland wrote: post #47

Earlier quoted context omitted.

THANK YOU. People look at me like I’m insane when I tell them that their overly-complicated pipeline could be easily handled by a couple of beefy servers. Or at best, they’ll argue that “this way, they don’t have to manage infrastructure.” Except you do - you absolutely do. It’s just been partially abstracted away, and some parts like OS maintenance are handled (not that that was ever the difficult part of managing s…

What I say is that we massively underestimate just how fast computers are these days

No worries, another fifteen layers of software abstraction will soak that up pronto.

Re: Why is everything so scalable?

#322

Post by roncesvalles »

hedora wrote: post #141
abujazar wrote: post #9

I've seen my share of insanely over-engineered Azure locked-in applications that could easily have been run on an open source stack on a $20 VM.

But what if payroll grows to 100M internal users?

I sense sarcasm but just to add on, most software problems actually have very predictable natural upper bounds. There are only X number of people in the world growing at a well-defined rate. There are only X number of people in your country, or in your TAM (e.g. if you're a restaurant) etc. This is especially true for B2B.

The need to accommodate runaway scale (unbounded N and unbounded rate of growth of N) is actually quite rare.

Re: Why is everything so scalable?

#323

Post by zem »

as an aside to the last bit about monorepos, from a dev point of view monorepos are pretty awesome. you never need to deal with complicated merge commits and you always have a consistent state of the world, enforced by ci. i would imagine maintaining the tooling and infrastructure to support said monorepo is pretty hard and painful, but as a user I love working in one.

Re: Why is everything so scalable?

#324

Post by selcuka »

treis wrote: post #255
yoyohello13 wrote: post #249

Earlier quoted context omitted.

I worked on a project for my company (a low volume basic web app) and I suggested we could just start the whole thing on one server. They brought on some Azure consultants and the project ballooned out to months of work and all kinds of services. I’m convinced most of the consultants were just piling on services so they could make more money.

I joke with my boss that all our shit ends up running on a single server in some Amazon data center. It's probably not true but if you add up everything we do it's pretty close to one big server.

It's probably not true, but the end result would be the same even if it was true.

Re: Why is everything so scalable?

#325

Post by stavros »

hamburglar wrote: post #226
CaptainOfCoit wrote: post #3

I've seen startups killed because of one or two "influential" programmers deciding they need to start architecturing the project for 1000TPS and 10K daily users, as "that's the proper way to build scalable software", while the project itself hasn't even found product-market fit yet and barely has users. Inevitably, the project needs to make a drastic change which now is so painful to do because it no longer fits the…

I was part of a small team that built a $300M company on Ruby and MySQL that made every scaling mistake you can possibly make. This was also the right decision because it forced us to stay lean and focus on what we needed right now, as opposed to getting starry-eyed about what it was going to be like when we had 10 million users. At every order of magnitude, we had sudden emergencies where some new part of the system…

I also imagine you were pretty agile by not having tons of complexity to grapple with every time you wanted to add a new feature.

Re: Why is everything so scalable?

#326

Post by resonious »

sgarland wrote: post #47
BirAdam wrote: post #29

Just to be honest for a bit here... we also should be asking what kind of scale? Quite a while ago, before containers were a thing at all, I did systems for some very large porn companies. They were doing streaming video at scale before most, and the only other people working on video at that scale were Youtube. The general setup for the largest players in that space was haproxy in front of nginx in front of several…

THANK YOU. People look at me like I’m insane when I tell them that their overly-complicated pipeline could be easily handled by a couple of beefy servers. Or at best, they’ll argue that “this way, they don’t have to manage infrastructure.” Except you do - you absolutely do. It’s just been partially abstracted away, and some parts like OS maintenance are handled (not that that was ever the difficult part of managing s…

I'll play devil's advocate a little bit here. But to be clear, I hate AWS and all of their crazy concepts and exorbitant pricing, so ultimately I think I'm on your side.

OS maintenance honestly is a bit hard for me. I need to know what to install for monitoring, I need to maintain scripts or Ansible playbooks. I need to update these and make sure they don't break my setup.

And the big kicker is compliance. I always work under SOC2, ISO27001, PCI-DSS, HIPAA, you name it. These require even more things like intrusion detection, antivirus, very detailed logging, backups, backup testing, web application firewall. When you just use AWS Lambda with DynamoDB, the compliance burden goes down a lot.

Re: Why is everything so scalable?

#327

Post by stavros »

JohnMakin wrote: post #42

If you’ve ever been in a situation where you do suddenly face scale and have to rip apart a legacy monolith that was built without scale in mind, you’ll chuckle at this article. It’s extremely painful.

I'm not saying "you're never going to scale", I'm saying "you aren't going to scale right now". There's a happy medium between overengineering everything and making everything so shoddily that you can never grow.

Re: Why is everything so scalable?

#328

Post by sgarland »

resonious wrote: post #326
sgarland wrote: post #47

Earlier quoted context omitted.

THANK YOU. People look at me like I’m insane when I tell them that their overly-complicated pipeline could be easily handled by a couple of beefy servers. Or at best, they’ll argue that “this way, they don’t have to manage infrastructure.” Except you do - you absolutely do. It’s just been partially abstracted away, and some parts like OS maintenance are handled (not that that was ever the difficult part of managing s…

I'll play devil's advocate a little bit here. But to be clear, I hate AWS and all of their crazy concepts and exorbitant pricing, so ultimately I think I'm on your side. OS maintenance honestly is a bit hard for me. I need to know what to install for monitoring, I need to maintain scripts or Ansible playbooks. I need to update these and make sure they don't break my setup. And the big kicker is compliance. I always w…

Yes, you need to write Ansible initially. But honestly, it’s not that much for your average application server. Turn on unattended-upgrades with anything critical to your application blacklisted, and you won’t have to touch it other than to bump version pins whenever you make a new golden image.

Re: compliance, other than SOC2 being a giant theater of bullshit, agreed that it adds additional work. My point is that the claims of “not having to manage infrastructure” is highly misleading. You get to skip some stuff, yes, but you are paying through the nose in order to avoid writing some additional config files.

Re: Why is everything so scalable?

#329

Post by rvitorper »

hsn915 wrote: post #13
_ZeD_ wrote: post #6

Honestly, in my experience, the only good reason to have microservices in a "software solution" is to be able to match 1 service -> 1 mantainer/team and have a big (read "nested", with multiple level of middle-managers) group of teams, each that may have different goals. In this way it's very easy to "map" a manager/team to a "place" in the solution map, with very explicit and documented interactions between them

Nice story but the places I've seen that make use of services, there's never a "1 server -> 1 team". It's more like 20 services distributed among 3 teams, and some services are "shared" by all teams

I can relate to this

Re: Why is everything so scalable?

#330

Post by sgarland »

simianwords wrote: post #156
gaoshan wrote: post #124

Earlier quoted context omitted.

Anyone that says, "they don’t have to manage infrastructure" I would invite them to deal with a multi-environment terraform setup and tell me again that about what they don't have to manage.

While terraform is not ideal it is much much more easy to deal with managed services in AWS than to deal with on premises baremetal servers. Most are biased because they like dealing with the kind of issues in on premises. They like dealing with the performance regressions, heat maps, kernel issues etc. Because why not? You are a developer and you need some way to exercise your skills. AWS takes that away and makes y…

I have literally never met a dev who wanted to deal with any kind of infrastructure.
Post reply on HN