Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is everything so scalable?

stavros.io

Post reply on HN
Print view View original

281–290 of 383 posts

Re: Why is everything so scalable?

#281

Post by closeparen »

eximius wrote: post #231
closeparen wrote: post #219

>Modules cannot call each other, except through specific interfaces (for our Python monolith, we put those in a file called some_module/api.py, so other modules can do from some_module.api import some_function, SomeClass and call things that way. This is a solution to a large chunk of what people want out of microservices. There are just two problems, both of which feel tractable to a language/runtime that really wan…

Visibility systems are great! > If a change module A has a problem, you must roll back the entire monolith, preventing a good change in module B from reaching users. eh. In these setups you really want to be fixing forward for the reason you describe - so you revert the commit for feature A or turn off the feature flag for it or something. You don't really want to be reverting deployments. If you have to, well, then…

There is a point in commit throughput at which finding a working combination of reverts becomes unsustainable. I've seen it. Feature flagging can delay this point but probably not prevent it unless you’re isolating literally every change with its own flag (at which point you have a weird VCS).

Re: Why is everything so scalable?

#282

Post by 8note »

DrScientist wrote: post #27

Isn't it simple as the following? Break your code into modules/components that have a defined interface between them. That interface only passes data - not code with behaviour - and signal the method calls may fail to complete ( ie throw exceptions ). ie the interface could be a network call in the future. Allow easy swapping of interface implementations by passing them into constructors/ using factories or dependenc…

the swap from interface to network call is still non-trivial.

you get to have new problems that are qualitatively different from before like timeouts, which can break the adsumptions in the rest of your code about say, whether state was updated or not, and in what order. you also then get to deal with thundering herds and circuit breakers and so on.

Re: Why is everything so scalable?

#283

Post by mannyv »

bigbuppo wrote: post #247
n_u wrote: post #229

Earlier quoted context omitted.

Could you explain this some more? How are your costs so low in comparison? Are you using serverless?

That's how the cloudy platforms get you. They're very cheap on the low end, until they're not.

No, it's because we changed the way we process our metrics.

Previously we processed our metrics by consolidating them into multidimensional entries on a minute basis.

We moved to single metric second-based collection, because it was getting too complicated to process and because we wanted second-by-second measurement to measure engagement more granularly. That increased our data retention tremendously. We're still under the cost for the other timestream products, but we'll be adjusting how we do that in a quarter or two.

Re: Why is everything so scalable?

#285

Post by mattbillenstein »

I think more engineers would benefit from "you can just build stuff" thinking. Like you don't have to use all the complicated whizz-bang tech that everyone else is using.

You can build a boring backend on Linux VMs without containers using open-source software - it's simpler or at least a different level of complexity compared to the big clouds and orchestration systems like k8s, and honestly, it's just more fun to work on - I almost never write yaml - it's a joy.

I wrote my own deployment system using this idea - machines, roles, software and services that map to those roles, idempotent operations, and a constantly-connected async rpc system to orchestrate it all. Written from scratch in a language I like with a config language I like. My deploys are often So, I guess - just build stuff using simple primitives. Write simple software - modules and functions and a lot of stateless code. Use postgres for persistence - it's really that good. Use nginx and dns load balancing - tried and true simple architecture.

Re: Why is everything so scalable?

#286

Post by Aeolun »

darkwater wrote: post #150
macNchz wrote: post #130

Earlier quoted context omitted.

Working on various teams operating on infrastructure that ranged from a rack in the back of the office, a few beefy servers in a colo, a fleet of Chef-managed VMs, GKE, ECS, and various PaaSes, what I've liked the most about the cloud and containerized workflows is that they wind up being a forcing function for reproducibility, at least to a degree. While it's absolutely 100% possible to have a "big beefy server arch…

You have to remove admin rights to your admins then, because scrappy enough DevOps/platform engineers/whatever will totally hand-edit your AWS infra or Kubernetes deployments. I suffered that first hand. And it's even worse that in the old days, because at least back in the day it was expected.

But then your next deployment goes, and it all rolls back, right?

And then it their fault, right?

I might have mild trauma from people complaining their artisanal changes to our environment weren’t preserved.

Re: Why is everything so scalable?

#287

Post by andrewmcwatters »

ipsento606 wrote: post #67
strken wrote: post #41

Earlier quoted context omitted.

I've seen senior engineers get fired and the business suffer a setback because they didn't have any way to scale beyond a single low spec VPS from a budget provider, and their system crashed when a hall full of students tried to sign up together during a demo and each triggered 200ms of bcrypt CPU activity.

> they didn't have any way to scale beyond a single low spec VPS from a budget provider they couldn't redeploy to a high-spec VPS instead?

[dead]

Re: Why is everything so scalable?

#289

Post by Aeolun »

coder543 wrote: post #164
wongarsu wrote: post #152

Earlier quoted context omitted.

Hetzner [1]. Bandwidth is 1 GBit/s. You can also get 10 GBit/s, that's hidden away a bit instead of being mentioned on the order page [2] 1: https://www.hetzner.com/dedicated-rootserver/matrix-ex 2: https://docs.hetzner.com/robot/dedicated-server/network/10g-...

I have wished for years that Hetzner would offer their bare metal servers in the U.S., and not just Hetzner Cloud.

Here is US Hetzner: https://ioflood.com/

Their prices have come down a lot. I used them when the servers still cost $200 a piece, but their support at the time was fantastic.

Re: Why is everything so scalable?

#290

Post by byroot »

otabdeveloper4 wrote: post #39
CaptainOfCoit wrote: post #3

I've seen startups killed because of one or two "influential" programmers deciding they need to start architecturing the project for 1000TPS and 10K daily users, as "that's the proper way to build scalable software", while the project itself hasn't even found product-market fit yet and barely has users. Inevitably, the project needs to make a drastic change which now is so painful to do because it no longer fits the…

> 1000TPS and 10K daily users That is not a lot. You can host that on a Raspberry Pi.

That entirely depends on what these transactions are meant to do.

I always find these debate weird. How can you compare one app’s TPS with another?

Post reply on HN