Learn how to design large-scale systems • Hacker News Board

Re: Learn how to design large-scale systems

#141

Post by mindcrime » Sat, Jul 14, 2018, 12:35 AM UTC

yread wrote: post #116

lalwanivikas wrote: post #84

Earlier quoted context omitted.

you need to provide more details to get any useful advice. but just based on what you have described, any db would do the job. add a caching layer and you have your low latencies. again, what is the traffic and bandwidth load like? peak and average values? what kind of data are you planning to store? small values but huge volumes or the opposite? a lot will change based on your system requirements.

To clarify: let's say I have servers in two locations A and B that are 200ms from each other. When I issue a write to the db in A I don't want to wait (multiples) of the 200ms before it returns. I don't really care whether the write appears to a reader at B in 5s or 50 minutes but of course the writes have to be at least causally consistent. I won't have millions (realistically not even thousands) of users and the da…

Consider Couchbase. It uses a combination of asynchronous writes and automatic replication to do a pretty good job of giving low latency writes even at high volume, while also ensuring data integrity. And since reads are served from the cache if possible, you usually get really good read performance as well.

Re: Learn how to design large-scale systems

#142

Post by NightlyDev » Sat, Jul 14, 2018, 12:51 AM UTC

mozumder wrote: post #37

What kind of numbers are they talking about for it to be "large-scale"? One well designed fast app server can serve 1000 requests per second per processor core, and you might have 50 processor cores in a 2U rack, for 50,000 requests per second. For database access, you now have fast NVMe disks that can push 2 million IOPS to serve those 50,000 accesses. 50,000 requests per second is good enough for a million concurre…

And 1K rps/core isn't something that's necessarily hard to achieve, if someone thinks otherwise.

I'm seeing about twice that on higly dynamic PHP pages with ~10 read/writes from/to MariaDB(running on the same machine).

Re: Learn how to design large-scale systems

#143

Post by dmarlow » Sat, Jul 14, 2018, 12:55 AM UTC

amorphid wrote: post #138

ris wrote: post #130

I'm quite tired of everyone wanting to build "large scale systems" and play at being Netflix. The truth of the matter is the vast vast majority of people will never need to do this with their project and instead will just end up making an expensive to maintain mess with way too many moving parts. At least as important as designing something that can scale up is designing something that can scale down . You don't know…

I never thought about scaling down as a skill until just now. I kind of assumed "scaling up" implied up && down, our maybe "scaling out" implied out && in. Interesting thought.

It is indeed interesting to consider things like connection draining and playing nicely with the LB. Even in scenarios where machines are just removed for non-scheduled reasons.

Re: Learn how to design large-scale systems

#144

Post by bovermyer » Sat, Jul 14, 2018, 1:03 AM UTC

You know what hasn't been done? A blog post about how to make a service that fulfills the needs of most people most of the time.

All of the online and print material about such things focus on how to achieve massive scale correctly. Don't get me wrong; this is valuable and, generally, sound advice.

However, it also ignores the majority of use cases for software.

I would love to see a blog post here from someone who has solved a very specific problem for a very small audience, and gotten a very enthusiastic response. That would be meaningful on a larger scale for me.

Re: Learn how to design large-scale systems

#145

Post by maliker » Sat, Jul 14, 2018, 1:08 AM UTC

ris wrote: post #130

I'm quite tired of everyone wanting to build "large scale systems" and play at being Netflix. The truth of the matter is the vast vast majority of people will never need to do this with their project and instead will just end up making an expensive to maintain mess with way too many moving parts. At least as important as designing something that can scale up is designing something that can scale down . You don't know…

It has given me peace of mind at work a couple times when I thought or said to management: “just bring the cluster down to 1 node, you can still support X users, and your server bill will be $500/year”

Still probably designed more system than needed (cluster). But scarier than that was seeing some DC/OS apps with $5,000/month in server costs even without user load.

Re: Learn how to design large-scale systems

#146

Post by 0xFACEFEED » Sat, Jul 14, 2018, 1:17 AM UTC

bovermyer wrote: post #144

You know what hasn't been done? A blog post about how to make a service that fulfills the needs of most people most of the time. All of the online and print material about such things focus on how to achieve massive scale correctly. Don't get me wrong; this is valuable and, generally, sound advice. However, it also ignores the majority of use cases for software. I would love to see a blog post here from someone who h…

This is because the people solving real world problems aren't writing books/tutorials/guides.

Real world system design is dirty. Mostly this is due to constraints (time, cost, etc). And no one starts with zero architecture and 10 million users.

Guides like this serve no purpose other than to fatten vocabularies and promote the "brand" of people who aren't actually doing the work (speakers, educators, etc).

Re: Learn how to design large-scale systems

#147

Post by NightlyDev » Sat, Jul 14, 2018, 1:26 AM UTC

I find it fun to thinker with high performance and high scalability designs, but I, as most others, have no need for it.

Start out small, make efficient systems and have scalability in the back of your head when doing so. Don't do as so many others: "Oh, this lib seems popular, let's just use that! Heck, the cart sometimes takes 8 minutes to load, we need to add more nodes on AWS!"

Yeah, stuff like that happens.

At least in my book optimization usually beats scalability as the place to start for more performance.

Re: Learn how to design large-scale systems

#148

Post by rhizome » Sat, Jul 14, 2018, 1:52 AM UTC

eanzenberg wrote: post #67

tw1010 wrote: post #22

Earlier quoted context omitted.

While I agree with you, a lack of knowledge about this stuff will unfortunately not help you pass any interviews.

Disagree! Myself (and I hope most companies) will jump at the devs who build simple solutions that work.

Sometimes it's a good idea for a sysadmin to know how to decompose something complicated into something simpler.

Re: Learn how to design large-scale systems

#149

Post by rhizome » Sat, Jul 14, 2018, 1:54 AM UTC

a012 wrote: post #62

sillysaurus3 wrote: post #44

Earlier quoted context omitted.

As of 2015: https://news.ycombinator.com/item?id=9222006 FreeBSD 10.1 Nginx 1.7.10 Racket 6.1.1 with some HN and FreeBSD specific patches. 2x 3GHz Intel Xeon-IvyBridge (E5-2690-V2-DecaCore) 8x16GB Kingston 16GB DDR3 2Rx4 SuperMicro X9DRI-LN4F+_R1.2A Adaptec 71605 9x 1000GB Western Digital WD RE4 2x 200GB Smart XceedIOPS SSD

This "single box" has a lot of computing units as on AWS EC2.

The figure I saw around maybe 2015 was that AWS (maybe Heroku) is 10x more expensive and bare metal is 10x more efficient/faster, lending a 100x difference in price-performance. Both factors are probably larger nowadays.

Re: Learn how to design large-scale systems

#150

Post by s-shellfish » Sat, Jul 14, 2018, 2:10 AM UTC

amorphid wrote: post #138

ris wrote: post #130

I'm quite tired of everyone wanting to build "large scale systems" and play at being Netflix. The truth of the matter is the vast vast majority of people will never need to do this with their project and instead will just end up making an expensive to maintain mess with way too many moving parts. At least as important as designing something that can scale up is designing something that can scale down . You don't know…

I never thought about scaling down as a skill until just now. I kind of assumed "scaling up" implied up && down, our maybe "scaling out" implied out && in. Interesting thought.

Being able to whittle down and simplify is an excellent skill to learn as a developer. It's my favorite and the one I find most fun.

It allows everyone to focus on their specific components without leaping ahead in assumptions about how each developer will use each piece in the future. Lots of those kinds of problems are more easily solved in a room together, planned out, and done together. At least, that's what I've learned from how NASA developed their most important, complex parts.

It's very easy to get ahead of oneself. Complexity grows by factors that are incredibly difficult to manage. Being able to simplify down to a context of parts that are moving and parts that are stable is a serene state of coding. Everything flows much easier that way.

There will likely always be bugs and issues, but minimizing them to the smallest number there can be is an ideal value to maintain in software development.