Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed Systems Reading List

dancres.github.io

Post reply on HN
Print view View original

61–70 of 77 posts

Re: Distributed Systems Reading List

#61

Post by Tomis02 »

tyu2 wrote: post #45
mjb wrote: post #42

Earlier quoted context omitted.

> Which is directly at odds with needing TLA+, because if you do need it, it means the complexity of the algorithms is so great, that you won't be able to keep them in your head and understand every aspect of their performance to make something work well. I don't think this is correct. What TLA+ allows us to do is be more creative in our design and choice of algorithms, while allowing the computer to help us reason a…

That's the thing, performance should dictate the algorithms, not the other way around and TLA+ can't make this process any easier, only harder. I get it's not the case at AWS, where distributed services AWS thinks customers might want is what dictates the choices, but this is an exception, not the rule and unless someone wants to work there they have no reason to be doing it this way, especially not for educational p…

Correctness dictates algorithms. That's what TLA+ helps you with.

Re: Distributed Systems Reading List

#62

Post by zcw100 »

shred45 wrote: post #25

I find "distributed systems" to be a huge source of imposter syndrome. Despite having worked almost exclusively with distributed applications for several years now, it is difficult to consider myself experienced. When I'm asked if I've worked with distributed systems, I don't think they are asking me if I've managed a Hadoop cluster. They are interested in building new applications using some of the primitives discus…

I don't trust anyone who comes back with quick answers without any hesitation or pause. I remember a lecture from a while ago, I can't remember who was giving it but they were someone well known, where they asked people to implement bubble sort. The results that came back ranged from something like 20 lines to 2000 lines and every single one had bugs. I don't know how anyone who works in this industry for any length of time doesn't follow up every answer with, "...but there's probably some problem that I'm not thinking of at the moment".

Re: Distributed Systems Reading List

#63

Post by ruang »

johnnujler wrote: post #50
abledon wrote: post #48

How long would it take to realistically get through all these books? for the average developer... i'm thinking 3-4 years? and this includes taking weekends outside of work to painfully work through every chapter (+ break weekends since working+studying this in evening would probably lead to burnout in 1 year)

I would wager around a year at max. Most materials related to Distributed Systems are information heavy(requiring you to read and remember) as opposed to being math heavy(with the possible exception of graph theory, which usually none of the fundamental texts expect the readers to know). Even many popular papers like raft, paxos, chubby, BitTorrent, gfs etc do not require much mathematical knowledge beyond basic arit…

Is linear algebra useful to learn for distributed systems?

I'm taking graph theory next semester and wondering what else might be useful. I took a distributed systems course already and we used no math at all.

Re: Distributed Systems Reading List

#64

Post by johnnujler »

ruang wrote: post #63
johnnujler wrote: post #50

Earlier quoted context omitted.

I would wager around a year at max. Most materials related to Distributed Systems are information heavy(requiring you to read and remember) as opposed to being math heavy(with the possible exception of graph theory, which usually none of the fundamental texts expect the readers to know). Even many popular papers like raft, paxos, chubby, BitTorrent, gfs etc do not require much mathematical knowledge beyond basic arit…

Is linear algebra useful to learn for distributed systems? I'm taking graph theory next semester and wondering what else might be useful. I took a distributed systems course already and we used no math at all.

Definitely, may be not directly, but it lies at the heart of software engineering(or more appropriately computer science). It helps with thinking differently, for example I don’t know if ndergrad courses teach distributed search algorithms using eigen vectors and values, but it helps to understand invariants and transformations better.

Re: Distributed Systems Reading List

#65

Post by webmaven »

0xbadcafebee wrote: post #57
calcorbin wrote: post #29

Earlier quoted context omitted.

I'm realizing I've only worked in distributed systems as well, but I'd never feel comfortable telling potential employers I'm an expert. Being an expert in distributed systems seems almost too broad. At a high level couldn't it be expertise at integration, accessible logging, and configuration?

Distributed systems research has been going on since the 70's and Unix Neckbeards have probably forgotten more about them than we have learned, so actually I think impostor syndrome is a bit warranted with them. The actual hard stuff is not even these papers, it's the implementations that are way more complex than some algorithm or architectural pattern. Anyone who says "X is better than Y" is fooling themselves beca…

> The only thing you can say for certain is that reducing the amount of components and complexity in the system often results in better outcomes.

No, there are a few other things that you can say for certain:

Watch out for positive-only feedback loops, you absolutely need negative feedback as well - or only. Eg. exponential back-off.

Sometimes, you just need a decentralized solution, rather than a distributed one, and you don't have to have the same answer at every scale (eg. distributed intra-datacenter, decentralized inter-datacenter, or vice-versa).

Loose coupling is your friend.

Sure, add an extra layer of indirection, but you probably need to pay more attention to cache invalidation than you think.

Throughput probably matters more than latency.

Reducing the size/number of writes will probably help more than trying to speed them up.

Multi-tenancy is a PITA for systems in general, and distributed ones are no exception (aside: there is probably a huge business for multi-tenancy-as-a-service, if anyone manages to solve it in a general-purpose way), but a series of per-customer single-tenant deployments may be worse, especially if they are all on different versions of the code. Here be dragons.

Don't overthink it. Start with a naive implementation and go from there (see loose coupling above).

Re: Distributed Systems Reading List

#66

Post by herodoturtle »

kortex wrote: post #33
herodoturtle wrote: post #19

Earlier quoted context omitted.

Hear hear. This is really important wisdom being shared. And with a single monolith you can comfortably handle several thousand concurrent users, not just several hundred. From my humble experience I've found that it is also relatively easier to migrate an established monolith to a semi-distributed system, than it is to build and scale a distributed system from the ground up whilst at the same time trying to figure o…

The trick is to find "fault lines" throughout your monolith - places where it's easy to create separation of concerns and narrow interfaces. Maintain and document these like they are external APIs. Only pass simple data through it (serialized or anything easy to serialize). When the time comes, cleave these chunks off and wrap them in RPC/REST/graphql. It's good form even for monoliths since it makes for an easy inte…

Good tip, and I learnt some new terminology too ("fault lines").

I guess the trick when building new monoliths is to identify these fault lines up front - as opposed to hopefully finding them later.

Will certainly guide my development going forward, thanks.

Re: Distributed Systems Reading List

#67

Post by parentheses »

shred45 wrote: post #25

I find "distributed systems" to be a huge source of imposter syndrome. Despite having worked almost exclusively with distributed applications for several years now, it is difficult to consider myself experienced. When I'm asked if I've worked with distributed systems, I don't think they are asking me if I've managed a Hadoop cluster. They are interested in building new applications using some of the primitives discus…

Distributed systems, in my experience, most often fall prey to "not invented here" syndrome.

Re: Distributed Systems Reading List

#68

Post by parentheses »

zcw100 wrote: post #62
shred45 wrote: post #25

I find "distributed systems" to be a huge source of imposter syndrome. Despite having worked almost exclusively with distributed applications for several years now, it is difficult to consider myself experienced. When I'm asked if I've worked with distributed systems, I don't think they are asking me if I've managed a Hadoop cluster. They are interested in building new applications using some of the primitives discus…

I don't trust anyone who comes back with quick answers without any hesitation or pause. I remember a lecture from a while ago, I can't remember who was giving it but they were someone well known, where they asked people to implement bubble sort. The results that came back ranged from something like 20 lines to 2000 lines and every single one had bugs . I don't know how anyone who works in this industry for any length…

I feel interviews are to blame. The interview is a place where doing this is often not OK. It signals to folks that that is expected day to day.

Re: Distributed Systems Reading List

#69

Post by avinassh »

libraryofbabel wrote: post #43

Good list, but is it still being actively updated? Not having Kleppmann’s seminal Designing Data-Intensive Applications (2017) on it would indicate no. Alex Petrov’s Database Internals: A Deep Dive Into How Distributed Data Systems Work (2019) is another essential recent reference that should be here. Not as broad as Kleppmann but dives a lot deeper into certain topics.

What are some more books / resources recommendations on the same?

Re: Distributed Systems Reading List

#70

Post by cbsmith »

lostcolony wrote: post #49
shred45 wrote: post #25

I find "distributed systems" to be a huge source of imposter syndrome. Despite having worked almost exclusively with distributed applications for several years now, it is difficult to consider myself experienced. When I'm asked if I've worked with distributed systems, I don't think they are asking me if I've managed a Hadoop cluster. They are interested in building new applications using some of the primitives discus…

Do you know what the CAP theorem is, and can explain it to me like I'm 5? Can you tell me how a SQL DB fits into it, and where something like DynamoDB fits into it? Congratulations, you are better than 95% of the people that I've interviewed out there saying they are experienced building distributed systems. Including system/solution architects.

Sad, but true. That said... the hiring pool tends to be biased towards people that others have passed on.
Post reply on HN