VMWare.
GNS3.
Docker.
Remember that large scale, complicated systems that work invariably start out as small simple systems that work, so build it virtually first, where you can break and re-build to your hearts content.
How to become good at system design?
21–29 of 29 posts
Re: How to become good at system design?
#22[deleted]
Re: How to become good at system design?
#23Earlier quoted context omitted.
Good judgement comes from a mix of positive and negative experiences while paying attention to the right things.
Without experience you simply cannot know right from wrong
Incorrect.
I learned that it was wrong to casually point a firearm at someone before I had the experience of firing a gun. I also first learned to direct my visual focus to the front sights. Knowing that rule enabled me to then gain more nuanced skill and helped me avoid building bad habits.
After 8 years in the software industry, I dearly wish that I’d been able to find such guidance about software estimation when I was starting out. At this point, I’m going to need some sort of therapy to be able to give software estimates.
Re: How to become good at system design?
#24Earlier quoted context omitted.
Without experience you simply cannot know right from wrong
Incorrect. I learned that it was wrong to casually point a firearm at someone before I had the experience of firing a gun. I also first learned to direct my visual focus to the front sights. Knowing that rule enabled me to then gain more nuanced skill and helped me avoid building bad habits. After 8 years in the software industry, I dearly wish that I’d been able to find such guidance about software estimation when I…
First, you didn’t actually learned those things, you trusted them.
Second, im not speaking about trivial common knowledge. There’s obviously always something useful to learn from theory in any area.
But its subtle things that make all the difference between average and really good in something, these ones you can’t really learn without gettyhands dirty and making own mistakes. Even though often it seems different looking backwards, but it’s nothing more than a curse if knowledge.
Re: How to become good at system design?
#25The book System Design Interview is quite good and pragmatic. Personally, I found Designing Data-Intensive Applications less useful because it works on a much lower level. Like if you want to know how database internals work, then read DDIA. If you want to understand how to author a scalable API with layers of caching and distributed data storage, read SDI.
SDI?
Re: How to become good at system design?
#26Earlier quoted context omitted.
Incorrect. I learned that it was wrong to casually point a firearm at someone before I had the experience of firing a gun. I also first learned to direct my visual focus to the front sights. Knowing that rule enabled me to then gain more nuanced skill and helped me avoid building bad habits. After 8 years in the software industry, I dearly wish that I’d been able to find such guidance about software estimation when I…
First, you didn’t actually learned those things, you trusted them. Second, im not speaking about trivial common knowledge. There’s obviously always something useful to learn from theory in any area. But its subtle things that make all the difference between average and really good in something, these ones you can’t really learn without gettyhands dirty and making own mistakes. Even though often it seems different loo…
> But its subtle things that make all the difference between average and really good in something, these ones you can’t really learn without gettyhands dirty and making own mistakes.
I strongly agree here. It was the subtle things about breath control and muscle-tension control were what I learned while practicing marksmanship.
It is simultaneously true that a person starting out can sometimes see they need guidance to learn the 'trivial common knowledge' that others in a field are already familiar with. These things that seem 'trivial' to experts are things that help them avoid the "many ways to fail at getting good at something while doing it a lot."
Re: How to become good at system design?
#27Please don't bother with the whole charade of you have to do it to learn it. This kind of advice places more importance on people working at internet companies which have achieved scale. If you are at these companies then you have nothing to worry, experience will teach you. If you are not, follow @cppr's advice and read through DDIA. It will give you good enough introduction to data systems - not just databases. The…
One topic I haven't seen widely discussed in books but happens in the real world is back pressure. I think you're getting to this with "attack your designs", though I find it hard to predict. Have you any thoughts, books or resources you turn to, to make sure the whole pipeline is balanced - or doesn't fall over when one part is overloaded?
You need to see what happens in you data processing chain. The pipeline is only "balanced" based on what Rps you want. Throughput calculations are easier to do. Latencies are harder to predict but honestly latency calculations without sensible benchmarks are horseshit.
Back pressure is refusing to let the queue unbounded or take in more requests than slots available. That's pretty much it.
Re: How to become good at system design?
#28In addition to the responses talking about getting better with software specifically, I think it's helpful to work get better about thinking in a systems way, generally. There are systems all around us, providing plenty of opportunity to practice systems thinking. For instance right now looking out my window at my backyard I see a drainage basin with frozen water there. I can start asking questions like "where did th…
I want to second the book recommendation, it changed how I viewed the world and politics.
I only got through about halfway, stocks and flows is all I got out of it.