Live data from Hacker News

Ask HN: CS papers for software architecture and design?

news.ycombinator.com

51–60 of 104 posts

Re: Ask HN: CS papers for software architecture and design?

#51
post #45
post #44

I used to be very interested in Software Architecture, in fact I've read many of the papers cited here. When I did a startup many years ago, I committed the mistake of paying too much attention to the architecture of the software [1] I was writing, and not enough attention to the product/customer side of it. The last couple of years I've been de-emphasizing software architecture as an interest, and have been paying m…

Agreed, good architecture doesn't come into play until you need to add a lot of complex features or scale significantly. Initially architecture doesn't really mean squat. I would concentrate on making the codebase flexible, but that's about it. I've regretted making some of my software with a cool but complex architecture when I should have been focused on what the product does.

> I would concentrate on making the codebase flexible, but that's about it

Arguably that's the property of good architecture. Standardizing your entire codebase also works, but only as long as those initial standards stay smart (1), and even then, that relies on discipline, tools, and incentives that are typically not all in place.

(1) A standard might be "all webservers will be written in Java 6". The benefits of choosing that standard tend to be front-loaded with a gradual decline into net-negative with no affordable path to a better standard.

Re: Ask HN: CS papers for software architecture and design?

#52

In general, CS (the science) does not often concern itself with the kind of architecture decisions day-to-day programmers deal with. Instead, the field tends to offer architecture advice in particularly hairy areas (DB design, distributed FT consensus, etc). In my opinion, that’s how it should be - the goals of academics are very different to those shipping code in the real world. A great developer will be aware of w…

That last line of yours is so very true. Whenever i find myself going down the rabbit hole i pick up the nearest relevant academic text and just speedread the relevant chapter to tailor my search doe relevant readymade solutions.

Developer life is like googlefu xtreme edition

Re: Ask HN: CS papers for software architecture and design?

#53
post #45

Earlier quoted context omitted.

Agreed, good architecture doesn't come into play until you need to add a lot of complex features or scale significantly. Initially architecture doesn't really mean squat. I would concentrate on making the codebase flexible, but that's about it. I've regretted making some of my software with a cool but complex architecture when I should have been focused on what the product does.

> I would concentrate on making the codebase flexible, but that's about it Arguably that's the property of good architecture. Standardizing your entire codebase also works, but only as long as those initial standards stay smart (1), and even then, that relies on discipline, tools, and incentives that are typically not all in place. (1) A standard might be "all webservers will be written in Java 6". The benefits of ch…

To design your software perfectly requires to know the future. You want the things that will change flexible and the rest simple. In practice, you will not hit the sweet spot and either over-engineer too much or hardcode too much (or both in different parts).

Re: Ask HN: CS papers for software architecture and design?

#54
post #44

I used to be very interested in Software Architecture, in fact I've read many of the papers cited here. When I did a startup many years ago, I committed the mistake of paying too much attention to the architecture of the software [1] I was writing, and not enough attention to the product/customer side of it. The last couple of years I've been de-emphasizing software architecture as an interest, and have been paying m…

Could you elaborate on the insights you gained ? At least on a high level . It would help a lot of people here .

I think most of the things people know, it's covered in books like Lean Startup, etc.

It's a bit like eating well and working out. We all know we should do it, but most people don't actually eat well and work out. Then, when you see somebody who does it and looks great, you ask them, "What's your secret?". But it's not a secret, it's just that most people don't do it, because it's hard :)

One story: When I was at FB, I happen to know that a team of size S conducted X experiments in 6 months (I can't disclose the number). As it happens, I have worked in similar size teams in other companies, and there the number was ~X/20, and sometimes 0. It don't matter how good your software architecture is, if you're trying out just 1 thing instead of 20 things... I call this velocity.

Another good story: many semi-successful companies end up in a place where there is an initial product/software which gets them a lot of growth, and then ~5 years into the startup, the bigger, more mature, bigger team decided that the old legacy code is holding them back, and they're going to REWRITE IT. Maybe in some fancy new language, or a fancy new architecture like micro-services. The estimate is 6-9 months to get to first light. But it will probably end up taking 3-5 years, because the legacy had a lot of fine-tuning in it, and it turns out many of the problems are hard to fix, moving the production to a new thing is REALLY HARD, and all those fancy new technologies are actually far from perfect, plus the current team doesn't have a lot of experience with it. Compare to this what Facebook did with its PHP codebase: at some point it became a bottleneck (the crappy language and the runtime speed), but there was a never a from-the-ground-rewrite. Instead they ended up writing several iterations of better runtimes, and since they were changing the runtime anyway, they "fixed up" the language (while keeping it mostly backward compatible). The new language is called Hack and the new runtime is called HHVM. The cool thing is, in all this time, there wasn't a rewrite, so they were able to keep shipping new features on Facebook, run A/B tests, iterate on the product. Compare this to one of the companies I worked at, where they did a rewrite , and now customers have to chose between the old and a new thing, it's not transparent, because . There's a book about Hack/HHVM, one of the Facebook guys wrote it, iirc the first chapter is about this whole story. See this blog post for links: http://bytepawn.com/hack-hhvm-second-system-effect.html

In general, the principles I've seen to work really well:

- write good code, but don't do big rewrites

- cont. delivery: always be shipping to master and production in small increments (don't have big git branches that aren't in production, at the end it will be scary to merge it and put it into production)

- cont. integration: have tests and run them on every commit (you probably need some testing nazis to enforce this...)

- if you (your team) can't write a good monolith, you (your team) also can't write a good MSA

- invest heavily in linting and other automated ways to catch and conform code when it is committed

- programming language doesn't matter that much, just pick one for each domain (eg. web, mobile, etc), and stop thinking about it, and don't let people waste their time on arguing over it too much; instead invest heavily in tooling that supports all the other aspects (like code reviews, perf, experimentation, etc)

- 1 other person should review and okay the code before it goes into production

- aggressivley remove obstacles from shipping stuff to production

- make it easy to run experiments, and run a lot of experiments

- don't hire people who just want to write code, or think their job stops there

Re: Ask HN: CS papers for software architecture and design?

#56
post #44

I used to be very interested in Software Architecture, in fact I've read many of the papers cited here. When I did a startup many years ago, I committed the mistake of paying too much attention to the architecture of the software [1] I was writing, and not enough attention to the product/customer side of it. The last couple of years I've been de-emphasizing software architecture as an interest, and have been paying m…

I had a similar journey although came away from that point headed in a slightly different direction. As you have studied the processes by which software is developed have you noticed a correlation between developing pieces that are architecturally important (many things are touched by the choices) and planning investment by the team?

Re: Ask HN: CS papers for software architecture and design?

#57

In general, CS (the science) does not often concern itself with the kind of architecture decisions day-to-day programmers deal with. Instead, the field tends to offer architecture advice in particularly hairy areas (DB design, distributed FT consensus, etc). In my opinion, that’s how it should be - the goals of academics are very different to those shipping code in the real world. A great developer will be aware of w…

i was going to make a similar comment but this sums it up very well.

CS papers work best when dealing with very specific, nitty gritty areas that require research after defining a hypothesis or problem to solve.

'Software architecture' is fairly broad and papers on those topics would read more like sociology papers than CS papers (e.g. how do people write code? here are some examples, here are some general patterns people like etc).

Looking back on reading that sort of stuff, it was nice but I very rarely apply it in day to day software development compared to readings that are more niche to the problem I'm solving that week.

Re: Ask HN: CS papers for software architecture and design?

#58
post #54

Earlier quoted context omitted.

Could you elaborate on the insights you gained ? At least on a high level . It would help a lot of people here .

I think most of the things people know, it's covered in books like Lean Startup, etc. It's a bit like eating well and working out. We all know we should do it, but most people don't actually eat well and work out. Then, when you see somebody who does it and looks great, you ask them, "What's your secret?". But it's not a secret, it's just that most people don't do it, because it's hard :) One story: When I was at FB,…

I agree with all of this for products. To a first approximation, you want to bump your head into reality as often as possible.

But I would say software architecture matters more for infrastructure like storage systems, operating systems, and programming languages than it does for products. I think a lot of the literature on software architecture is about those domains.

Those things are harder to do iteratively. And language choice matters more in those domains.

Programming is a huge field now, and a lot of choices are domain-specific. Including how much you should care about architecture and programming languages.

Re: Ask HN: CS papers for software architecture and design?

#59
post #44

I used to be very interested in Software Architecture, in fact I've read many of the papers cited here. When I did a startup many years ago, I committed the mistake of paying too much attention to the architecture of the software [1] I was writing, and not enough attention to the product/customer side of it. The last couple of years I've been de-emphasizing software architecture as an interest, and have been paying m…

I had a similar journey although came away from that point headed in a slightly different direction. As you have studied the processes by which software is developed have you noticed a correlation between developing pieces that are architecturally important (many things are touched by the choices) and planning investment by the team?

What I've seen is that most orgs, once they reach a certain size (100-200 people), start to spend a significant time planning. Part of this is human nature: once there are enough people, there will be managers, and plans are something managers show their hire ups to prove they're in control and doing a good job. But, I don't want to sound cynical, clearly planning is useful.

I used to think that spending too much time on planning is a waste of time , but then I noticed that teams at Facebook also spend a lot of time at the beginning of each half (6 mo) doing planning. However, I wasn't able to pick up tricks or patterns that made FB good at this, other than obvious/useless things like all the people making the plans were really smart / domain experts. I did notice that the plans/planning process was very lose, there was no methodology, often the outcome was a bunch of bullet points. And they were very conscious about the value of the plan: they knew that if things go well and they move quickly, probably a lot of things will change in 3-4 months, and they will diverge from the plan. The value is thinking things through, agreeing on goals, spinning up teams, building things (eg. dashboards) to track progress, etc.

Another story about the value of planning: at another company we hired a manager, and one of the things this guy did was introduce Capacity Planning. At the core it's a simple thing: make a big matrix, where the rows are the projects people want to work on, and the columns are the available people/teams. Collect all the projects, and make people put down their estimates for each (requires X time from team Y, etc). The value is, it's a very explicit way to show what people want to work on, obviously it will be more than you can actually do. So then you can make a very explicit choice about what you're doing and what you're not doing in the next X months. Back when we did this, there was an interesting insight the first time around: since we could count man days, we learned we're spending iirc ~70% of our time keeping the lights on (infra, etc), and only ~30% working on new things.

But despite this insight, I thought it's a waste of time, it's too slow/formal/sluggish. But then I saw FB also spends significant time on planning (although it's much less formal, because they can get away with it, for a long list of reasons), so I thought okay, spending time planning is probably the way to go.

Then I went to a company which was suffering from a focus problem. Every 2 weeks they'd have a prioritization meeting, where projects would get re-prioritized, which leads to obvious problems. Here I realized, for this problem, Capacity Planning would be a good thing, because one of the things it gets you is a long(er) term commitment to certain projects.

Overall, my takeaway is to spend 10-20% of time on planning and related meta-activities. Be conscious that the value is in thinking things through etc, and the actual path may diverge, but that's okay. Depending on the culture, there are a number of things that planning gives you, eg. a commitment to work on X for Y time. Make metrics and dashboards to track things, but don't overdo it.

Re: Ask HN: CS papers for software architecture and design?

#60
post #58
post #54

Earlier quoted context omitted.

I think most of the things people know, it's covered in books like Lean Startup, etc. It's a bit like eating well and working out. We all know we should do it, but most people don't actually eat well and work out. Then, when you see somebody who does it and looks great, you ask them, "What's your secret?". But it's not a secret, it's just that most people don't do it, because it's hard :) One story: When I was at FB,…

I agree with all of this for products. To a first approximation, you want to bump your head into reality as often as possible. But I would say software architecture matters more for infrastructure like storage systems, operating systems, and programming languages than it does for products. I think a lot of the literature on software architecture is about those domains. Those things are harder to do iteratively. And l…

You're probably right! But, most software engineers aren't writing a compiler/database/OS, and some of the ones who are, probably shouldn't :)
Post reply on HN