Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't solve the solved problems

blog.collections.me

Post reply on HN
Print view View original

31–38 of 38 posts

Re: Don't solve the solved problems

#31

Post by InclinedPlane »

alexjeffrey wrote: post #27
InclinedPlane wrote: post #25

Sigh. "Don't reinvent the wheel." Classic advice. And it has some merit. But it's a broken idea. Google? Not the first search company. The iPod? Not the first mp3 player. The iPhone? Not the first smartphone. Hacker news? Not the first tech oriented community site. "Don't reinvent the wheel... unless you think you can do much better." Also, if you're merely iteratively improving the wheel, do so whenever you like. We…

I read this post as less "don't reinvent the wheel", more "if you're building a car, don't build your own wheel when you could buy one from a wheel manufacturer" - maybe if you did it yourself you could make improvements, but it's far more likely that a company where the problem you're trying to solve is their bread and butter will provide a better solution.

I think the core problem is that the question "build or buy?" is formulated in a way such that it rests along the incorrect axis. It's like giving people advice on whether they should go outside or not, whether they should fall in love or not, whether they should start a company or not, etc.

The proper answer is: "it depends." But that's unsatisfying because it's too complex. But in reality you can't eliminate that complexity, it's an irreducible complexity. Just as there isn't one simple, objective system which you can consult to reliably get an answer to the question "should you go outside or stay indoors?" there isn't one for whether you should use something off the shelf or build something yourself. Look at, for example, SpaceX, who built the first generation of engines for their Falcon 9 rockets using turbopumps (one of the most complex components of a high performance rocket) provided by 3rd party companies. However, in the subsequent generation of engines (the Merlin-1D) SpaceX has designed and built the turbopumps themselves, and this is responsible for significant improvements in the engine.

Ultimately I think it's more important to take a step back and make a conscious effort to avoid making big decisions based on instinct or inertia and begin to cultivate a set of processes, elements of corporate culture, and habits which encourage thoughtful analysis and introspection about these choices. People should be more thorough about evaluating the advantages and disadvantages of doing things different ways and be mindful of when it doesn't seem to make much difference. And also mindful of the opportunity to change your mind in the future. Sometimes it can be worthwhile to rebuild a tool that already exists in the industry just for the purposes of acquiring the experience and knowledge of having done so. And sometimes even when taking something in house might easily save millions of dollars a year it still might not be worth it due to other opportunity costs.

It's a complicated question, and it's always going to be a complicated question. Instead of pretending it's an easy question people should be working on acquiring the skills to make sure they can answer the question well with some degree of reliability.

Re: Don't solve the solved problems

#32

Post by alexjeffrey »

InclinedPlane wrote: post #31
alexjeffrey wrote: post #27

Earlier quoted context omitted.

I read this post as less "don't reinvent the wheel", more "if you're building a car, don't build your own wheel when you could buy one from a wheel manufacturer" - maybe if you did it yourself you could make improvements, but it's far more likely that a company where the problem you're trying to solve is their bread and butter will provide a better solution.

I think the core problem is that the question "build or buy?" is formulated in a way such that it rests along the incorrect axis. It's like giving people advice on whether they should go outside or not, whether they should fall in love or not, whether they should start a company or not, etc. The proper answer is: "it depends." But that's unsatisfying because it's too complex. But in reality you can't eliminate that c…

I think your example also speaks to the argument I put forward to some extent - SpaceX initially chose to use a 3rd party turbopump as building an efficient turbopump isn't their objective. However, at a later stage when the 3rd party turbopumps became an issue (whether that be because they were faulty, not efficient enough or any number of reasons) SpaceX decided to build their own.

This seems like the correct way round to me for the majority of cases where subcomponents of your product could be externalised - a 3rd party should be the primary go-to choice unless a company cannot be found that fits the requirements. The issue I see in development (and have been guilty of in the past) is simply building everything when libraries could easily cover the use case - "not invented here" syndrome. That's the real problem.

[edit] Also I think it's clear that there'll always be cases where the default doesn't apply, for example when working with a new field/concept or where the 3rd party options are too far from the mark, but for the majority of development (especially web development) there's likely a library/service to match many of your requirements.

Re: Don't solve the solved problems

#33

Post by chacham15 »

alexjeffrey wrote: post #32
InclinedPlane wrote: post #31

Earlier quoted context omitted.

I think the core problem is that the question "build or buy?" is formulated in a way such that it rests along the incorrect axis. It's like giving people advice on whether they should go outside or not, whether they should fall in love or not, whether they should start a company or not, etc. The proper answer is: "it depends." But that's unsatisfying because it's too complex. But in reality you can't eliminate that c…

I think your example also speaks to the argument I put forward to some extent - SpaceX initially chose to use a 3rd party turbopump as building an efficient turbopump isn't their objective. However, at a later stage when the 3rd party turbopumps became an issue (whether that be because they were faulty, not efficient enough or any number of reasons) SpaceX decided to build their own. This seems like the correct way r…

What you are missing is that every library has its own dependencies and set of libraries. Gitlab uses nginx, unicorn, mysql, and a few other technologies. Ganglia uses apache, python, perl, and a few others. If you continue to add components you end up with seven different web servers, application servers, and little spare system resources and we havent even started talking about how to get these components to play nicely together! In conclusion, yes, it is nice if you can simply outsource to a library, but many times you have to bend over backwards to do so and it simply isnt worth it.

Re: Don't solve the solved problems

#34

Post by benjah »

kevinastone wrote: post #21
azov wrote: post #5

That's a hell lot of things to change, break, go down, lose your data, start charging ridiculous prices, be discontinued, etc... I'm all for SaaS and not re-inventing wheels, but I'd be nervous if I had to rely on so many external dependencies.

I think the vendor lock-in ( cough heroku) is the bigger risk rather than reliability.

I've heard this a lot lately. Heroku uses super standard stuff (nginx,varnish,postgres,etc). I could move my stuff off Heroku in a matter of days. I used to do a lot of sysadmin work. I go with them because I can't devote time to sysadmin work anymore. Same reason I don't run my own mail servers anymore.

Re: Don't solve the solved problems

#35

Post by InclinedPlane »

alexjeffrey wrote: post #32
InclinedPlane wrote: post #31

Earlier quoted context omitted.

I think the core problem is that the question "build or buy?" is formulated in a way such that it rests along the incorrect axis. It's like giving people advice on whether they should go outside or not, whether they should fall in love or not, whether they should start a company or not, etc. The proper answer is: "it depends." But that's unsatisfying because it's too complex. But in reality you can't eliminate that c…

I think your example also speaks to the argument I put forward to some extent - SpaceX initially chose to use a 3rd party turbopump as building an efficient turbopump isn't their objective. However, at a later stage when the 3rd party turbopumps became an issue (whether that be because they were faulty, not efficient enough or any number of reasons) SpaceX decided to build their own. This seems like the correct way r…

One of the things at play here is the Dunning-Kruger effect. It's pretty common for there to be lots of different libraries and little tools and whatnot which duplicate existing tools but do so poorly. And part of the reason why is that it's often easy for people to look at those other tools, see the faults in them and think to themselves that they could do so much better with very little effort. But often this is wrong due to an inflated sense of ones own capabilities.

Re: Don't solve the solved problems

#36

Post by InclinedPlane »

chacham15 wrote: post #33
alexjeffrey wrote: post #32

Earlier quoted context omitted.

I think your example also speaks to the argument I put forward to some extent - SpaceX initially chose to use a 3rd party turbopump as building an efficient turbopump isn't their objective. However, at a later stage when the 3rd party turbopumps became an issue (whether that be because they were faulty, not efficient enough or any number of reasons) SpaceX decided to build their own. This seems like the correct way r…

What you are missing is that every library has its own dependencies and set of libraries. Gitlab uses nginx, unicorn, mysql, and a few other technologies. Ganglia uses apache, python, perl, and a few others. If you continue to add components you end up with seven different web servers, application servers, and little spare system resources and we havent even started talking about how to get these components to play n…

It depends a lot on scale. If you're just throwing together a web app that isn't going to have crazy amounts of traffic or business then maybe a big messy dependency chain is ok. But every one of those dependencies is a risk and a liability, and the more valuable and significant your project becomes the more important it is to lock down risk and reduce liabilities. Which means tracking down and eliminating dependencies.

Re: Don't solve the solved problems

#37

Post by kevinastone »

benjah wrote: post #34
kevinastone wrote: post #21

Earlier quoted context omitted.

I think the vendor lock-in ( cough heroku) is the bigger risk rather than reliability.

I've heard this a lot lately. Heroku uses super standard stuff (nginx,varnish,postgres,etc). I could move my stuff off Heroku in a matter of days. I used to do a lot of sysadmin work. I go with them because I can't devote time to sysadmin work anymore. Same reason I don't run my own mail servers anymore.

I'd argue mail servers (as SMTP endpoints) are more interchangeable than Heroku. They have a completely proprietary configuration and deployment process.

I can understand that competent sysadmins can recreate heroku's underlying hosting, but there's still a decent switching cost (especially if your whole team is accustomed to the heroku way). I've consulted with several companies that have 5-figure a month hosting on heroku that they can't migrate after all their investment.

If SendGrid changes my plan (or somehow degrades), it's a simple config change to send my email through another provider. A more apt comparison would be if your mail provider used a custom ReST endpoint instead of SMTP (say, as MailChimp does).

Re: Don't solve the solved problems

#38

Post by benjah »

kevinastone wrote: post #37
benjah wrote: post #34

Earlier quoted context omitted.

I've heard this a lot lately. Heroku uses super standard stuff (nginx,varnish,postgres,etc). I could move my stuff off Heroku in a matter of days. I used to do a lot of sysadmin work. I go with them because I can't devote time to sysadmin work anymore. Same reason I don't run my own mail servers anymore.

I'd argue mail servers (as SMTP endpoints) are more interchangeable than Heroku. They have a completely proprietary configuration and deployment process. I can understand that competent sysadmins can recreate heroku's underlying hosting, but there's still a decent switching cost (especially if your whole team is accustomed to the heroku way). I've consulted with several companies that have 5-figure a month hosting on…

I agree. If you don't have any good sysadmin in your team, you are likely to be stuck at heroku. But then again, if you didn't have a sysadmin, you wouldn't have built it yourself. Starting at heroku or similar clone, still makes sense if your new thing may not make it past the first year. I guess it also depends what the business model is.
Post reply on HN