Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Buy Don't Build

jrott.com

Post reply on HN
Print view View original

201–210 of 227 posts

Re: Buy Don't Build

#201

Post by simonw »

pembrook wrote: post #194
simonw wrote: post #193

Earlier quoted context omitted.

That's an easy answer if you control the budget. If you're an engineer in a larger software organization, you probably don't have that spending authority. Which means you have to spend time convincing the people with that authority to spend that money. That's enough of a friction point that many engineers won't bother - which is why I talk about the invisible cultural damage that this problem causes. You and I know t…

Certainly a problem, but the time you’d need to spend convincing higher ups is still trivial compared to the time you’d need to spend to build, maintain, and update the tool.

And what if the higher ups say no?

In larger organizations budgets are complicated things. Your higher up with spending authority may agree with you, but they've hit the budget for their group and so your request for more logging is competing with a request to pay for a new compliance auditing tools or upgraded server capacity for machine learning models.

I'm certainly not arguing build over buy here - but this challenge is a genuine issue which I don't think gets enough consideration.

Re: Buy Don't Build

#202

Post by elcritch »

ummonk wrote: post #138
whoisjuan wrote: post #10

I own a Screenshot API service (shameless plug: https://getscreenshot.rasterwise.com/ ) and this exactly the argument that I make for small utilitarian services like mine. You shouldn't build them because buying them is several orders of magnitude cheaper. I have a small paragraph from a blog post where I explain this with some math that is probably crappy but captures the idea: "When you spend time in areas that are…

I think the screenshot service is a perfect example of a service suited for buy don't build - it's something complex to implement but with a really simple API to integrate with.

Yah it’s a pretty ideal example. Another might be CI. It seems from reading other comments that bad examples in buying external products are from complex “business apps”.

Re: Buy Don't Build

#203

Post by acdha »

simonw wrote: post #116

One thing to pay close attention to when making a build v.s. buy decision is the impact that billing models will have on your usage of a tool. Take logging for example. If you buy a log aggregation platform like Splunk Cloud or Loggly the pricing is likely based on the quantity of data you ingest per day. This can set up a weird incentive. If you are already close to the limit of your plan, you'll find that engineers…

I’ve seen this repeatedly over my career. At the large organizations I’ve worked at (.edu, .gov), unless you have a large general tools budget it’s surprisingly hard to beat the open-source tools - simply running a procurement takes months and might require arguments with non-technical staff who don’t understand why certain options are not truly equivalent even if the salesperson swears they are. This is how you end up spending 7+ figures on enterprise tools while the technical staff are all using Prometheus, Grafana, and ELK because they actually work and are available to the entire group.

On the “clearly crazy to invest in building your own” front I think it’s useful to add the nuance about how much of that time is purely related to the product and how much is the tuning, integration, understanding the data, etc. which every product requires. I think we’re prone to underestimate the latter and have something like Splunk save less time and cost more than expected because the commodity part it optimized for wasn’t as much of the whole as anticipated.

Re: Buy Don't Build

#204

Post by PeterStuer »

'Buy' often is not available at the granular cost:benefit price-point that your specific application needs, especially in the startup and scale-up phase.

For the service provider there is a fairly fixed to linear base overhead so it does not make sense to make an special offering for the 1/10th or less of functionality you might require at 1:10th of the price. It doesn't make economic sense.

So for the small business it can mean they get away with a homegrown minimal service at a fraction of the cost, even though the 'quality' of the produced for general applications might also just be a fraction of integrating the 'off the shelf' offering.

I've seen some startups running in major deficits simply because they went all out on integrating external services the aggregated cost far outstripped the margin their market allowed for.

Re: Buy Don't Build

#205

Post by hansvm »

papito wrote: post #43

The most egregious cases of violating this involve trying to create your own CMSs, search engines, and God forbid - databases. There are of course, exceptions. AirTable wrote their OWN database engine, but they knew what they were getting into, and they did it right.

I can't speak to CMS, but are search engines and databases really such a big deal? If postgres or influx or something kinda fits your workload then it's probably not worth the time, but on the other hand I've seen both custom databases and the cludges people have written to work around off-the-shelf DBs, and I vastly prefer the former both in terms of performance and maintainability in a problem space (similarly with search engines).

Re: Buy Don't Build

#206

Post by KenanSulayman »

jokethrowaway wrote: post #147
simonw wrote: post #116

One thing to pay close attention to when making a build v.s. buy decision is the impact that billing models will have on your usage of a tool. Take logging for example. If you buy a log aggregation platform like Splunk Cloud or Loggly the pricing is likely based on the quantity of data you ingest per day. This can set up a weird incentive. If you are already close to the limit of your plan, you'll find that engineers…

Memories of developers not logging stuff on datadog not to go over limit. We ended up running our own ELK stack and it was much better.

ELK is killing it. Especially with their apm server.

Re: Buy Don't Build

#207

Post by simonw »

simonw wrote: post #116

One thing to pay close attention to when making a build v.s. buy decision is the impact that billing models will have on your usage of a tool. Take logging for example. If you buy a log aggregation platform like Splunk Cloud or Loggly the pricing is likely based on the quantity of data you ingest per day. This can set up a weird incentive. If you are already close to the limit of your plan, you'll find that engineers…

I posted an expanded version of this comment on my blog, with a suggestion that maybe there are some internal process improvements that could help address the problem: https://simonwillison.net/2020/Dec/13/build-vs-buy/

"Thinking about this more, I realize that this isn’t a technology problem: it’s a process and culture problem. So there should be a process and cultural solution.

One thing that might work would be to explicitly consider this issue in the vendor selection conversations, then document it once the new tool has been implemented.

A company-wide document listing these tools, with clear guidance as to when it’s appropriate to increase capacity/spend and a documented owner (individual or team) plus contact details could really help overcome the standard engineer’s resistance to having conversations about budget."

Re: Buy Don't Build

#208

Post by Renevith »

watermelon59 wrote: post #165

I hope this doesn't get buried, because there's something that I feel like I'm a bit of a unicorn about. > The problem then with that is everyone who is working on those services is usually trying to get off of them. After all, no one wants to work on something that their boss doesn’t care about. > Yes, your CI/CD system is absolutely critical, but it’s easy for executives to not think about. This leads to a failure…

This is very much how I feel, which is why I've started a dedicated team for this at Wealthfront, with the support of our CTO, our VP of engineering, and our president (who used to be the CTO). My team is specific to the backend but we have equivalent "infrastructure" teams for the frontend platforms too.

If anyone feels like it's hard to get your company to care about this stuff or even let you work on it, maybe it's time to consider a move? :-) You can reach me at andyf@wealthfront.com if you prefer not to reply here.

Re: Buy Don't Build

#209

Post by Renevith »

Mandatum wrote: post #184
watermelon59 wrote: post #165

I hope this doesn't get buried, because there's something that I feel like I'm a bit of a unicorn about. > The problem then with that is everyone who is working on those services is usually trying to get off of them. After all, no one wants to work on something that their boss doesn’t care about. > Yes, your CI/CD system is absolutely critical, but it’s easy for executives to not think about. This leads to a failure…

Seems like your interests align to some SRE and DevOps roles, but you'd definitely need to shop around and it'd only work at large companies.

I agree that you need to shop around; some places value this far more than others. But I don't think the company needs to be huge. We have small but dedicated teams for this on each platform (backend, iOS, Android, web) and we only have ~100 total engineers.

Re: Buy Don't Build

#210

Post by ric2b »

pembrook wrote: post #194
simonw wrote: post #193

Earlier quoted context omitted.

That's an easy answer if you control the budget. If you're an engineer in a larger software organization, you probably don't have that spending authority. Which means you have to spend time convincing the people with that authority to spend that money. That's enough of a friction point that many engineers won't bother - which is why I talk about the invisible cultural damage that this problem causes. You and I know t…

Certainly a problem, but the time you’d need to spend convincing higher ups is still trivial compared to the time you’d need to spend to build, maintain, and update the tool.

But building can be a lot more interesting and at the end of the day you'll get the same salary (unless you get enough equity to care about it).
Post reply on HN