This article is dangerous in romanticizing the “not invented here” culture at many big tech companies and seems rooted more in the 90s than present day. The world of open source tooling and easily re-usable SAAS offerings means everyone has access to the best tools, whether you’re a small startup or a big company. Anyone who longs for internal, corporate tooling baffles me when they can use things that actually have…
The internal build tools I had access to at Amazon are unrivaled in the public domain. I DO miss them, at every job.
Tools you’d miss if you left a company
31–40 of 112 posts
Re: Tools you’d miss if you left a company
#32For 99% of engineering organizations, internal engineering tooling is not your source of competitive advantage. Your advantage in the market comes from understanding the needs of your target user better than anyone else, and building a product that meets those needs. For every piece of infrastructure or developer tooling you need, you have to make a buy vs. build decision based on cost/benefit analysis. Your cost to…
I do not think it’s that simple. I understand the arguments for buying vs build. However, if you’re a technical heavy organization, relying on buying things also has its costs. You’re constrained on your product offerings because of the thing you bought. The decision is not that simple and requires a lot of work and foresight. As such, my guiding principle has been to consider the layer at which the thing you’re tryi…
Exactly, for a lot of problem domains, the cost of integrating a third party solution is nearly equal to the cost of writing from scratch, because the hard part isn’t the problem space but the environment-specific configuration.
Re: Tools you’d miss if you left a company
#33If you’re not building complicated new systems and just gluing things together, you’re not doing engineering? 1) definitely gate keeping 2) good engineering is about what not to build as much as it is what to build.
Re: Tools you’d miss if you left a company
#34For 99% of engineering organizations, internal engineering tooling is not your source of competitive advantage. Your advantage in the market comes from understanding the needs of your target user better than anyone else, and building a product that meets those needs. For every piece of infrastructure or developer tooling you need, you have to make a buy vs. build decision based on cost/benefit analysis. Your cost to…
> If you're at a mega-corp with a huge engineering staff:
Netflix is an odd choice for number two on a list of two. I’d guess it has an order of magnitude fewer engineers than Facebook.
So what’s the rough cutoff here? 1000 software engineers?
Re: Tools you’d miss if you left a company
#35This article is dangerous in romanticizing the “not invented here” culture at many big tech companies and seems rooted more in the 90s than present day. The world of open source tooling and easily re-usable SAAS offerings means everyone has access to the best tools, whether you’re a small startup or a big company. Anyone who longs for internal, corporate tooling baffles me when they can use things that actually have…
The internal build tools I had access to at Amazon are unrivaled in the public domain. I DO miss them, at every job.
Re: Tools you’d miss if you left a company
#36Wanted to see what Facebook Scuba was, and found this comment thread that really highlights the point of this blog. https://news.ycombinator.com/item?id=13463016
Re: Tools you’d miss if you left a company
#37This article is dangerous in romanticizing the “not invented here” culture at many big tech companies and seems rooted more in the 90s than present day. The world of open source tooling and easily re-usable SAAS offerings means everyone has access to the best tools, whether you’re a small startup or a big company. Anyone who longs for internal, corporate tooling baffles me when they can use things that actually have…
Re: Tools you’d miss if you left a company
#38Earlier quoted context omitted.
The internal build tools I had access to at Amazon are unrivaled in the public domain. I DO miss them, at every job.
I'm asking this sincerely: What is/was so great about them?
The build tool was multi-language. It allowed depending on arbitrary packages which had been imported to the Amazon package repository. It allowed package owners to annotate the packages with guidance - experimental, deprecated, forbidden (in the case of security issues), etc. You could also declare conflicts, which would notify consumers at compiletime and force them to resolve in some way. The tool deferred to a number of standard build tools in whatever language(s) you were using; it was just about getting and packaging the dependencies.
When you committed code, a build was submitted to a distributed build system. It would run your build, and then it would run the builds of every package which declared a dependency on your version. If those builds failed, your build failed (so, bump your version or make your change backwards compatible). On completion, it imported an immutable bundle of your artifact + dependencies to the deployment infrastructure.
This is the part I miss - the build stuff was great but I largely find that open source and paid options aren't so bad here. What I miss is how easy it was to manage the journey of a built package to your machines. They had a tool called Pipelines that had a visualization of this progression. Each stage, with associated machines, was linked here. The tool knew how to add stages of environments, each with their own set of deployment configurations. You could set up approval workflows: integration tests, manual approvals, etc. You could feed one pipeline into another. For each artifact, you could configure autobuilds into your pipeline, so that new versions flowed as long as tests/approvals allowed. There was support for sanity checks: if those failed, the tool would automatically rollback. In fact, if any stage fails, your pipeline would block, and you'd be notified. In some cases, a newer build that was fully functional could unblock your pipeline.
Pipelines was a pleasure to use - it really just got the hell out of my way, and nothing I've used since is as simple to integrate with.
And tying this all together, there was a tool that would allow you to initialize the end to end infrastructural pieces needed. You go to a wizard, tell it your language and purpose (webapp, CLI tool, service) and it initializes a repository, package, and pipeline with environments. Get a coffee, come back, and Do Your Job.
I imagine similar things exist at places like Google, but man everywhere else I've worked, so much developer productivity has been lost not even approximating the level of "let me do the interesting stuff" that Amazon provided.
Re: Tools you’d miss if you left a company
#39Earlier quoted context omitted.
I do not think it’s that simple. I understand the arguments for buying vs build. However, if you’re a technical heavy organization, relying on buying things also has its costs. You’re constrained on your product offerings because of the thing you bought. The decision is not that simple and requires a lot of work and foresight. As such, my guiding principle has been to consider the layer at which the thing you’re tryi…
> The costs of integrating a bought tool are also overlooked in most of these conversations. External tools are marketed well but don’t always deliver on their promises of being easy to use or setup. Exactly, for a lot of problem domains, the cost of integrating a third party solution is nearly equal to the cost of writing from scratch, because the hard part isn’t the problem space but the environment-specific config…
Re: Tools you’d miss if you left a company
#40Earlier quoted context omitted.
The internal build tools I had access to at Amazon are unrivaled in the public domain. I DO miss them, at every job.
What's the closest OSS equivalent and how could they be improved?