Live data from Hacker News

On Internal Engineering Practices at Amazon

jatins.gitlab.io

101–110 of 130 posts

Re: On Internal Engineering Practices at Amazon

#101
post #91

I worked at a startup and then switched to a bigger product company: Zoho. If there's one thing I'd take away for the rest of my career from Zoho, it would be frugality in adopting the latest of tech. When NoSQL was all the rage, the company stood firm that relation databases had rock-solid mathematical foundation and stayed away from the bandwagon. It paid off. When every other company wrote blogs about rewriting th…

I have shared this blog posting more than I care to admit. https://mcfunley.com/choose-boring-technology It amazes me that people will put their companies and employees at risk by using the new cool stuff just because everyone else is ( looking at you k8s )

I was in a management discussion at a prior company where people were actually making technology decisions based on how our stack would look in job ads and whether it would excite potential candidates.

Re: On Internal Engineering Practices at Amazon

#102
I worked at Amazon in the late 90s. So my experience is most likely not relevant anymore, but I will make a few observations. First, I see that many commenters disagree with the OP, they had a different experience of Amazon, one where they were working with infrastructure that was responsive. modern, easy to use etc. It very possible that both observations are correct. In a large company, not all parts of the company will be using the same infrastructure at the same time. Indeed, I would be dangerous for the entire company to upgrade lockstep to a new technology infrastructure. Second, in most companies, innovation is not measured by the novelty or newness of the language or framework you use, but by the business impact your product or service makes. Much of Amazon's innovation was, and is, around business models. Indeed, when I worked at AMZN, I was writing C code (to power a website) using beautifully efficient database access code written by Sheldon Kaphan. There was nothing remotely advanced about the language. It took 9 months for me to get a 3 line code change into production. And I was using technology that predated Apollo (it was called Huston) There is nothing particularly wrong about that either (it was a potty mouth filter and was blocking some obscure swear words, and no one was too worried that the component it was part of didn't ship for the best part of a year.) I now run my own company, and I both manage technology, people as well as write code. I find myself exercising the same conservatism with respect to code and infrastructure that I found at Amazon, and for the same reasons. It is expensive and potentially company destroying to switch languages and core technologies. It is best not done, or if done at all, done with a lot of care and slowly.

Re: On Internal Engineering Practices at Amazon

#103

One misconception people get wrong, from my perspective, about most FAANG is that you get to work with new and shiny things, especially new langauges. This is really more applicable to startups where risk taking is in the DNA, you will mostly get really strong pushback because there is just too much effort involved to support more than two or three languages at scale. There are normally niche languages, but they are…

Very good comment, +1. People don't really get the rabbit hole that is necessary in order to introduce a new technology at a large company. Just off the top of my head, a few of the issues: * Not everyone has followed the broader technology world outside the big company. It's totally possible to imagine a situation where nobody in your management chain or team has even heard of something relatively mainstream like Do…

"...or it has never been used with a codebase big enough to take >1 day to build..."

I've not worked in this scale of enterprise environments so I find this tidbit fascinating. Why does the codebase take over a day to build? Does that time include rebuilding and deploying the underlying infrastructure?

I'm genuinely interested here. I can only imagine a few highly niche cases myself so I'm curious what I'm missing here.

Re: On Internal Engineering Practices at Amazon

#104

What amuses me is that most of retractions come from ex-Amazonians not from the current staff. This is the only company i know dealing with that much criticism from engineering. Even more to add is that the article is more or less fresh and at Amazon's scale i doubt any major changes had undergone in the last 10 months.

I am a current Sr. SDE at Amazon in the Retail org. I would agree with most of the rebuttals, and I don't have too much to add, since they do a pretty good job summarizing.

Re: On Internal Engineering Practices at Amazon

#105
post #90

Earlier quoted context omitted.

> Apollo had a few layers to it but would, more or less, matched kubernetes plus additional tooling Have you looked at the codebase? Apollo and Brazil are tiny compared to Kubernetes & friends. You could reimplement them with a small script on top of traditional Linux package management (modulo the web UI).

As someone who worked on Apollo, I'd challenge this. Without divulging any details, Apollo is a non trivial system.

Yeah, I work adjacent to the Apollo org (and have worked near Pfrheak, too. Hi!). I would love to see a HackerNews commenter's idea of what it would take to re-implement Apollo. I'll bring popcorn.

Re: On Internal Engineering Practices at Amazon

#106

Earlier quoted context omitted.

I was an intern at AWS this past summer(and will be returning as a new grad in May) and this article is not accurate based on my experience. My code (which was part of our team's actual production system, not a toy project) ran on EC2 instances in an autoscaling group. There was no manually provisioning servers. I worked with a Kinesis Stream and several other resources and there was a way for us to programmatically…

The Java claim is just factually false. Java/C++/Python/Ruby/Go/Perl/Scala, even Rust, are used in Amazon for various projects. What might be true is that, consider how critical your service would be, the more it is, people will lean towards to more conservative/mature/enterprise-ish languages, not exotic ones, because that inherently means risk. And Java is the go-to option under that consideration, just because, it…

Plus, you know, https://aws.amazon.com/corretto/

We do actually vend a JDK. Yes, it's conservative. It's also necessary.

Re: On Internal Engineering Practices at Amazon

#107

Earlier quoted context omitted.

There is a comment above you from an Amazon Principal Engineer: https://news.ycombinator.com/user?id=jcrites His profile says "Architect and cofounder of Simple Email Service. Creator of Cloud Desktop, a cloud-based development environment used by most Amazon engineers. Technical lead for Amazon's strategy for using AWS." Can't get more "From the horse's mouth than that" We are generally asked not to comment on stuff…

I'm aware current employees comment over here. Two things to mention: principal/architect roles are always based not only on a merit of skill but politics. So, taken with a grain of salt. Moreover, if you inspect the person's comments you will notice how "legally" clean they are. Even the phrasing looks the same for both comments "based on my experience", "this information if wrongful" etc. Looks like those were refi…

translation of the first two paragraphs is "I've decided what I think already and if you say you work at Amazon now and are happy it's prima facie evidence that you can't be trusted to be objective about it"

Re: On Internal Engineering Practices at Amazon

#108

Earlier quoted context omitted.

I'm aware current employees comment over here. Two things to mention: principal/architect roles are always based not only on a merit of skill but politics. So, taken with a grain of salt. Moreover, if you inspect the person's comments you will notice how "legally" clean they are. Even the phrasing looks the same for both comments "based on my experience", "this information if wrongful" etc. Looks like those were refi…

translation of the first two paragraphs is "I've decided what I think already and if you say you work at Amazon now and are happy it's prima facie evidence that you can't be trusted to be objective about it"

Sort of.

If it's a liability to say nasty stuff about the employer and most of the ex-ones throw some heat the nearest possible conclusion is that the truth tends to bow to statements that are observed as outdated and controversial by present employees.

It just can't be a smear campaign against just that one company.

Re: On Internal Engineering Practices at Amazon

#109

Earlier quoted context omitted.

Very good comment, +1. People don't really get the rabbit hole that is necessary in order to introduce a new technology at a large company. Just off the top of my head, a few of the issues: * Not everyone has followed the broader technology world outside the big company. It's totally possible to imagine a situation where nobody in your management chain or team has even heard of something relatively mainstream like Do…

"...or it has never been used with a codebase big enough to take >1 day to build..." I've not worked in this scale of enterprise environments so I find this tidbit fascinating. Why does the codebase take over a day to build? Does that time include rebuilding and deploying the underlying infrastructure? I'm genuinely interested here. I can only imagine a few highly niche cases myself so I'm curious what I'm missing he…

Hard to find updated publicly-available information on this, but it was claimed that in 2003, Windows took 12 hours to build, on a multi-machine build farm: https://www.itprotoday.com/windows-server/supersite-flashbac...

Re: On Internal Engineering Practices at Amazon

#110

Earlier quoted context omitted.

I agree. Disclaimer: Also ex-amazonian, left like one year ago.

What happened like one year ago?

Oh, this is the thread which is reserved for everyone who left one year ago from Amazon. If you left two years ago, you are supposed to comment in a different thread. That's just how large companies work.
Post reply on HN