Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On Internal Engineering Practices at Amazon

jatins.gitlab.io

Post reply on HN
Print view View original

61–70 of 130 posts

Re: On Internal Engineering Practices at Amazon

#61

Post by School-Cotton »

just_passing_by wrote: post #45

What amuses me is that most of retractions come from ex-Amazonians not from the current staff. This is the only company i know dealing with that much criticism from engineering. Even more to add is that the article is more or less fresh and at Amazon's scale i doubt any major changes had undergone in the last 10 months.

Amazon (like many other big companies) strongly discourages employees from commenting publicly on subjects related to the company.

It might seem draconian, but it makes a lot of sense. Talking about a smaller company in a forum like HN might be completely anodyne, but comments about Amazon could easily have repercussions like getting picked up by the press and spun into some crazy story, or alerting people to some strategic information that the commenter isn't even aware is sensitive. For example, posting "hey we're at Amazon and we're moving from system X to system Y" could generate surprised and angry phone calls with the CEO of the vendor of X (I have a real example in mind that happened because of a stackoverflow post), or could cause the stock price of the vendor of Y to jump, causing insider trading concerns... best to just avoid it.

So it's very natural that company policy would strongly discourage such public commentary, and most current employees follow that.

Re: On Internal Engineering Practices at Amazon

#62

Post by School-Cotton »

ilovecaching wrote: post #48

One misconception people get wrong, from my perspective, about most FAANG is that you get to work with new and shiny things, especially new langauges. This is really more applicable to startups where risk taking is in the DNA, you will mostly get really strong pushback because there is just too much effort involved to support more than two or three languages at scale. There are normally niche languages, but they are…

Very good comment, +1.

People don't really get the rabbit hole that is necessary in order to introduce a new technology at a large company. Just off the top of my head, a few of the issues:

* Not everyone has followed the broader technology world outside the big company. It's totally possible to imagine a situation where nobody in your management chain or team has even heard of something relatively mainstream like Docker.

* Your company uses custom build, deployment, and dependency management systems. Someone will have to do the work to implement support for the new technology in these.

* If the new technology has its own opinionated ideas about how to do any of the above, like Rust crates, npm, pip, etc., forget about it. You need to interact with previously existing internal code which means you need to fit into the already existing build/deployment/dependency solution.

* Your company has a bunch of custom internal services. You need to create bindings for these services' APIs in the new language.

* Your company might be doing some esoteric stuff that the new technology has no support for, like for example if your network stack does HTTP in some sort of custom-tuned way for performance, and the new language's HTTP library only supports a subset of everything that's possible.

* The new technology may have correctness or performance issues that have never been discovered because it has never been used on many thousands of servers where a 1% difference in CPU makes a huge difference, or it has never been used with a codebase big enough to take >1 day to build, or with binaries larger than several GB, etc.

And the biggest one...

* Every day, many new people are joining the company and beginning to ramp up on the (internally) mainstream solutions that have institutional inertia. This will dwarf the speed at which you can convince people to switch to the new technology.

The pattern I have seen is that something starts off being used for one-off scripts that don't have a lot of dependencies on the existing gargantuan infrastructure, then very, very gradually gaining mindshare internally and becoming more supported.

Re: On Internal Engineering Practices at Amazon

#63

Post by dmnd »

School-Cotton wrote: post #49
throwaway772643 wrote: post #19

I will be joining Amazon in about a month. Is there any chance I'll be able to work on OSS and/or "modern" tech (e.g. containers, Go, etc.) without a ton of push-back? It also seems Amazon is obsessed with reinventing wheels and keeping their stuff internal, which is worrying. Is there any chance to introduce solid OSS tools to the development process? (whatever they might be)

I worked there for two years (summer 2013-summer 2015), so long enough to get an idea of the culture. Take this post with the caveat that I have no idea whether it's changed since then and if so to what extent. > Is there any chance I'll be able to work on OSS and/or "modern" tech (e.g. containers, Go, etc.) without a ton of push-back? Sure there's a greater chance than zero, but not if only motivated by it being "mo…

> Amazon contributes less to open source than a few other famous tech companies, though I'd say Apple is on par.

Really? Apple has impactful OSS like LLVM, WebKit, Swift.

Re: On Internal Engineering Practices at Amazon

#64

Post by School-Cotton »

dmnd wrote: post #63
School-Cotton wrote: post #49

Earlier quoted context omitted.

I worked there for two years (summer 2013-summer 2015), so long enough to get an idea of the culture. Take this post with the caveat that I have no idea whether it's changed since then and if so to what extent. > Is there any chance I'll be able to work on OSS and/or "modern" tech (e.g. containers, Go, etc.) without a ton of push-back? Sure there's a greater chance than zero, but not if only motivated by it being "mo…

> Amazon contributes less to open source than a few other famous tech companies, though I'd say Apple is on par. Really? Apple has impactful OSS like LLVM, WebKit, Swift.

Fair enough, I edited out the comment about Apple.

Re: On Internal Engineering Practices at Amazon

#65

Post by just_passing_by »

School-Cotton wrote: post #61
just_passing_by wrote: post #45

What amuses me is that most of retractions come from ex-Amazonians not from the current staff. This is the only company i know dealing with that much criticism from engineering. Even more to add is that the article is more or less fresh and at Amazon's scale i doubt any major changes had undergone in the last 10 months.

Amazon (like many other big companies) strongly discourages employees from commenting publicly on subjects related to the company. It might seem draconian, but it makes a lot of sense. Talking about a smaller company in a forum like HN might be completely anodyne, but comments about Amazon could easily have repercussions like getting picked up by the press and spun into some crazy story, or alerting people to some st…

>Amazon could easily have repercussions like getting picked up by the press and spun into some crazy story

Didn't NY Times expose help to shape Amazon workplace for good?

Re: On Internal Engineering Practices at Amazon

#66

Post by IloveHN84 »

To me, it sounds like a company who is still working with the mentality of 90s. The same I'm facing at work, where new technology is treated as scary and there's fear of change something that (somehow, only God knows how) is working and none is allowed to touch such things

Re: On Internal Engineering Practices at Amazon

#67

Post by [deleted] »

School-Cotton wrote: post #55
ecarrlee wrote: post #31

Earlier quoted context omitted.

I was an intern at AWS this past summer(and will be returning as a new grad in May) and this article is not accurate based on my experience. My code (which was part of our team's actual production system, not a toy project) ran on EC2 instances in an autoscaling group. There was no manually provisioning servers. I worked with a Kinesis Stream and several other resources and there was a way for us to programmatically…

I can confirm that the claim "you can't even use a language other than Java" is completely false. Java is often the best tool for the job because of how much institutional inertia there is behind it, but certainly nobody will stop a team from using a better tool. PHP is banned at Amazon, and that's the exception that proves the rule: a language being banned is so unusual that I'm aware specifically of the example of…

[deleted]

Re: On Internal Engineering Practices at Amazon

#68

Post by jhayward »

m0zg wrote: post #57
jhayward wrote: post #50

Earlier quoted context omitted.

> Containers are "bare metal" Please explain this use of jargon. Are you saying that a Docker container is "bare metal" in some culture? A VM is bare metal? What do you mean by container, and bare metal?

Containers are just namespaces for things within the Linux kernel. Unlike with VMs, you're not running separate instances of the OS, it's all run by just one kernel instance, and that kernel usually runs directly on the hardware, that is, on "bare metal". That "ubuntu" base image you can spin up does not actually run the Ubuntu kernel. As a result, bare metal containers incur none of the "virtualization penalty" that…

Ah, you're using it to mean "no virtualization". Got it, thanks.

I tend to use the term to mean single-tenant, user-supplied OS, non-virtualized myself but I see where you're coming from.

Re: On Internal Engineering Practices at Amazon

#70

Post by jypepin »

> "It's complicated, so it's gotta be good. I must be dumb to not get it."

Having worked with aws a lot recently, this article doesn't surprise me at all actually. When you see the low quality of UI and documentation for most of their tools that users pay for, I wouldn't expect their internal tooling to be any better.

I'm not saying their tech is bad, once things work, it works great - I'm talking about the usability of those tools as I try to use them, and it makes me feel the same as the OP's quote.

Post reply on HN