Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On Internal Engineering Practices at Amazon

jatins.gitlab.io

Post reply on HN
Print view View original

91–100 of 130 posts

Re: On Internal Engineering Practices at Amazon

#91

Post by geggam »

lewisjoe wrote: post #88

I worked at a startup and then switched to a bigger product company: Zoho. If there's one thing I'd take away for the rest of my career from Zoho, it would be frugality in adopting the latest of tech. When NoSQL was all the rage, the company stood firm that relation databases had rock-solid mathematical foundation and stayed away from the bandwagon. It paid off. When every other company wrote blogs about rewriting th…

I have shared this blog posting more than I care to admit.

https://mcfunley.com/choose-boring-technology

It amazes me that people will put their companies and employees at risk by using the new cool stuff just because everyone else is ( looking at you k8s )

Re: On Internal Engineering Practices at Amazon

#92

Post by scott_s »

m0zg wrote: post #57
jhayward wrote: post #50

Earlier quoted context omitted.

> Containers are "bare metal" Please explain this use of jargon. Are you saying that a Docker container is "bare metal" in some culture? A VM is bare metal? What do you mean by container, and bare metal?

Containers are just namespaces for things within the Linux kernel. Unlike with VMs, you're not running separate instances of the OS, it's all run by just one kernel instance, and that kernel usually runs directly on the hardware, that is, on "bare metal". That "ubuntu" base image you can spin up does not actually run the Ubuntu kernel. As a result, bare metal containers incur none of the "virtualization penalty" that…

It's tempting to say that containers don't have any virtualization costs, but they often do for networking. For some applications, the addition of another networking driver doesn't matter. For others, it does.

Re: On Internal Engineering Practices at Amazon

#93

Post by andrewguenther »

vonseel wrote: post #82

As far as I know, Node.js was used inside in a limited capacity, but they had an alternative for npm for security reasons, and you had to get an npm package approved to use it internally. FWIW, this is a very good thing . A company as large as Amazon should do this with all of their repositories; even a small start-up should be doing this to mitigate suspicious packages / third-party code vulnerabilities.

Node is picking up internally as a build environment for frontend JS which all used to be done in Ruby.

Re: On Internal Engineering Practices at Amazon

#94

Post by geggam »

jypepin wrote: post #70

> "It's complicated, so it's gotta be good. I must be dumb to not get it." Having worked with aws a lot recently, this article doesn't surprise me at all actually. When you see the low quality of UI and documentation for most of their tools that users pay for, I wouldn't expect their internal tooling to be any better. I'm not saying their tech is bad, once things work, it works great - I'm talking about the usability…

If you are using the UI to leverage AWS you arent really leveraging AWS.

AWS is designed to be used via automation and the API.

Disclaimer : I dont work for AWS but have spent many years building relatively large stacks on AWS (thousands of ec2 instances with monthly spend being a couple supercars in value)

Re: On Internal Engineering Practices at Amazon

#95

Post by mmartinson »

geggam wrote: post #91
lewisjoe wrote: post #88

I worked at a startup and then switched to a bigger product company: Zoho. If there's one thing I'd take away for the rest of my career from Zoho, it would be frugality in adopting the latest of tech. When NoSQL was all the rage, the company stood firm that relation databases had rock-solid mathematical foundation and stayed away from the bandwagon. It paid off. When every other company wrote blogs about rewriting th…

I have shared this blog posting more than I care to admit. https://mcfunley.com/choose-boring-technology It amazes me that people will put their companies and employees at risk by using the new cool stuff just because everyone else is ( looking at you k8s )

Definitely, but, I'm sure that adopting new technologies at the right time can be a huge competitive edge for startups. Sure it can fail horribly in some places where hindsight will show that the trusted stack was the right choice, but it could also mean winning a race to market and continued relevance for a company if it works out.

Re: On Internal Engineering Practices at Amazon

#96

Post by andrewguenther »

throwaway772643 wrote: post #19

I will be joining Amazon in about a month. Is there any chance I'll be able to work on OSS and/or "modern" tech (e.g. containers, Go, etc.) without a ton of push-back? It also seems Amazon is obsessed with reinventing wheels and keeping their stuff internal, which is worrying. Is there any chance to introduce solid OSS tools to the development process? (whatever they might be)

> Is there any chance I'll be able to work on OSS and/or "modern" tech (e.g. containers, Go, etc.) without a ton of push-back?

As long as it is the right choice, yes. As far as I know, I built the first service internally that was entirely container based, but did so because it was the right tool for the job. Container based services are getting a ton of traction internally now, especially Fargate-based ones.

You're going to have a hard time making a case for Go though. I have not a single time been convinced that Go was the best tool for the job in 5 years at Amazon.

> It also seems Amazon is obsessed with reinventing wheels and keeping their stuff internal, which is worrying.

I have not found this to be true. But I can see why people might think this. Amazon built some state of the art tooling quite some time ago, and it's starting to show it's age. Rather than drop the internal stuff for new OSS alternatives, they've continued to add modern features to the internal tools, which I think is the right choice overall considering the scale at which they're used and integrated.

Again, it is about the right tool for the job. If you present a compelling case to use an OSS alternative then more than likely you'll be able to use it.

Re: On Internal Engineering Practices at Amazon

#98

Post by jrockway »

nerdponx wrote: post #87
jrockway wrote: post #85

Earlier quoted context omitted.

I worked on third-party package approvals at Google. The reasoning behind reviews was largely due to license compliance. If the license said "you have to display this license to end-users" then we had to make sure that the license was machine-readable and would be automatically bundled into the build to be displayed in that "open source licenses" section of pretty much every app ever. If the license said "by linking…

This all sounds like justified inconvenience in the name of safety at scale. As long as you have a good relationship with the package approval guys this is a fine workflow in my experience.

Yeah, I thought it was fine (I was both a reviewer and a user).

I mostly posted to provide some contrast to the folks that are saying things like "there are too many open-source libraries and they need to be approved to make sure there aren't any security problems", which is not what we did. We did not attempt to limit the use of libraries, nor did we vouch for the security-readiness of packages before they were allowed to be checked in to source control. If you think there are too many npm libraries and are looking for an example of a big tech company saying "no more!", this is not it. Use all the libraries!

Re: On Internal Engineering Practices at Amazon

#99

Post by johnnyfaehell »

vonseel wrote: post #82

As far as I know, Node.js was used inside in a limited capacity, but they had an alternative for npm for security reasons, and you had to get an npm package approved to use it internally. FWIW, this is a very good thing . A company as large as Amazon should do this with all of their repositories; even a small start-up should be doing this to mitigate suspicious packages / third-party code vulnerabilities.

This is a good idea in large companies but the only time I've ever seen this implemented was when a developer wanted who wanted to become lead dev so started hogging permissions to things and set up an internal packagist that only he could add packages to. It didn't end well for him really.

Re: On Internal Engineering Practices at Amazon

#100

Post by geggam »

mmartinson wrote: post #95
geggam wrote: post #91

Earlier quoted context omitted.

I have shared this blog posting more than I care to admit. https://mcfunley.com/choose-boring-technology It amazes me that people will put their companies and employees at risk by using the new cool stuff just because everyone else is ( looking at you k8s )

Definitely, but, I'm sure that adopting new technologies at the right time can be a huge competitive edge for startups. Sure it can fail horribly in some places where hindsight will show that the trusted stack was the right choice, but it could also mean winning a race to market and continued relevance for a company if it works out.

Startups are complete risk and should jump on the new cool and shiny toy ship.

When you start making money or wish to start making money you need to stabilize the insanity.

I have done both.

Post reply on HN