Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On Internal Engineering Practices at Amazon

jatins.gitlab.io

Post reply on HN
Print view View original

121–130 of 130 posts

Re: On Internal Engineering Practices at Amazon

#121

Post by vonseel »

johnnyfaehell wrote: post #99
vonseel wrote: post #82

As far as I know, Node.js was used inside in a limited capacity, but they had an alternative for npm for security reasons, and you had to get an npm package approved to use it internally. FWIW, this is a very good thing . A company as large as Amazon should do this with all of their repositories; even a small start-up should be doing this to mitigate suspicious packages / third-party code vulnerabilities.

This is a good idea in large companies but the only time I've ever seen this implemented was when a developer wanted who wanted to become lead dev so started hogging permissions to things and set up an internal packagist that only he could add packages to. It didn't end well for him really.

That's unfortunate, but how can you be sure this was his motivation or reasoning for adding an internal repo manager? I'm only asking because I learned the practice from an engineer / CTO whom I looked up to greatly early in my career, and if I were to go work somewhere and push leadership to implement Artifactory/Nexus/etc., I hope people wouldn't misunderstand my intentions. Anyone pushing for something like that is naturally likely to be the repo administrator in the beginning, and that may look like hogging permissions even if the developers intent is something entirely different? Catch my drift?

Re: On Internal Engineering Practices at Amazon

#122

Post by geggam »

alfalfasprout wrote: post #114
geggam wrote: post #91

Earlier quoted context omitted.

I have shared this blog posting more than I care to admit. https://mcfunley.com/choose-boring-technology It amazes me that people will put their companies and employees at risk by using the new cool stuff just because everyone else is ( looking at you k8s )

This is one position... albeit an extreme one. Kubernetes has worked perfectly for me in multiple places now. Including where I am now at a large unicorn. I agree, picking up all the shiny new stuff on hacker news is generally a bad idea. But foresaking new technologies altogether can backfire. Part of it is how you handle failures. You could always have an aerospace-grade QA process... but for a consumer web product…

Problem most people forget is you still have to manage and run the servers or pay someone to do so.

The experience required to manage k8s isnt typically a consumer grade website type admin.

opex or capex ... pick where you want the spend

Re: On Internal Engineering Practices at Amazon

#123

Post by m0zg »

scott_s wrote: post #119
m0zg wrote: post #113

Earlier quoted context omitted.

There is no "another networking driver" in containers. It's the same networking driver.

For standard Docker container deployments, there is. See: https://docs.docker.com/network/

That's not a "driver" in the traditional sense, meaning it doesn't talk to the actual hardware directly. It's just a bridge or iptables.

Re: On Internal Engineering Practices at Amazon

#124

Post by geggam »

blub wrote: post #112
geggam wrote: post #100

Earlier quoted context omitted.

Startups are complete risk and should jump on the new cool and shiny toy ship. When you start making money or wish to start making money you need to stabilize the insanity. I have done both.

Something that's "complete risk" should take even more risk? Doesn't seem logical.

so when you are swimming you hold one leg in the air or do you keep everything in the water so you can swim faster ?

Re: On Internal Engineering Practices at Amazon

#125

Post by throwaway504 »

tanilama wrote: post #59
ecarrlee wrote: post #31

Earlier quoted context omitted.

I was an intern at AWS this past summer(and will be returning as a new grad in May) and this article is not accurate based on my experience. My code (which was part of our team's actual production system, not a toy project) ran on EC2 instances in an autoscaling group. There was no manually provisioning servers. I worked with a Kinesis Stream and several other resources and there was a way for us to programmatically…

The Java claim is just factually false. Java/C++/Python/Ruby/Go/Perl/Scala, even Rust, are used in Amazon for various projects. What might be true is that, consider how critical your service would be, the more it is, people will lean towards to more conservative/mature/enterprise-ish languages, not exotic ones, because that inherently means risk. And Java is the go-to option under that consideration, just because, it…

The L8 manager of our org declared that all projects must be on the JVM, and if the language wasn't Java, then we would need to submit a six pager explaining why. Perhaps he was just being conservative, but to me there's something off-putting when those types of decisions come from managers rather than from the engineers.

That said, our org did manage to have a few projects that aren't Java, so I guess I would have to concede that the Java claim is factually false as well. But it sure doesn't feel like I am empowered to choose something other than Java.

Re: On Internal Engineering Practices at Amazon

#126

Post by mundu_wa_hinya »

discodave wrote: post #26
throwaway772643 wrote: post #19

I will be joining Amazon in about a month. Is there any chance I'll be able to work on OSS and/or "modern" tech (e.g. containers, Go, etc.) without a ton of push-back? It also seems Amazon is obsessed with reinventing wheels and keeping their stuff internal, which is worrying. Is there any chance to introduce solid OSS tools to the development process? (whatever they might be)

AWS SDE here. The short answer is, in order to get your team to adopt something, you need to make the case that it's better for customers (including things like migration costs). If the modern thing is more efficient, is higher availability, increases velocity, and so on then the case can be made. Some specific examples based on things you cite: * For an example of something OSS or "modern" coming from AWS, checkout…

Worked for EC2 as an SE ~ 5 years ago. We used to handle rack downs and page the relevant team if there was a large set of instances for the said team impacted. We once had a couple of hundreds of Amazon.com (merchant team) instances impacted. We paged the team and they were like "don't page us for anything less than 10 racks of our instances down". The burgers didn't even feel the impact. Their automation was insane.

Re: On Internal Engineering Practices at Amazon

#127

Post by HereBeBeasties »

IloveHN84 wrote: post #66

To me, it sounds like a company who is still working with the mentality of 90s. The same I'm facing at work, where new technology is treated as scary and there's fear of change something that (somehow, only God knows how) is working and none is allowed to touch such things

New technology is scary. It should be scary. If you don't find it scary you're probably not looking at it right. It's usually full of risk, which if you are in management (or a remotely responsible employee) needs to be evaluated and if possible contained, and ultimately needs to pay for itself (i.e. Offers a pay-off that more than outweighs the risk).

If we are honest with ourselves, most new tech in this industry totally fails that litmus test.

There are an awful lot of shiny-bauble chasers and snake oil salesmen both inside and outside companies.

I don't know what your precise situation is and obviously it's very much a spectrum rather than black and white, but a healthy aversion to promises made by the authors and salespeople of relatively unproven new things is extremely valuable.

Re: On Internal Engineering Practices at Amazon

#128

Post by scott_s »

m0zg wrote: post #123
scott_s wrote: post #119

Earlier quoted context omitted.

For standard Docker container deployments, there is. See: https://docs.docker.com/network/

That's not a "driver" in the traditional sense, meaning it doesn't talk to the actual hardware directly. It's just a bridge or iptables.

shrug It's additional software in the networking stack, and it has a performance penalty. I think it's fair to call it a "virtualization cost," since you're paying for the container's networking abstraction.

Re: On Internal Engineering Practices at Amazon

#129

Post by jypepin »

geggam wrote: post #94
jypepin wrote: post #70

> "It's complicated, so it's gotta be good. I must be dumb to not get it." Having worked with aws a lot recently, this article doesn't surprise me at all actually. When you see the low quality of UI and documentation for most of their tools that users pay for, I wouldn't expect their internal tooling to be any better. I'm not saying their tech is bad, once things work, it works great - I'm talking about the usability…

If you are using the UI to leverage AWS you arent really leveraging AWS. AWS is designed to be used via automation and the API. Disclaimer : I dont work for AWS but have spent many years building relatively large stacks on AWS (thousands of ec2 instances with monthly spend being a couple supercars in value)

yes, but before using the API you need to know which API to use. If you are looking into using a new service, say, analytics. Which coupling of services do I use, between cloudwatch, firehose, etc etc. Or reading documentation for example. They have terrible documentation.

Someone using their API/cli is already familiar with aws and how it works. The point, before getting to that point, you need good docs and UI to allow people to discover and learn.

Re: On Internal Engineering Practices at Amazon

#130

Post by _msw_ »

School-Cotton wrote: post #55
ecarrlee wrote: post #31

Earlier quoted context omitted.

I was an intern at AWS this past summer(and will be returning as a new grad in May) and this article is not accurate based on my experience. My code (which was part of our team's actual production system, not a toy project) ran on EC2 instances in an autoscaling group. There was no manually provisioning servers. I worked with a Kinesis Stream and several other resources and there was a way for us to programmatically…

I can confirm that the claim "you can't even use a language other than Java" is completely false. Java is often the best tool for the job because of how much institutional inertia there is behind it, but certainly nobody will stop a team from using a better tool. PHP is banned at Amazon, and that's the exception that proves the rule: a language being banned is so unusual that I'm aware specifically of the example of…

And, yet, there are some places where PHP is used (under explicit conditions and exceptions).
Post reply on HN