Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Laying myself off from Amazon

daniel.do

Post reply on HN
Print view View original

141–150 of 374 posts

Re: Laying myself off from Amazon

#141

Post by sbarre »

rad_gruchalski wrote: post #14
nyarlathotep_ wrote: post #3

Worked for AWS for just under a year and a half; this mirrors my experience exactly. Poorly designed, failure-prone, brittle internal tooling was a time and energy sink to the point where even the most trivial deployment change was a nail-biter. Automated tooling and tests that were ostensibly created to make life easier were the number one pain point and constantly failed in obscure ways that required cutting ticket…

> Poorly designed, failure-prone, brittle internal tooling was a time and energy sink to the point where even the most trivial deployment change was a nail-biter. Considering rather good quality APIs they expose to the external world, this is shocking to read.

Internal tools never get the same attention as public-facing products.

Public facing products have competition. Internal tools have a captive audience that must use them, no matter what.

Re: Laying myself off from Amazon

#142

Post by hinkley »

nyarlathotep_ wrote: post #3

Worked for AWS for just under a year and a half; this mirrors my experience exactly. Poorly designed, failure-prone, brittle internal tooling was a time and energy sink to the point where even the most trivial deployment change was a nail-biter. Automated tooling and tests that were ostensibly created to make life easier were the number one pain point and constantly failed in obscure ways that required cutting ticket…

I worked a contract there and the team barely understood how some of those configuration panes functioned. It was always a case of staring at someone else's config and then at yours until you found enlightenment. And the target kept changing, so there was a very bad week where my computer worked for 80 minutes total. I coded like hell during that time, pair programmed with the other contractors or did math on a whiteboard for the rest.

Another guy on the team intimated once in a while that he didn't like the team dynamics, but he played his cards pretty close to the chest. After he gave notice, on what we all thought was supposed to be his last day, we showed up in the morning just to find his computer and all of his equipment in a pile on a desk. We never did figure out if we or he were off by one or it was his middle finger to the team.

All the more mysterious because later that day we did discover his middle finger: The lead dev, who seemed to be one of the more reasonable people on the team, and the resident know-it-all both discovered that someone had fiddled with their configurations in ways that broke the system but produced obscure errors and hard to notice differences (like punctuation).

I took a couple lessons away from that project, and reinforced some that I'd already had. Perhaps the most important of which is: Bragging rights for being first mover are very, very short-lived. Once everyone else has copied it, the most accurate adjective for your version is 'old', or 'primitive', not 'first', and absolutely under no circumstances 'best'. Developers don't like bullshit. Don't pretend like your liabilities are assets.

Re: Laying myself off from Amazon

#143

Post by mancerayder »

danrocks wrote: post #98
outside1234 wrote: post #55

Why does anyone work there?

Former Amazonian here. The learning is unmatched. I've been in different FAANGs but the way that Amazon pushes one to learn and challenge the status quo is unthinkable. I never saw "no, you can't do that, that system/business/area is sacred". Everything is up for grabs all the time. While there are pathological side effects to this (AWS promotion-per-launch, constantly battle for scope), the amount of knowledge one c…

>AWS promotion-per-launch

Can you elaborate? You mean a new service gets someone promoted?

Re: Laying myself off from Amazon

#144

Post by shitlord »

spike021 wrote: post #138
fdgsdfogijq wrote: post #28

Earlier quoted context omitted.

You should have just switched teams. Amazon has amazing teams working on deep technology, they also have huge systems with technical debt and stress. People dont realize how different it can be inside the company.

Before I left Amazon I tried transferring to multiple teams after speaking with hiring managers. They all (IIRC 4 separate ones) wanted either a full on-site loop or at minimum two coding interviews). I was an L5 with about 5-6 years of experience, 1.5 at that point at Amazon.

Hiring managers have a lot of latitude at Amazon, so there isn't a consistent standard. I don't want to share too much PII but I was in a similar situation. I was able to transfer teams after an informal chat with a few engineers on the team (discussing past projects, the new team's product vision, etc.). There was no whiteboarding involved.

Re: Laying myself off from Amazon

#145

Post by scubbo »

belval wrote: post #111
scubbo wrote: post #59

Earlier quoted context omitted.

This is really baffling to me. In fact, since leaving Amazon in June, I've been really frustrated by how much extra work setting up a CI/CD pipeline is in (say) Drone and Argo/Flux than with Amazon's internal tooling. You can set up a standard Amazon pipeline with a single command and answering a few prompts about naming, whereas with the open-source systems you have to hand-craft everything yourself, hack in a way t…

Perhaps it's just a green engineer thing. I know GitHub actions, CircleCI, Drone, etc... I don't know pipelines and sometimes it feels like dark magic to me. As far as the other tooling goes, I often find that they are very Java-oriented (which is fine considering the AWS background) and everything in Python feels janky.

Ah! OK, now "dark magic" I can totally understand. Pipelines _is_ a little opaque, which is a totally fair criticism. In my experience it does _most_ of the things that you would want if you know a few fairly common (and easily searchable) incantations ("How do I add a new stage to this pipeline?", "How do I put tests after this stage?", etc.) - but I can definitely sympathise with the feeling of "this tool is doing a bunch of stuff for me and I don't understand how it's doing it". Thanks!

One point that I've found _really_ strange after transitioning from Pipelines to other CI/CD (I'm using Drone and Argo, but I think this applies for Flux as well) is that infrastructure-definition repos define the _image-version_ that they deploy to a particular stage, rather than just defining the source repo (and automatically deploying the latest image-version which has passed all preceding tests). That seems...odd. Unless you hack together your own automation[0], that means you need two actions to get new App Code out to an environment:

* Push the App Code commit (and have a new image-version built)

* Make a change to your Infra Code, updating the desired image version

Am I missing something?

---

I'm surprised to hear that you consider AWS to be Java-oriented. When I think AWS, I think TypeScript - to the extent that the primary reason I started learning TypeScript was to be able to write CDK more fluently (and, then, to write Lambdas without a whole bunch of Java type boilerplate - though Python also works there).

[0] Which I've done, but the fact that I had to hand-build it suggests I'm doing something that the tool author's discourage

Re: Laying myself off from Amazon

#146

Post by jerglingu »

delecti wrote: post #121

Has the tooling gotten dramatically worse in the past 5 years? I willingly left almost exactly 5 years ago because my team's responsibilities had shifted too much, but I remember the tooling being pretty great. Also, everyone being more or less on the same stack also meant that there was a ton of resources to solve problems, just search the wiki for your error message. I still miss how well Odin worked for service cr…

I left about a year ago, but the universality of tools and stack was not a thing in the ~3 years I was there. Retail and AWS are split and use different stacks. Additionally, the company was in a drawn out multi-year process of deprecating old tooling (including Odin), so you had teams or orgs that were in-between these old internal services and new internal services (or off internal services altogether and on "native AWS"). If cross-team interoperability demanded that certain tooling was to be used, many times you could inadvertently hold back or screw over another team by deciding to stay or move to the new tooling.

Not to say it was all bad - some of it was certainly necessary. Especially for security - IMO the systems of Amazon are great with security, but not necessarily the spirit of the company. Without all that bureaucratic annoyance, there 100% would be existential leaks or breaches coming out of Amazon with how fast and loose everybody treated security in the name of "Deliver Results" .

Re: Laying myself off from Amazon

#147

Post by innocentoldguy »

I just laid myself off from Amazon too after 2+ years. I just couldn't take the janky-as-shit internal tooling, the myriad pointless/redundant meetings, and pointlessly stressful and toxic work environment anymore.

Also, I enjoyed my first two managers at Amazon, but my third manager was a tool, which is a systemic problem at Amazon.

I'm glad to have Amazon in the rearview mirror of my career.

Re: Laying myself off from Amazon

#148

Post by Chamix »

lozenge wrote: post #62
fdgsdfogijq wrote: post #28

Earlier quoted context omitted.

You should have just switched teams. Amazon has amazing teams working on deep technology, they also have huge systems with technical debt and stress. People dont realize how different it can be inside the company.

How can you tell ahead of time if the team you're switching to has it any better?

Tech survey results, ops load via CTI tickets, CR/LOC stats, accolades, COEs, promotion rate, turnover/attrition....

There are so many data sources that are all queryable via API. There are a couple basic greasemonkey scripts already floating around but as an SDE it really isn't that hard to write something yourself... It does take a bit of experience to get a good heuristic on weighting the different data points but you can do way better than blindly guessing.

I will say as someone who HAS intelligently looked through this data for most teams at Amazon, there is an absolutely MASSIVE difference between the top 10% of teams and the median, and an even larger difference between than and the bottom 10%, which by all metrics appear to truly be hell on earth.

Re: Laying myself off from Amazon

#150

Post by innocentoldguy »

gruffle wrote: post #81
olladecarne wrote: post #36

My experience has been similar at other large companies (MSFT, Google). My advice is to look for founder-led teams (not companies). If there are plenty of founders still on the team, they will likely care a lot about the product and code and hold everyone to high standards which make the work easier in the long run. On the other hand when I've been on teams where all the original founders left, it's been a constant s…

Yup, and the other flipside is devs being extremely nitpicky and regurgitating irrelevant Amazon principles BS in code reviews because they're trying to get promoted.

Agreed! If I never hear "You should employ a bias towards " again it will be too soon.
Post reply on HN