Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 2002 mandate for internal communication systems at Amazon

sametab.com

Post reply on HN
Print view View original

161–170 of 187 posts

Re: The 2002 mandate for internal communication systems at Amazon

#161

Post by dodobirdlord »

everdrive wrote: post #87
solarengineer wrote: post #45

Earlier quoted context omitted.

An ex Netflix person, who has since moved to Amazon, spoke at a client place three weeks ago. He casually mentioned things such as "we forgive the first time, and we fire the second time". From how he spoke, we felt that this may be the norm in Silicon Valley and related places. I have much respect for what he has achieved, so I didn't interrupt to question such a fear-inducing mindset.

With such important services as streaming Seinfeld, it's easy to see why such a scorched earth policy is necessary.

There are compounding productivity boosts available when a team can all trust each other to basically never cut corners or make sloppy mistakes. Removing a tenth team member who is not up to the bar of the rest of the team can make the remaining 9 members each more than 11% more productive.

Of course, this strategy has its downsides. You can't ever hire juniors. You can't really hire people in and train them up at all, because everything has been built under the assumption that only experts will ever touch it. This makes an organization that operates like this inherently parasitic to the industry, only capable of hiring in experienced employees from other companies.

Re: The 2002 mandate for internal communication systems at Amazon

#162

Post by throwawayy98121 »

Hi! I’m a senior engineer at Amazon. Throwaway account but I’ll try to respond to questions if anyone cares to ask.

Yeah we use services heavily, but there’s plenty of teams dumping data to S3 or using a data lake.

There’s also the “we need to do this but management doesn’t see value so let’s dump it on the intern or SDE 1, who we won’t really mentor or guide and then blame, forcing them to switch teams as soon as they can.”

If you work at another company and think we have our stuff figured out at Amazon, we really don’t. We have brilliant people, many of who are straight up assholes who will throw you under the bus. We have people who are kind and will help you gain all kinds of engineering skills. We also have people who are scum of the Earth shit people who work at Amazon because I don’t think any other sane company or workplace would tolerate them. We have extremes on the garbage people end of the spectrum, unfortunately.

Sorry long rant - point being - it’s good to learn how we do things. The internal email on services is pretty unique. I learned about it when I was an SDE 1 back in the day. But - don’t take it as gospel. It doesn’t mean you need to build services.

I can think of any number of examples where we follow anti patterns because no one gives a shit about the pattern, whether it’s a service, a bucket, a queue, or a file attached to the system used for scrum tasks, or shit passed over email... we care about value at the end of the day. If you don’t provide sufficient value at Amazons bar, they have no problem tossing you out the window.

Re: The 2002 mandate for internal communication systems at Amazon

#163

Post by nameistaken »

flukus wrote: post #149
jhallenworld wrote: post #110

Earlier quoted context omitted.

IBM has an appliance for you: https://www.ibm.com/products/datapower-gateway It's a middleware appliance that allows any API to talk to any other API and handles just about any data format. You can script it with javascript or XSLT. It can handle ad-hoc things like ftp polling. It has the added benefit that you can add security for outside facing clients. Disclaimer: I helped develop this appliance (but I no longer w…

Pricing: Contact Us. So basically it will be expensive, require an army of IBM consultants and become yet another integration point instead of actually solving anything.

Cries in Webseal.

Re: The 2002 mandate for internal communication systems at Amazon

#164

Post by jugg1es »

flukus wrote: post #149
jhallenworld wrote: post #110

Earlier quoted context omitted.

IBM has an appliance for you: https://www.ibm.com/products/datapower-gateway It's a middleware appliance that allows any API to talk to any other API and handles just about any data format. You can script it with javascript or XSLT. It can handle ad-hoc things like ftp polling. It has the added benefit that you can add security for outside facing clients. Disclaimer: I helped develop this appliance (but I no longer w…

Pricing: Contact Us. So basically it will be expensive, require an army of IBM consultants and become yet another integration point instead of actually solving anything.

IBM's business model is totally antiquated and exhausting for modern processes. We have had a nightmare trying to get IBM MDM's solution to finally admit they were not actually cloud-ready after saying repeatedly that they were. No TLS support for DB2 out of the box for K8 support, documentation sucks. But contact us for pricing. IBM sucks.

Re: The 2002 mandate for internal communication systems at Amazon

#165

Post by Infernal »

invisiblerobot wrote: post #113
Infernal wrote: post #63

Earlier quoted context omitted.

I know of one CIO (between 10k and 1k employees) who operates by tell/train/terminate. Tell the staff what to do, if they can’t then train them, if they’ve been trained and can’t/won’t then terminate them.

Isnt that competence?

My intention wasn’t to judge the position, just pass along the data point.

Re: The 2002 mandate for internal communication systems at Amazon

#166

Post by cpitman »

leoh wrote: post #5
Twirrim wrote: post #2

Yegge's post was very interesting reading, and I took similar learnings away from it. I was at Amazon at the time, however, and there were things that certainly weren't true any more: >3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service inte…

How do services talk to each other without an API? Is it something like "put a non-well-documented object into a queue?"

Another favorite:

You integrate a "service" by creating and linking a library that implements the entire service, including data access. Now try tracking down everything accessing the "service" database, or rolling out an upgrade to the "service".

Re: The 2002 mandate for internal communication systems at Amazon

#167

Post by wanderer2323 »

saalweachter wrote: post #124
morelisp wrote: post #15

Earlier quoted context omitted.

A queue if you're lucky! There's also: - Hire an intern / "Customer Service Representative" / "Technical Account Specialist" to manually copy data from one service into another - Dump some file in a directory and hope something is treating that directory like a queue - Read/write from the same database (/ same table) Or the classic Unix trajectory of increasingly bad service communication: - Read/write from the same…

Don't forget the most important step. "Think of the acronym CSV. Don't look up the definition of the format, just meditate on the idea of the format for a bit. Then write your data in the format you have just imagined is CSV, making whatever choices you feel personally best or most elegant regarding character escapes. Pass this file on to your downstream readers, assuring them it is a CSV file, without elaborating on…

This is awesome, where does it come from? Google does not give me anything.

Re: The 2002 mandate for internal communication systems at Amazon

#168

Post by [deleted] »

wanderer2323 wrote: post #167
saalweachter wrote: post #124

Earlier quoted context omitted.

Don't forget the most important step. "Think of the acronym CSV. Don't look up the definition of the format, just meditate on the idea of the format for a bit. Then write your data in the format you have just imagined is CSV, making whatever choices you feel personally best or most elegant regarding character escapes. Pass this file on to your downstream readers, assuring them it is a CSV file, without elaborating on…

This is awesome, where does it come from? Google does not give me anything.

[deleted]

Re: The 2002 mandate for internal communication systems at Amazon

#169

Post by badpun »

thaeli wrote: post #10
leoh wrote: post #5

Earlier quoted context omitted.

How do services talk to each other without an API? Is it something like "put a non-well-documented object into a queue?"

Some methods I've commonly seen in Enterprise Duct Tape: Screen scrape the other service and do data exchange via a Selenium script. Directly interact with the other service's database. CSV files and nightly batch jobs.

Directly interacting with other service's DB is an "Enterprise Integration Pattern" AFAIR? It can make sense in lots of cases.

Re: The 2002 mandate for internal communication systems at Amazon

#170

Post by arethuza »

badpun wrote: post #169
thaeli wrote: post #10

Earlier quoted context omitted.

Some methods I've commonly seen in Enterprise Duct Tape: Screen scrape the other service and do data exchange via a Selenium script. Directly interact with the other service's database. CSV files and nightly batch jobs.

Directly interacting with other service's DB is an "Enterprise Integration Pattern" AFAIR? It can make sense in lots of cases.

Reading data from another application's database is pretty common (although even this can cause chaos if done without some care) but writing to application databases is often a very bad idea and often explicitly forbidden by CRM/ERP vendors.
Post reply on HN