Live data from Hacker News

The 2002 mandate for internal communication systems at Amazon

sametab.com

11–20 of 187 posts

Re: The 2002 mandate for internal communication systems at Amazon

#11

> 6) Anyone who doesn’t do this will be fired. So I've never worked at a company over 150 people. Is this... a normal thing for an email? Maybe I'm just one of those softies but an email with that line would throw me off my day and cause a serious hit to my morale and confidence of working there.

[deleted]

Re: The 2002 mandate for internal communication systems at Amazon

#12
post #5
post #2

Yegge's post was very interesting reading, and I took similar learnings away from it. I was at Amazon at the time, however, and there were things that certainly weren't true any more: >3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service inte…

How do services talk to each other without an API? Is it something like "put a non-well-documented object into a queue?"

Call out to a shell? Pass structs between C-based programs using sockets? Use runtime marshalling? Make your "API" be just different arbitrary JSON objects mapping to maps? Hide the entire API behind a lazy cache with undocumented side effects when cache misses occur? I wouldn't consider any of these "API"s although they are interfaces.

Re: The 2002 mandate for internal communication systems at Amazon

#13
I worked at an organization that had a similar declaration. Here's how it played out:

1. Everyone is super excited for other teams to share their data

2. Everyone wants an exception from sharing their own data because it's too hard or too sensitive to share.

3. Eventually everything gets shared, but it takes 3-4 times longer than it really should.

Re: The 2002 mandate for internal communication systems at Amazon

#14
post #10
post #5

Earlier quoted context omitted.

How do services talk to each other without an API? Is it something like "put a non-well-documented object into a queue?"

Some methods I've commonly seen in Enterprise Duct Tape: Screen scrape the other service and do data exchange via a Selenium script. Directly interact with the other service's database. CSV files and nightly batch jobs.

This is one of the niches where (S)FTP and batch processing is still alive and kicking.

Re: The 2002 mandate for internal communication systems at Amazon

#15
post #5
post #2

Yegge's post was very interesting reading, and I took similar learnings away from it. I was at Amazon at the time, however, and there were things that certainly weren't true any more: >3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service inte…

How do services talk to each other without an API? Is it something like "put a non-well-documented object into a queue?"

A queue if you're lucky!

There's also:

- Hire an intern / "Customer Service Representative" / "Technical Account Specialist" to manually copy data from one service into another

- Dump some file in a directory and hope something is treating that directory like a queue

- Read/write from the same database (/ same table)

Or the classic Unix trajectory of increasingly bad service communication:

- Read/write from the same local socket and (hopefully) same raw memory layouts (i.e. C structs) (because you've just taken your existing serialized process and begun fork()ing workers)

- that, but with some mmap'd region (because the next team of developers doesn't know how to select())

- that, but with a local file (because the next team of developers doesn't know how to mmap())

- that, but with some NFS file (for scaling!)

- that, but with some hadoop fs file (for big data!)

Obviously all of these are at some level an 'application programming interface'. But then, technically so is rowhammering the data you want into the next job.

Re: The 2002 mandate for internal communication systems at Amazon

#16
post #9

> 6) Anyone who doesn’t do this will be fired. So I've never worked at a company over 150 people. Is this... a normal thing for an email? Maybe I'm just one of those softies but an email with that line would throw me off my day and cause a serious hit to my morale and confidence of working there.

I strongly believe that Steve was exaggerating for effect here. In my 17 years at Amazon I have never seen or heard of a threat of this nature. The overall intent of the email was to tell teams to decouple, decentralize, and to own their own destinies.

Thanks for the reality check. Phew.

Re: The 2002 mandate for internal communication systems at Amazon

#18
post #2

Yegge's post was very interesting reading, and I took similar learnings away from it. I was at Amazon at the time, however, and there were things that certainly weren't true any more: >3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service inte…

Rules are meant to be broken as the saying goes.
Post reply on HN