IIRC when Yegge accidentally posted that rant, the entirety of Amazon corp got IP banned from Hacker News from _everyone_ rushing to view and comment.
The 2002 mandate for internal communication systems at Amazon
111–120 of 187 posts
Re: The 2002 mandate for internal communication systems at Amazon
#112Earlier quoted context omitted.
Some methods I've commonly seen in Enterprise Duct Tape: Screen scrape the other service and do data exchange via a Selenium script. Directly interact with the other service's database. CSV files and nightly batch jobs.
I raise you a service which gets it's configuration from a table on confluence page
Confluence is at least canonically XHTML so this is better than a lot of data lake bullshit I've seen.
Re: The 2002 mandate for internal communication systems at Amazon
#113> 6) Anyone who doesn’t do this will be fired. So I've never worked at a company over 150 people. Is this... a normal thing for an email? Maybe I'm just one of those softies but an email with that line would throw me off my day and cause a serious hit to my morale and confidence of working there.
I know of one CIO (between 10k and 1k employees) who operates by tell/train/terminate. Tell the staff what to do, if they can’t then train them, if they’ve been trained and can’t/won’t then terminate them.
Isnt that competence?
Re: The 2002 mandate for internal communication systems at Amazon
#114Yegge's post was very interesting reading, and I took similar learnings away from it. I was at Amazon at the time, however, and there were things that certainly weren't true any more: >3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service inte…
(Former Amazonian, part of the team that drove the change to SOA at the time) > Now I think that this internal email is what has actually stuck with me the most. Bezos realized that he had to change the internal communication infrastructure [..]. > He understood that a radical organizational change was required to arrange the internal dynamics in a way that would allow the creation of something like AWS. This is quit…
> Second, Kindle and SimpleDB (now Dynamo)
Minor nitpick, these are two separate services. SDB is still independently accessible and probably will be for a while. That being said they definitely don't encourage any new use of SDB and push Dynamo instead.
Kind of a shame because a modern NoSQL store with SQL support would be great for rapid prototyping before a concrete data schema is established.
Re: The 2002 mandate for internal communication systems at Amazon
#115Yegge's post was very interesting reading, and I took similar learnings away from it. I was at Amazon at the time, however, and there were things that certainly weren't true any more: >3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service inte…
(Former Amazonian, part of the team that drove the change to SOA at the time) > Now I think that this internal email is what has actually stuck with me the most. Bezos realized that he had to change the internal communication infrastructure [..]. > He understood that a radical organizational change was required to arrange the internal dynamics in a way that would allow the creation of something like AWS. This is quit…
(Another former Amazonian from the same team)
ozgune is correct as far as I remember.
The beginnings of AWS, though, were for the retail site. We exposed search and browse and item metadata via an API first for a deal with AOL to provide them with product search and then later we opened it up to everyone. People built stuff like Simple Amazon on top of it, which I thought was pretty nice. I don't think this part of AWS exists anymore (please correct me if I'm wrong).
The first service exposed as what we'd today consider AWS was SQS and at the time it was kind of a head scratcher. Only later did we understand that it was glue for other services.
S3 and EC2 were quite a bit later I think.
Re: The 2002 mandate for internal communication systems at Amazon
#116Earlier quoted context omitted.
(Former Amazonian, part of the team that drove the change to SOA at the time) > Now I think that this internal email is what has actually stuck with me the most. Bezos realized that he had to change the internal communication infrastructure [..]. > He understood that a radical organizational change was required to arrange the internal dynamics in a way that would allow the creation of something like AWS. This is quit…
> Second, Kindle and SimpleDB (now Dynamo) Minor nitpick, these are two separate services. SDB is still independently accessible and probably will be for a while. That being said they definitely don't encourage any new use of SDB and push Dynamo instead. Kind of a shame because a modern NoSQL store with SQL support would be great for rapid prototyping before a concrete data schema is established.
I actually really liked Simple DB as a place for configuration.
Re: The 2002 mandate for internal communication systems at Amazon
#117Earlier quoted context omitted.
Some methods I've commonly seen in Enterprise Duct Tape: Screen scrape the other service and do data exchange via a Selenium script. Directly interact with the other service's database. CSV files and nightly batch jobs.
This is one of the niches where (S)FTP and batch processing is still alive and kicking.
its also quite common in fixed data logging applications, such as exports from bms.
Re: The 2002 mandate for internal communication systems at Amazon
#118Earlier quoted context omitted.
How do services talk to each other without an API? Is it something like "put a non-well-documented object into a queue?"
A queue if you're lucky! There's also: - Hire an intern / "Customer Service Representative" / "Technical Account Specialist" to manually copy data from one service into another - Dump some file in a directory and hope something is treating that directory like a queue - Read/write from the same database (/ same table) Or the classic Unix trajectory of increasingly bad service communication: - Read/write from the same…
> Dump some file in a directory and hope something is treating that directory like a queue
Or it's unencrypted files on an FTP server containing literally the lifeblood of the American economy: https://engineering.gusto.com/how-ach-works-a-developer-pers... -_-
Re: The 2002 mandate for internal communication systems at Amazon
#119Earlier quoted context omitted.
Information doesn't have to flow through managers specifically and it certainly doesn't have to flow up the management chain and down again, but there ideally should be preferred funnels for the communication and breaking out of it should be an exception, or pre arranged, and not the norm. Otherwise you'll have, say, a particularly useful person on a team see his time filled with requests coming from a lot of differe…
If you have smart employees respectful of other's time it shouldn't be a problem. I would tend to agree with Musk on this one. Processes are often put in place to counteract poor hiring. If you need inefficiency raising processes to defend your business from your own employees, maybe you don't have the right employees.
The problem is if 12 teams all are independently interested in talking to a specific person. Even if that could be condensed in a few meetings organizing it should not fall on the unlucky employee of interest.
Handling these complexities of scale are exactly what managers are for.
Re: The 2002 mandate for internal communication systems at Amazon
#120Earlier quoted context omitted.
I'm assuming Yegge was referring to the RPC framework.
"an internal communication system" does sound like something like an "RPC framework", but Yegge's paraphrase actually says "It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter. Bezos doesn’t care." I read this as saying different teams/services don't have to use the same thing either. That doesn't sound like an "RPC framework" or "an internal communications system" at al…
This edict was before my time at Amazon, so I can't speak to whether there was an RPC framework in existence when this was mandated.
By the time I arrived, however, there was a cross-language RPC framework that integrated with Amazon's monitoring, request tracing, and build infrastructure (for building and releasing client versions). It was very full-featured and the de-facto system for creating a service. Most of our communication in my organization was done using this framework, and systems that violated the "only communicate over a service boundary" mandate were real problem children.