Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The “API Mandate” memo at Amazon

chrislaing.net

Post reply on HN
Print view View original

111–120 of 136 posts

Re: The “API Mandate” memo at Amazon

#111

Post by ItsMonkk »

flakiness wrote: post #14

Is this the same doc as "Distributed Computing Manifesto" mentioned in Werner Vogels' blog? [1] These legendary Amazon Memos haven't leaked, unlike Billg's various memos [2]. That is... journalistically unfortunate, I would say. I even wonder current Amazonian actually has the access to these docs. [1] https://www.allthingsdistributed.com/2019/08/modern-applicat... [2] https://lettersofnote.com/2011/07/22/the-interne…

I posted a comment on libraries vs frameworks yesterday[0], here's the relevant section.

> Frameworks are easier to setup initially, but they do not scale. Why is it that Microsoft Windows has 13 different dialog generations? Because each is a framework on top of a framework on top of a framework. It's amazing that they can even get that done.

> On the other side, OSS is generally built on libraries. When the 2 UNIX devs were in a basement building UNIX and were able to out-compete Multics[1], they did it because they were building libraries that could talk with each-other using pipes around the boundary. Applications that communicate based on input-output with no internal state behave just like pure functions do. Pure functions compose. When Linus built git in 10 days, he was able to do this because the core idea of git isn't actually that much work. The library is built out of composable blocks that neatly come together. Microsoft's TFS Source Control is a framework that acts on your behalf and therefore the bigger the project gets, you need n^2 people to work on it.

Amazon's API mandate is the same exact unification as UNIX's pipe effect was to Operating Systems. Amazon's internal teams are building libraries whereas all other companies started at the same time were building frameworks. Jeff was brilliant to see this at the time, and I expect that this memo will be seen as just as pivotal as the Toyota Production System, and it might already have that prestige to some. Unfortunately for other companies, you can't retrofit into it and rewrites are always a terrible idea[2].

[0]: https://news.ycombinator.com/item?id=27570917

[1]: https://www.youtube.com/watch?v=3Ea3pkTCYx4, thanks to this HN comment(https://news.ycombinator.com/item?id=27494671) for this reference.

[2]: https://www.joelonsoftware.com/2000/04/06/things-you-should-...

Re: The “API Mandate” memo at Amazon

#112

Post by jimbokun »

travisgriggs wrote: post #42

How far does this principle scale? I've heard of this strategy and thought "that's kinda cool." Amazon is obviously successful in its domain, so it would be easy to assume some causality. And yet, there's another big tech company, fruit symbol I think, that has played the long game of continually fine tuning the interaction points to make the integration of their parts more than the whole. They get credit for attenti…

> And yet, there's another big tech company, fruit symbol I think, that has played the long game of continually fine tuning the interaction points to make the integration of their parts more than the whole.

There was a recent article about another concise memo making the case to Steve Jobs for an iOS App Store, which Jobs quickly approved.

So I think the same kind of systems thinking and understanding technological ramifications of decisions at the very top, also played a large role in Apple's success.

Re: The “API Mandate” memo at Amazon

#113

Post by Macha »

jayd16 wrote: post #54

>It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter. So how does Amazon manage this btw? I don't find AWS SDKs to be all that consistent or inconsistent. They are usually good _enough_. Is that all it takes? By contrast, Google seems to spend a lot of time on their single repo, build the world, approach. For the most part it seems beloved, or at least people try to recr…

I've definitely been burned by inconsistency on Amazon API inconsistency. At least 3 years ago, Cloudformation, Data Pipeline and an EMR specific API all have three differing sets of params for defining an EMR cluster and just because something was supported in one of them didn't mean it was even in the others.

Re: The “API Mandate” memo at Amazon

#114

Post by femto113 »

simonw wrote: post #13

Does anyone know who else was involved in constructing this memo? "There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever" Was Bezos deeply enough involved in Amazon's engineering to set those rules himself, or was the text of the memo influenced by a senior engineering group that he was worki…

This was less about a memo showing up one day out of the blue and more like an attempt to bring resolution to a series of long, heated, and not terribly productive debates that took raged through the development teams over many months. It's worth knowing that the "before" state was that almost every team exposed their functionality via bespoke C/C++ libraries that everybody else linked to, and this resulted in enormous (for the time) spaghetti binaries. Want to write a little script that needs one piece of information from the database with customer information in it? No problem, just link the customer team's client library, and its 100MB of direct dependencies and 800MB of indirect dependencies. A handful of teams (notably those that already had to interface with third parties) were trying a different way (like http services written in Java) but they got a lot of side eye (or even more direct "you're doing it wrong" remarks) from the "core" developers.

Re: The “API Mandate” memo at Amazon

#115

Post by phkahler »

amazondrone12 wrote: post #98

As a current Amazon employee I can confirm this is no longer the case or only applies to such a select few pieces of software that it doesn't actually mean what you think it does. The amount of day to day stuff that is relied upon that runs on greasemonkey scripts and web scraping is insane. API's for a ton of things don't exist or are have heavy gatekeeping. (eg. contact x to get onboarded to our API and they just g…

Sounds like they're allowing people to treat the systems as "infrastructure" and "applications" with only the infra having exposed APIs? That's super common in software of all sorts. There are a lot of people who can make good use of a decent API and appreciate it, but are not experienced (or smart, or have time, etc) enough to design a good API themselves. Most software is mortar, not bricks.

Re: The “API Mandate” memo at Amazon

#116

Post by goodpoint »

xyzelement wrote: post #94
yosamino wrote: post #53

Earlier quoted context omitted.

How did we end up in a place where we demand the system of government be democratic, with all the emotional language of freedom, self-determination and so on. And then we carved out an exception and made it so our places where we work are run as oppressive dictatorships... Dont agree with what the bossman said?: "You're fired!" And the people love it so much, they even democratically elected the poster child of that…

// And then we carved out an exception and made it so our places where we work are run as oppressive dictatorships... That's just a bunch of random words you're saying. Reality: we're building a company so we can create wealth and support our families and achieve something. We have a plan for doing it. If you're not following the plan and just sabotaging everything, you're doing exactly that - sabotaging. This is dif…

> we're building a company so we can create wealth and support our families

Whose families? Most private companies state very clearly that mission is to "maximize shareholder's value". That's the whole point of being privately owned. Hiring and paying employers is a necessary evil.

That's why they need legal regulation, public supervision, even whistleblower employees.

Re: The “API Mandate” memo at Amazon

#117

Post by jimbokun »

drewcoo wrote: post #21
lxe wrote: post #8

Things like these are taken as dogma or a religion -- and are applied to all things in an organization, for better or for worse. When there's a case for tighter coupling and less services, (and yes, there are cases for it), this memo gets brought up and microservices win the argument.

What is an organizational case for tighter coupling and fewer services? Downsizing with an intent to not grow in the same direction again?

> Downsizing with an intent to not grow in the same direction again?

More like we're not really big enough yet to justify multiple services, since everything still runs on a single beefy machine and we don't have enough experience running the system yet to really know where to put the service boundaries.

Re: The “API Mandate” memo at Amazon

#118

Post by jimbokun »

kpmah wrote: post #75

I've worked at a place where this was cargo-culted and it worked horribly. All the worst aspects of 'services-first' design. I think this works much better when each team is working on what could be regarded as a complete end-to-end product e.g. a database. It works poorly when each team is working on part of a product.

If you don't get the boundaries between the various services right, the effort is going to fail spectacularly.

Re: The “API Mandate” memo at Amazon

#119

Post by Aperocky »

sgt101 wrote: post #105

I worked at a company that read the Amazon memo virtually straight after it was published and decided to copy it. The CIO at the time predicated every system's teams' bonus on it. No API, no bonus. What followed was 18mths of people building enterprise messaging systems and "buses" of various types. Hundreds of pages of documentation on APIs was prepared and released. I think 2 API's with maybe 40 methods actually we…

That's the wrong way to do it though.

You can't just have a free for all mentality when building APIs. The platform has to be there first.

Everyone's API should just need simple descriptive json files and then a client can be generated to consume that API easily. Everyone can declare those json files and it would work everywhere across the business.

This free for all with no standardization is bound to be a disaster.

Re: The “API Mandate” memo at Amazon

#120

Post by weswpg »

willshepherdson wrote: post #103
wging wrote: post #57

Earlier quoted context omitted.

The "text of the memo" is really just the article's author not understanding the context of Steve Yegge's years-later retelling, from which that text comes. "His Big Mandate went something along these lines" (Yegge's words right before the quoted text) doesn't even imply there was a singular 'memo' involved, and definitely (and obviously) wasn't meant to say that the text was actually what Bezos wrote. So the footnot…

Came here to say this - the next line made it even more clear that Yegge is making a joke. Here's a mirror of the original essay: https://gist.github.com/chitchcock/1281611 Ha, ha! You 150-odd ex-Amazon folks here will of course realize immediately that #7 was a little joke I threw in, because Bezos most definitely does not give a shit about your day.

and then the next line clarifies that only the last part of the whole story was a joke:

> #6, however, was quite real, so people went to work. Bezos assigned a couple of Chief Bulldogs to oversee the effort and ensure forward progress, headed up by Uber-Chief Bear Bulldog Rick Dalzell.

Post reply on HN