Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon’s distributed computing manifesto (1998)

allthingsdistributed.com

Post reply on HN
Print view View original

21–30 of 61 posts

Re: Amazon’s distributed computing manifesto (1998)

#21

Post by wging »

It’s interesting to think about how much of a perspective shift this must have been, especially the service oriented bits. Interestingly, I think it might not have even been made completely in the authors’ minds at the time of this proposal. (Which is understandable, of course. It’s a proposal, not a retrospective on already accepted ideas.)

For example,

> In the case of DC processing, customer service and other functions need to be able to determine where a customer order or shipment is in the pipeline. The mechanism that we propose using is one where certain nodes along the workflow insert a row into some centralized database instance to indicate the current state of the workflow element being processed.

definitely doesn’t seem to reflect the hiding of a database behind an interface. (From a workflow node’s perspective, rows in that centralized database should be an implementation detail it has no knowledge of.)

Then again, this is part of their pitch for workflow processing, not service-oriented architecture.

Re: Amazon’s distributed computing manifesto (1998)

#22

Post by PaulDavisThe1st »

As one of the main designers of the original system (but who had left by the time this architectural change was done), that is an interesting read. Always good to see the things that we missed in 1994/1995, even though we believed we were thinking far, far ahead.

Re: Amazon’s distributed computing manifesto (1998)

#23

Post by amcvitty »

nevir wrote: post #19

> We propose moving towards a three-tier architecture where presentation (client), business logic and data are separated. This has also been called a service-based architecture. The applications (clients) would no longer be able to access the database directly, but only through a well-defined interface that encapsulates the business logic required to perform the function. It is really interesting to see a recent(ish)…

A big part of the difference has to be that if you have a small number of developers (esp. n=1) and you can deploy everything at once, then those layers just get in the way of fast change. It seems Amazon were optimising for the ability to distribute data because they had big volume, and hide its form so they could change it without having to change lots of applications.

Of course, there’s some cargo culting around services where people jump to that architecture before they need it, but for most apps YAGNI. it’s cool that their architecture was driven by clear needs “just in time” to allow them to continue to scale

Re: Amazon’s distributed computing manifesto (1998)

#24

Post by tsss »

nevir wrote: post #19

> We propose moving towards a three-tier architecture where presentation (client), business logic and data are separated. This has also been called a service-based architecture. The applications (clients) would no longer be able to access the database directly, but only through a well-defined interface that encapsulates the business logic required to perform the function. It is really interesting to see a recent(ish)…

Nowadays you separate service by business capability and not by "layer". Layers just lead to a dependencies and dependencies lead to bad reliability and terrible development speed.

Re: Amazon’s distributed computing manifesto (1998)

#25

Post by ajkjk »

nevir wrote: post #19

> We propose moving towards a three-tier architecture where presentation (client), business logic and data are separated. This has also been called a service-based architecture. The applications (clients) would no longer be able to access the database directly, but only through a well-defined interface that encapsulates the business logic required to perform the function. It is really interesting to see a recent(ish)…

For what it's worth, Amazon's architecture for the core retail business has, if anything, moved even further up in abstraction. Tighter coupling is something that simple usecases can afford. Large scale but low-complexity can be closely coupled. High-complexity can't be.

The thing about Amazon's systems is that they are horrendously complex. In ~2016 I was working on the warehousing software, and it was a set of some hundreds of microservices in the space, which also communicated (via broad abstraction) to other spaces (orders, shipments, product, accounting, planning, ...) which were abstractions over 100s of other microservices.

So what I observed at the time was a broad increase in abstraction horizontally, rather than vertically. This manifesto describes splitting client-server into client-service-server; the trend two decades later was splitting into , often with services that simply aggregated the results of their subdomains for general consumption in other domains.

I'm sure things have only gotten more complicated since then (in particular, a large challenge at the time was the general difficulty in producing maintainable asynchronous workflows, so lots of work was being done synchronously or very-slightly-asynchronously that should have been done in long-running or even lazy workflows).

Re: Amazon’s distributed computing manifesto (1998)

#26

Post by derefr »

tsss wrote: post #24
nevir wrote: post #19

> We propose moving towards a three-tier architecture where presentation (client), business logic and data are separated. This has also been called a service-based architecture. The applications (clients) would no longer be able to access the database directly, but only through a well-defined interface that encapsulates the business logic required to perform the function. It is really interesting to see a recent(ish)…

Nowadays you separate service by business capability and not by "layer". Layers just lead to a dependencies and dependencies lead to bad reliability and terrible development speed.

What Amazon were describing here is simply the division between a frontend web gateway service (or, in modernity, client-delivered SPAs); an API backend service to serve the XHRs of the web-gateway / SPA; and some kind of DBMS where user-visible query schema is separable from storage architecture via e.g. views. I don't think there's any modern system that doesn't have those things, no?

Re: Amazon’s distributed computing manifesto (1998)

#27

Post by ajkjk »

wging wrote: post #21

It’s interesting to think about how much of a perspective shift this must have been, especially the service oriented bits. Interestingly, I think it might not have even been made completely in the authors’ minds at the time of this proposal. (Which is understandable, of course. It’s a proposal, not a retrospective on already accepted ideas.) For example, > In the case of DC processing, customer service and other func…

Anecdotally, it was at least 2015 before the DC processing system was actually mostly operating against service-oriented interfaces (when I left in 2016 we had a few old tools left that still talked to the databases directly :/ ).

Re: Amazon’s distributed computing manifesto (1998)

#28

Post by [deleted] »

nevir wrote: post #19

> We propose moving towards a three-tier architecture where presentation (client), business logic and data are separated. This has also been called a service-based architecture. The applications (clients) would no longer be able to access the database directly, but only through a well-defined interface that encapsulates the business logic required to perform the function. It is really interesting to see a recent(ish)…

[deleted]

Re: Amazon’s distributed computing manifesto (1998)

#29

Post by p_l »

manv1 wrote: post #20

This is one example of the CEO making something happen that essentially birthed AWS. Bezos, of all people, was like "make it happen." And it did. It was basically work for no reason except future proofing. Having someone up the food chain OK this much work for the future (and no hard dollar benefit) is highly unusual. And besides that they've done some incredible things with their infrastructure, like authorization.…

It's easy till you break IAM itself with your policy complexity and random services start dying because other AWS components few layers deep cnst get IAM to parse

Re: Amazon’s distributed computing manifesto (1998)

#30

Post by asim »

PaulDavisThe1st wrote: post #22

As one of the main designers of the original system (but who had left by the time this architectural change was done), that is an interesting read. Always good to see the things that we missed in 1994/1995, even though we believed we were thinking far, far ahead.

I'm sure it would have been nice to have that tech in 94 and yet at the same time I get the feeling it had to play out the way it did for Amazon to succeed. Without the first part of the journey Amazon would not have gone on to build AWS.
Post reply on HN