Live data from Hacker News

Distributed Systems and the End of the API

writings.quilt.org

51–60 of 102 posts

Re: Distributed Systems and the End of the API

#51

Earlier quoted context omitted.

"Layman's REST" is very much RPC, yes. Fielding's REST is very much not.

Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns every…

> Services are naturally centered arounds verbs (commands and queries) and not nouns (resources)

Can you qualify that? Queries are presumably queries on resources, and which commands did you have in mind apart from creating or updating resources?

Re: Distributed Systems and the End of the API

#52

Earlier quoted context omitted.

Interesting, thank you; but in that quote isn't Fielding actually comparing RPC and HTTP , not REST?

Yes, you're right, I was being a bit sloppy there. The Uniform Interface (and Layered System) is one of the ways in which HTTP does follow RESTful principles, though, so the thrust is still the same.

Agreed. But I think it's a stretch to conclude that Fielding doesn't think the hypermedia principle is an important distinction between RPC and REST:

From his blog post I linked to earlier, titled REST APIs must be hypertext-driven [0]:

> "I am getting frustrated by the number of people calling any HTTP-based interface a REST API. Today’s example is the SocialSite REST API. That is RPC. It screams RPC. There is so much coupling on display that it should be given an X rating.

"What needs to be done to make the REST architectural style clear on the notion that hypertext is a constraint?"

[0] http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte...

Re: Distributed Systems and the End of the API

#53
Hmm the article seems based on some false assumptions. I'd argue that the whole point of REST as an architectural style is to be stateless and async. Of course you would use an ESB of some kind rather than point-to-point if you want to protect yourself from failure of a solution component - REST lends itself well to that or to building error-handling in the client. And isn't 'turning operations into data' what we are doing by switching from a verb-based model to a noun-based one?

Re: Distributed Systems and the End of the API

#54

Earlier quoted context omitted.

FYI information theory entropy and physics entropy really aren't the same thing: http://physics.ucsd.edu/do-the-math/2013/05/elusive-entropy/

Well that seemed excessively pedantic IMHO. It actually didn't touch much on information theory, and where it did, many of the comments disagree. I'll cite the Landauer limit[1] as what (yes, arguably) connects the entropy of information theory to the entropy of physics.[2] Also, I only mentioned physics because the article did, quoting Lamport " Most people view concurrency as a programming problem or a language pro…

Here's an article which discusses Lamport's view: "The physics of distributed information systems"[1]. The first sentence: "This paper aims to present distributed systems as a new (interesting) area of applications of statistical physics, and to make the the case that statistical physics can be quite useful in understanding such systems."

It has several mentions of statistical physics, but (curiously) no mentions of entropy. It does however discuss the Byzantine Generals problem, which of course is the problem the bitcoin blockchain solves.

1. http://iopscience.iop.org/1742-6596/473/1/012017/pdf/1742-65...

Re: Distributed Systems and the End of the API

#55
post #30

Rest is less platform dependent than SOAP/RPC. I see that as the main benefit. JSON is easier to work with than XML. The whole idea of service oriented architectures is that users don't need to care about the tech stack details of your service. Rest and JSON do a better job of realizing that vision than SOAP/XML. I don't think anyone's claimed that Rest is a design pattern to end all woes. Maybe we haven't given due…

Actually JSON makes it damn near impossible to uniformally implement REST. You need HATEOS. This means that there has to be a semantic of following links in resources. JSON lacks this ability. ATOM or RSS, both XML, have linking. Heck, XML at a language level supports document linking. HTTP + JSON != REST.

Re: Distributed Systems and the End of the API

#56

Earlier quoted context omitted.

Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns every…

> Services are naturally centered arounds verbs (commands and queries) and not nouns (resources) Can you qualify that? Queries are presumably queries on resources, and which commands did you have in mind apart from creating or updating resources?

Not OP, but a thought anyway: these worldviews are sort of duals of each other, IMHO. You can consider a "noun" to be one instance of a verb's application, i.e., an action. For example, "send an email" or "purchase an item", both verbs, translate to creating new nouns representing (respectively) an email in my Outbox or a transaction. The advantage of the noun worldview is that the nouns can describe the history of verbs and the resulting application state, e.g., I can look at and manage the whole ledger of transactions, whereas verbs are just ad-hoc manipulations of that state.

Re: Distributed Systems and the End of the API

#57
post #30

Rest is less platform dependent than SOAP/RPC. I see that as the main benefit. JSON is easier to work with than XML. The whole idea of service oriented architectures is that users don't need to care about the tech stack details of your service. Rest and JSON do a better job of realizing that vision than SOAP/XML. I don't think anyone's claimed that Rest is a design pattern to end all woes. Maybe we haven't given due…

Actually JSON makes it damn near impossible to uniformally implement REST. You need HATEOS. This means that there has to be a semantic of following links in resources. JSON lacks this ability. ATOM or RSS, both XML, have linking. Heck, XML at a language level supports document linking. HTTP + JSON != REST.

Interesting. I looked into HATEOAS more and found this link to a PayPal API:

https://developer.paypal.com/docs/integration/direct/paypal-...

I think I'll try to follow Rest better. Anyhow, I still prefer JSON. To me it's a more concise way to explain data, with nice syntax for arrays and maps. I'd rather jump through a few hoops to implement true Rest than try to craft verbose XML schemas for simple things like arrays and maps. ATOM and RSS I think are good for their main use cases but aren't as generic of syntax as JSON.

Re: Distributed Systems and the End of the API

#58
Fun stuff, amusing that the definition of a distributed system used; "Where a computer that you never heard of can bring your system down." is actually one of Leslie Lamport's more famous quotes.

When I joined Sun in '86 I thought it was the pinnacle of technological excellence to be a kernel programmer, and I joined the Systems Group, the notional center of the Sun universe, in 1987. However I discovered that the primary reason you had to be picky about kernel programmers what that their bogus pointer references crashed the machine (as they occurred in kernel mode with full privileges) but discovered that network programmers could crash the whole world with their bugs. So clearly they must be in a pantheon above kernel programmers. :-)

The author has come to discover that in the network world things can die anywhere, and this makes reasoning about such systems very complicated. Having been a part of the RPC and CORBA evolution I keenly felt the challenges of making APIs that "looked" like function calls to a programmer but took place across a network fabric and thus introduced error conditions that couldn't exist in locally called routines. (like the inability to return from the function due to a network partition for a simple example).

Lamport's work in this space is brilliant and inspired. Network systems can be analysed and reasoned about as physical systems when they exhibit discontinuities when considered as simple algorithms. The value here is to realize that a large number of physical systems tolerate a tremendous amount of randomness and continue to work as intended (windmills for example) while many algorithms only work consistently given a set of key invariants.

I gave a talk that was inspired by Dr. Lamports work titled 'Java as Newtonian Physics' which was a call to action to create a set of invariants, in the spirit of physical laws, that would govern the behavior and capabilities of distributed systems. It was way early for its time (AOL dialup connections were still a thing) but much of the same inspiration (presumably from Lamport) made it into the Google Spanner project.

As with many things, at a surface level many people learn an API which does something under the covers across the network but having come up through their education thinking of everything as an API they don't fundamentally grasp the notion of distributed computation. Then at some point in their experience there will be that 'ah ha' moment when suddenly everything they know is wrong, which really means they suddenly see a bigger picture of things. It makes distributed systems questions in interviews an excellent litmus test for understanding where people are in their journey.

Re: Distributed Systems and the End of the API

#59

Distributed API's are a big part of Ethereum. I think the Merkle tree of the bitcoin blockchain (and the Patricia tree of the Ethereum blockchain) might even qualify as a semilattice. In fact, its by the physics of information theory that a cryptographic blockchain solves the consensus problem. Specifically, information theory emerges from the laws of thermodynamics: Maxwell's demon is essentially what secures one's…

FYI information theory entropy and physics entropy really aren't the same thing: http://physics.ucsd.edu/do-the-math/2013/05/elusive-entropy/

Okay, last one. Most concise counter-argument: http://en.wikipedia.org/wiki/Brute-force_attack#Theoretical_...

Derived from the Landauer limit.

Re: Distributed Systems and the End of the API

#60
As the author of EtherPad I'm familiar with CRDT, which is a cousin of OT. They don't really replace APIs, unless you are using an API to synchronize data, which is only one of many things you might be trying to do.

In other words, if you're building EtherPad or Wave, use a fancy data structure for the collaborative document. Otherwise, don't. Meteor's DDP provides a nice model, where the results of RPCs stream in asynchronously.

Post reply on HN