Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed Systems and the End of the API

writings.quilt.org

Post reply on HN
Print view View original

41–50 of 102 posts

Re: Distributed Systems and the End of the API

#41

Post by mantrax6 »

steveklabnik wrote: post #29
mantrax5 wrote: post #26

Earlier quoted context omitted.

Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns every…

I agree that many services are simply CRUD wrappers. That doesn't have much to do with the nature of the architecture Fielding proposes. I would be interested in some citations from Fielding which demonstrate that RPC is its organizational principle. I don't think they're there, though.

Fieldman's work doesn't exist in vacuum. He talks about HTTP, and HTTP has the verbs it has. It's hard enough to find consistent behavior in HTTP servers and proxies with "PUT" and "DELETE" let alone anything else. But even ignoring that, the verbs the spec talks about are limited. And that's a big problem.

As for RPC, essentially any communication between machines is a RPC. It's message passing ("call") and if the message arrives on the other end it's processed by a message handler ("procedure").

No server can just reach into another server's RAM and get or modify a resource directly. The interaction happens entirely by the will of the message receiver, and in exactly the way the receiver wants (not the sender).

So if we'll be building everything on an appropriate abstraction, it better match what really happens (messages, message handlers) and not some wishful thinking abstraction layered on top (resources, resource modification).

RPC is not REST's problem. The problem is the limited commands (PUT, POST, DELETE) and the single possible query type (GET) that we need to work with.

When a payment gateway has to represent a simple "process payment" command with a series of bogus abstractions like a "POST /payment/transaction/new", you know REST is the wrong tool for the job.

Re: Distributed Systems and the End of the API

#42

Post by scotth »

scotth wrote: post #36
mantrax5 wrote: post #26

Earlier quoted context omitted.

Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns every…

Aside: Why do you have so many accounts mantrax? At least one of them is dead, and I'm sure you can imagine why.

HN hellbans, so you might not realize you're not being heard.

I'm seeing [dead] markers from matrax6 and mantrax7 (although oddly enough, not consistently).

Can't comment on the quality of the code. Your problems sound atypical.

Re: Distributed Systems and the End of the API

#43

Post by bodhi »

mantrax5 wrote: post #26
steveklabnik wrote: post #24

Earlier quoted context omitted.

"Layman's REST" is very much RPC, yes. Fielding's REST is very much not.

Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns every…

So, honest question. I've been musing over REST and RPC for a while, and was trying to come up with some domains that were verb-oriented instead of noun-oriented. The only thing I could come up with was message passing, a-la XMPP or streaming content.

What are some other problem domains that are better represented in verb-oriented terminology?

Re: Distributed Systems and the End of the API

#44

Post by steveklabnik »

mantrax6 wrote: post #41
steveklabnik wrote: post #29

Earlier quoted context omitted.

I agree that many services are simply CRUD wrappers. That doesn't have much to do with the nature of the architecture Fielding proposes. I would be interested in some citations from Fielding which demonstrate that RPC is its organizational principle. I don't think they're there, though.

Fieldman's work doesn't exist in vacuum. He talks about HTTP, and HTTP has the verbs it has. It's hard enough to find consistent behavior in HTTP servers and proxies with "PUT" and "DELETE" let alone anything else. But even ignoring that, the verbs the spec talks about are limited. And that's a big problem. As for RPC, essentially any communication between machines is a RPC. It's message passing ("call") and if the m…

> He talks about HTTP, and HTTP has the verbs it has.

Sure, but REST is not HTTP. Yes, it was created to describe the architecture of HTTP, but you can do REST over other protocols, and even HTTP is not truly RESTful. Indeed, Fielding's thesis devotes an entire section to this: http://www.ics.uci.edu/~fielding/pubs/dissertation/evaluatio...

> and proxies with "PUT" and "DELETE"

This is due to the HTML spec, not anything else. In fact, this was almost changed in HTML5, but the problems with adding arbitrary verbs to HTML.FORM weren't adequately addressed. You could make that case, if you felt you'd solved the problems.

> As for RPC, essentially any communication between machines is a RPC

This characterization of RPC means that _everything_ is RPC, which means it's not a very useful way to compare architectures.

> The problem is the limited commands (PUT, POST, DELETE) and the single possible query type (GET) that we need to work with.

HTTP != REST. That said, something very similar to this is called the 'uniform interface constraint,' and is definitional to REST.

By the way, HTTP can have extension verbs too, so you're not actually limited to that.

Anyway, you still haven't shown me how Fielding's REST is RPC, or at least, in the way that software architects talk about RPC. http://www.ics.uci.edu/~fielding/pubs/dissertation/evaluatio... may be instructive.

Re: Distributed Systems and the End of the API

#45

Post by ademarre »

mantrax6 wrote: post #39
scotth wrote: post #36

Earlier quoted context omitted.

Aside: Why do you have so many accounts mantrax? At least one of them is dead, and I'm sure you can imagine why.

None of them are dead. As for why, long story short, because HackerNews' code is really poorly written. Gateway errors, horrendous response times, bogus limits and false positives. My usage patterns are likely not typical, but this is hardly an excuse for such a poor UX. For example, I couldn't even respond right now through mantrax5... Next time I'll just take the alternative and instead of making more accounts, I'l…

Have you reported these HN bugs?

Re: Distributed Systems and the End of the API

#46

Post by steveklabnik »

ademarre wrote: post #38
mantrax5 wrote: post #26

Earlier quoted context omitted.

Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns every…

The problem with REST is that many (I dare say most ) who think they are applying it really aren't. I think this is quite relevant: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte... What sets REST apart from RPC is the Hypermedia-as-the-Engine-of-Application-State principle (HATEOAS). With HATEOAS, interaction semantics are removed from URIs and defined in terms of link relations. This decouples clients…

While I agree with you that that's certainly an interesting part of the split between RPC and REST, you might be curious to know that Fielding doesn't think so.

> What makes HTTP significantly different from RPC is that the requests are directed to resources using a generic interface with standard semantics that can be interpreted by intermediaries almost as well as by the machines that originate services.

http://www.ics.uci.edu/~fielding/pubs/dissertation/evaluatio...

Re: Distributed Systems and the End of the API

#47

Post by ademarre »

steveklabnik wrote: post #46
ademarre wrote: post #38

Earlier quoted context omitted.

The problem with REST is that many (I dare say most ) who think they are applying it really aren't. I think this is quite relevant: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte... What sets REST apart from RPC is the Hypermedia-as-the-Engine-of-Application-State principle (HATEOAS). With HATEOAS, interaction semantics are removed from URIs and defined in terms of link relations. This decouples clients…

While I agree with you that that's certainly an interesting part of the split between RPC and REST, you might be curious to know that Fielding doesn't think so. > What makes HTTP significantly different from RPC is that the requests are directed to resources using a generic interface with standard semantics that can be interpreted by intermediaries almost as well as by the machines that originate services. http://www…

Interesting, thank you; but in that quote isn't Fielding actually comparing RPC and HTTP, not REST?

Re: Distributed Systems and the End of the API

#48

Post by steveklabnik »

ademarre wrote: post #47
steveklabnik wrote: post #46

Earlier quoted context omitted.

While I agree with you that that's certainly an interesting part of the split between RPC and REST, you might be curious to know that Fielding doesn't think so. > What makes HTTP significantly different from RPC is that the requests are directed to resources using a generic interface with standard semantics that can be interpreted by intermediaries almost as well as by the machines that originate services. http://www…

Interesting, thank you; but in that quote isn't Fielding actually comparing RPC and HTTP , not REST?

Yes, you're right, I was being a bit sloppy there. The Uniform Interface (and Layered System) is one of the ways in which HTTP does follow RESTful principles, though, so the thrust is still the same.

Re: Distributed Systems and the End of the API

#49

Post by mantrax5 »

steveklabnik wrote: post #44
mantrax6 wrote: post #41

Earlier quoted context omitted.

Fieldman's work doesn't exist in vacuum. He talks about HTTP, and HTTP has the verbs it has. It's hard enough to find consistent behavior in HTTP servers and proxies with "PUT" and "DELETE" let alone anything else. But even ignoring that, the verbs the spec talks about are limited. And that's a big problem. As for RPC, essentially any communication between machines is a RPC. It's message passing ("call") and if the m…

> He talks about HTTP, and HTTP has the verbs it has. Sure, but REST is not HTTP. Yes, it was created to describe the architecture of HTTP, but you can do REST over other protocols, and even HTTP is not truly RESTful. Indeed, Fielding's thesis devotes an entire section to this: http://www.ics.uci.edu/~fielding/pubs/dissertation/evaluatio... > and proxies with "PUT" and "DELETE" This is due to the HTML spec, not anyth…

Regarding RPC, you're right, I guess I'm not quite following the classical understanding of it. I tend to work with actor systems, where everything (in-process or intra-process) is message passing, so I tend to see the message layer first, and anything else merely a protocol implemented on top of it. Hence my response.

As for HTTP, if HTTP itself isn't truly RESTful, the only protocol I've ever seen REST implemented on, I'll have to resign from this debate, because it becomes completely abstract.

I hope the right protocol comes along to show us what REST might really be.

I'm spotting a bit of a pattern in REST supporters, where if some understanding of REST is found to have faults, it's called "not truly RESTful". Reminds me of how Agile failures tended to be explained as "not truly Agile"... but that's another story.

I judge a system by the implementations, not by the theory. If REST is this beautiful unattainable ideal that almost no one can truly implement, I tend to think the fault doesn't lie with the implementers.

It's quite likely REST simply has very few applications itself, but the applications have the potential to become all-encompassing (like the web itself). But in this case the issue remains REST is not suitable for 99% of the services people expose on-line, as each of those services has a specific scope and application, where REST's "network effects" of uniform resource interface and so on don't apply.

Re: Distributed Systems and the End of the API

#50

Post by kylebrown »

marktangotango wrote: post #40
kylebrown wrote: post #21

Distributed API's are a big part of Ethereum. I think the Merkle tree of the bitcoin blockchain (and the Patricia tree of the Ethereum blockchain) might even qualify as a semilattice. In fact, its by the physics of information theory that a cryptographic blockchain solves the consensus problem. Specifically, information theory emerges from the laws of thermodynamics: Maxwell's demon is essentially what secures one's…

FYI information theory entropy and physics entropy really aren't the same thing: http://physics.ucsd.edu/do-the-math/2013/05/elusive-entropy/

Well that seemed excessively pedantic IMHO. It actually didn't touch much on information theory, and where it did, many of the comments disagree. I'll cite the Landauer limit[1] as what (yes, arguably) connects the entropy of information theory to the entropy of physics.[2]

Also, I only mentioned physics because the article did, quoting Lamport "Most people view concurrency as a programming problem or a language problem. I regard it as a physics problem."

Unfortunately the article didn't elaborate any more on the precise type of physics problem in question (Maybe Lamport does elsewhere), whether the physics of computational complexity or the physics of information theory, or something else. But even those two sub-fields have many connections and similarities (as does pretty much everything in physics and math. such connections are the bread-and-butter of theoreticians).

1. http://en.wikipedia.org/wiki/Von_Neumann-Landauer_limit

2. http://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_i...

Post reply on HN