Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's TCP vs. RPC All over Again

systemsapproach.substack.com

Post reply on HN
Print view View original

111–116 of 116 posts

Re: It's TCP vs. RPC All over Again

#111

Post by cryptonector »

vidarh wrote: post #110
cryptonector wrote: post #108

Earlier quoted context omitted.

There's several ways of doing service discovery for HTTP services like this in an enterprise, using 3xx replies to redirect if you move the service. The simplest is to just use a URI authority that you can change as needed. The next simplest is to publish URI RRs in DNS. The next simplest is to use the .well-known URI local-part namespace to include information for client-side routing. Granted, the last two are recen…

Or we could just use DNS A or CNAME records and not have to do any of those, the way e.g. CoreDNS w/k8s or ldapdns on top of openldap, and similar does. There are use cases for more complicated service discovery, especially discovery of external services, but for the scenario I described, which in involved discovery within our own network this all just would have added extra complexity for no benefit that would have…

I'm not saying that you shouldn't have used SMTP. It really is appropriate for this sort of thing. If you wanted an HTTP interface you might still make it inject messages into SMTP, with HTTP being only used for a browser UI, say, or with the HTTP server as a proxy for some reason.

My objection had been to LDAP. LDAP is too heavy-weight by comparison to HTTP w/ a JSON API.

Re: It's TCP vs. RPC All over Again

#112

Post by [deleted] »

cryptonector wrote: post #109
remram wrote: post #107

Earlier quoted context omitted.

RPC usually means stateful client so REST goes out the window.

Browser clients of RESTful APIs can be stateless in a very stateful way because the DOM and JS allows the client to keep state. The statelessness of HTTP is supposed to be about the HTTP layer itself, not the application, and this is true as much about the client as about the server. Consider a database application where the client is a browser and the server is just something like PostgREST, and then you see that th…

[deleted]

Re: It's TCP vs. RPC All over Again

#113

Post by derefr »

vidarh wrote: post #105
derefr wrote: post #98

Earlier quoted context omitted.

> and we had a DNS server where the zones could be updated via SMTP Interesting — what was the message format (MIME type) you chose to use to describe the zone changes? Actually, were they even patch-files at all, or did you just fetch the zone data file from your DNS server, update it client-side, and then send an updated complete copy back in the body of the email message, to get de-enveloped directly into /var/nam…

The DNS for service discovery was fairly static, and not what we updated via SMTP. We treated the DNS server in question simply as a key-value store where a message would set/replace a specific record. The use case was that we we're setting a handful of records when people registered a domain, so those DNS servers were user visible, unlike the service discovery. The main reason for not considering LDAP at the time fo…

To be clear, I'm not saying you should have used LDAP as a changeset submission protocol instead of SMTP; I'm saying you should have used LDIF as a changeset format for sending via SMTP, instead of inventing your own changeset description format to send over SMTP. :)

> We treated the DNS server in question simply as a key-value store where a message would set/replace a specific record.

Interesting! That doesn't sound like any DNS server I'm aware of. In BIND et al, a zone was a file, and to update a zone you needed to modify a zone file and SIGHUP the daemon; which in turn meant that whatever you were using to mechanistically modify zones, needed to be able to parse and regenerate an entire zone file at a time. (It's actually a lot like what's required to mechanistically update-patch Kubernetes YAML resource manifests, now that I think about it.)

Or do you mean that you had exactly one record per zone in this key-value store? If so, how did that work exactly? Zones need at least an SOA record, no? Or, I guess, maybe not, if this nameserver was only exposed within the Intranet, without the requirement of being an authoritative source that other DNS servers could fetch and cache from.

Re: It's TCP vs. RPC All over Again

#114

Post by vidarh »

derefr wrote: post #113
vidarh wrote: post #105

Earlier quoted context omitted.

The DNS for service discovery was fairly static, and not what we updated via SMTP. We treated the DNS server in question simply as a key-value store where a message would set/replace a specific record. The use case was that we we're setting a handful of records when people registered a domain, so those DNS servers were user visible, unlike the service discovery. The main reason for not considering LDAP at the time fo…

To be clear, I'm not saying you should have used LDAP as a changeset submission protocol instead of SMTP; I'm saying you should have used LDIF as a changeset format for sending via SMTP, instead of inventing your own changeset description format to send over SMTP. :) > We treated the DNS server in question simply as a key-value store where a message would set/replace a specific record. Interesting! That doesn't sound…

The DNS server was our own.

Writing an authoritative-only DNS server is trivial (few hundred lines), and since it knew it was always authoritative, it'd synthesise a SOA record for every zone it answered for.

The zones had a very minimal set of records: Either they had NS records pointing to where the user wanted the domain delegated, or to us. If they pointed to us they had an A and MX record which might point to us or to wherever the customer wanted.

It was not exposed only within the intranet (the service discovery DNS was exposed only via the internet; separate thing)

Re: It's TCP vs. RPC All over Again

#115

Post by hardwaresofton »

KaiserPro wrote: post #86
hardwaresofton wrote: post #29

If you'd like to watch Ousterhoust explain Homa and his vision of replacing TCP: https://www.youtube.com/watch?v=o2HBHckrdQc Spoiler: Ousterhout is right, and the data shows he is right. Maybe not everyone knows it yet, or it's the kind of "right" that not everyone will ever get to (and we'll settle for "good enough" for much longer), but he is almost certainly right. Not to disparage the poster of this article of co…

> Spoiler: Ousterhout is right, and the data shows he is right. Maybe not everyone knows it yet, or it's the kind of "right" that not everyone will ever get to (and we'll settle for "good enough" for much longer), but he is almost certainly right. No, Ousterhout is not "right", Ousterhout has put forward an hypothesis, backed up by _some_ data that's not really borne out from real world experiences in the datacentre.…

A bit late but for the average (small, let’s say) data center that isn’t interested, then he’s offering a valid (if not unlikely) solution right?

Re: It's TCP vs. RPC All over Again

#116

Post by KaiserPro »

hardwaresofton wrote: post #115
KaiserPro wrote: post #86

Earlier quoted context omitted.

> Spoiler: Ousterhout is right, and the data shows he is right. Maybe not everyone knows it yet, or it's the kind of "right" that not everyone will ever get to (and we'll settle for "good enough" for much longer), but he is almost certainly right. No, Ousterhout is not "right", Ousterhout has put forward an hypothesis, backed up by _some_ data that's not really borne out from real world experiences in the datacentre.…

A bit late but for the average (small, let’s say) data center that isn’t interested, then he’s offering a valid (if not unlikely) solution right?

I'm not sure. Having run a smaller datacenter (~2 megawatts) We got far more bang for our buck by upgrading the core switch fabric and moving to dual 40gig(from dual 10 gig) ethernet for the "heavy" nodes.

Sure it cost more in terms of capex, but once it was installed we didn't need to change anything else. The cost of having an engineering team porting mission critical stuff to a new transport mechanism would have been astronomical, not only in opex, but in lost capacity to do other business critical tasks.

Post reply on HN