The proposed feed solution looks a lot like the CouchDB replication protocol, as another object in a convergent evolution space to consider.
The Valley of Webhooks
31–40 of 108 posts
Re: The Valley of Webhooks
#32This is a nice writeup of the problems in using Webhooks for State Synchronization. I also noticed that the proposed solution is a pseudo IETF-style draft protocol called SCROLL... that happens to be remarkably similar to an actual IETF draft I am bringing to IETF 127 this November called "Braid-HTTP Subscriptions."
Both drafts request a subscription with a GET plus a header:
Scroll Request:
GET /scroll/feed/customers
Prefer: stream
Braid Request:
GET /customers
Subscribe:
The author noted that it's hard to get adoption. Well, the reason that Webhooks are so common is that they are bog-standard HTTP. For this to get adopted, we need to put it into bog-standard HTTP. So we need to go to the IETF, and and extend HTTP in a general way to support state synchronization. It should just work for any existing HTTP media type (not just JSON), and any resource/URL (not just special /scroll/* URLs), and any way of marking timestamps (not just the ordered strings proposed in SCROLL).
Then we can bake this stuff into HTTP, and thus into all our bog-standard libraries, utilities, and code, and you won't have to reimplement the same sync-logic-over-webhooks again, and again, and again.
Reach out if you're interested!
Re: The Valley of Webhooks
#33This is a nice writeup of the problems in using Webhooks for State Synchronization. I also noticed that the proposed solution is a pseudo IETF-style draft protocol called SCROLL... that happens to be remarkably similar to an actual IETF draft I am bringing to IETF 127 this November called "Braid-HTTP Subscriptions." Both drafts request a subscription with a GET plus a header: Scroll Request: GET /scroll/feed/customer…
I reached out through email :)
Just one correction. My spec doesn't force /scroll/ URL's, just proposes it as a convention.
Re: The Valley of Webhooks
#34If you have two or more services that need to agree about state, and you have some set of rules that govern what state changes are valid, and you don't want to mess around with any of this "what do we do when we miss and update vs when we get two of the same update" nonsense, and the services aren't in a position to query the same database, then you should really consider a permissioned blockchain. Consensus hard, bu…
99% of the time (and all 3 times in the blog post), there is only one source of truth for any piece of data, and state transitions are completely arbitrary. Blockchain is almost always the wrong solution.
> Blockchain is almost always the wrong solution.
Especially since in most of the cases where it's not-totally-insane to use, the right solution is still the classic distributed database which already existed. In those, the ledger is kept among a predefined/controlled node-membership... as opposed to a bloated mass of workarounds and limitations to make it barely survive being ungovernable.
I've seen some boosters pivot to saying "private blockchain is good", but that's contradictory buzzword nonsense. It's like selling a blog as "single-user Twitter" or advertising a regular car as "user-controlled autonomous vehicle."
Re: The Valley of Webhooks
#35This topic always surprises me. I do not understand the sequence of logic that leads people to build synchronisation mechanisms based only on webhooks. Webhooks aren't at-least-once, nor at-most-once, nor are they guaranteed in-order. Some people build systems to make them more reliable, but if you really care about the data you need to think of a webhook delivery as best-effort, a bit like UDP. That's before you get…
Maybe the LLM has written maximum 50 words of the article by just being directed to switch things around and improve grammar and or internal consistency
Detecting LLM-written text is a critical skill these days. If you believe only 50 words of this was slop, then that's a skill you could improve.
Re: The Valley of Webhooks
#36This is a nice writeup of the problems in using Webhooks for State Synchronization. I also noticed that the proposed solution is a pseudo IETF-style draft protocol called SCROLL... that happens to be remarkably similar to an actual IETF draft I am bringing to IETF 127 this November called "Braid-HTTP Subscriptions." Both drafts request a subscription with a GET plus a header: Scroll Request: GET /scroll/feed/customer…
Is it accurate to say this is something like long polling except you continue to hold the connection open for subsequent updates? Does this mean a server potentially needs to hold open a very large number of connections (one per client) even if there are no updates?
And why formalize on HTTP rather than on a similar protocol over websockets?
Re: The Valley of Webhooks
#37Earlier quoted context omitted.
"And here’s the absurd part: that history exists." "It’s a jigsaw puzzle where the manufacturer had the original picture, cut it up, mailed me the pieces one at a time, lost a few in the post, mailed some twice, and printed nothing on the box." "and that’s the entire problem: nothing announces a gap." "None of this is any provider’s bug. Their webhooks work exactly as documented. The problem is what a webhook is: a n…
I think my is-this-LLM alert is triggered not by mere use of phrases / constructions beloved of popular LLMs, but by things like unmotivated magpie-ish use of those phrases. I didn't get that reading this (I didn't read the whole piece but I had read the parts you quote before reading your comment). Often the LLM-beloved constructions are good usage in the right contexts.
Re-reading, I think the article is in a grey area, and I believe that there is a decent amount of human experience underpinning it. It certainly isn't 100% LLM, but it's a long way from 100% human.
But I wrote my commment after reading the article then the spec (https://welidev.github.io/scroll/), and so the spec was "top of mind".
The spec is just awash in LLM-isms. The cadence and rhetorical style are very Claudish. The visual style is basically "Claude's artifact plugin" (it may not be exactly that but it is an incredibly distinct signature). So the experience of reading the spec is very much an "AI slop" experience.
The reason I object to this is that the way these LLMs write is really well-tuned to gloss over small but critical details. And "small but critical details" are sort of the whole field of distributed systems.
This seems to be most true for Anthropic models (I am assuming there is some cultural defect in the way they give feedback), but it seems to be pretty universal, unless you give them some really strong stylistic anchor to a different style.
(As an aside, I sometimes wonder if this is part of the reason that LLMs seem from the outside to be succeeding disproportionately at mathematics: mathematics papers and mathematical notation may be a strong enough cultural force to override Anthropic's lack of taste and unlock the true power of the model).
I'm not saying there might not have been plenty of human guidance, but either way I don't think there's quite enough substance to this (based on everything I wrote in my comment) for this to feel like "a solution" either way.
Re: The Valley of Webhooks
#38Gerard mentions it super quickly, but another massive issue with webhooks generally is local development. Yes, you can use a tunnel, but that requires all engineers on a team to add their own tunnel urls. This causes even more issues when you use the platform as a source of truth, like for auth or payments. With WorkOS specifically, your whole team develops with one shared development sandbox. You run into issues when your local dev auth (in postgres) is not synced with the shared dev sandbox that WorkOS has since not all team members have their dev environments running at once. So yeah, then you use events API. But WorkOS only preserves the events API data for 90 days (and u have make 3 calls since its a max of 30 days per call). So then you load all the data with the state API first, then you start running the events API. It's a mess.
Tried to talk about this on X until the CEO of WorkOS wanted to bring it in private, then proceeded not to help at all. https://x.com/grinich/status/1913035839866835297?s=20
Re: The Valley of Webhooks
#39The core tradeoff here is "Push vs. Pull". Webhooks are a model for pushing data to subscribers. A traditional API allows clients to request and pull data. The data flow is in the same direction, but control flow is opposite.
Pull-oriented models are much easier to reason about and should be preferred where possible (cybernetically they are a closed loop, vs. push models which could literally just be a barrage of UDP packets). But they do have a little bit of overhead which makes them the wrong tool for some cases, like live-streamed entertainment or massive telemetry flows which value performance (latency, throughput) over missing a few packets.
Re: The Valley of Webhooks
#40The core tradeoff here is "Push vs. Pull". Webhooks are a model for pushing data to subscribers. A traditional API allows clients to request and pull data. The data flow is in the same direction, but control flow is opposite. Pull-oriented models are much easier to reason about and should be preferred where possible (cybernetically they are a closed loop, vs. push models which could literally just be a barrage of UDP…
You mean pull-oriented models are easier to reason about, right?