Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Valley of Webhooks

weli.dev

Post reply on HN
Print view View original

91–100 of 108 posts

Re: The Valley of Webhooks

#91

Post by weli »

dzonga wrote: post #67
toomim wrote: post #32

This is a nice writeup of the problems in using Webhooks for State Synchronization. I also noticed that the proposed solution is a pseudo IETF-style draft protocol called SCROLL... that happens to be remarkably similar to an actual IETF draft I am bringing to IETF 127 this November called "Braid-HTTP Subscriptions." Both drafts request a subscription with a GET plus a header: Scroll Request: GET /scroll/feed/customer…

the solution is good but what tends to happen is webhook providers are not gonna work on such a solution - why - because it puts 'work' on them. hell this is without the proposal for a new protocol - just a 'GET' stream or paginated one like the author said. whereas with web hooks - a consumer has to do all the work - as the article above outlined.

Doesn't webhooks put even more work on them? You need to create a webhook dashboard, sometimes with multiple environments and administration endpoints, a way to set HMAC secrets, retriability and deliverability mechanisms, a local tunnel CLI for development, etc. With scroll if you already own a event log system the only work you need to do is exposing it to the client securely, same API keys as the rest of your existing REST api's.

Re: The Valley of Webhooks

#92

Post by marklar423 »

This is a great article and exactly mirrors my experience consuming webhooks. My team built almost the same workarounds as the author.

The solution my team eventually settled on to the dedup, buffering, and race problems was:

  * When a webhook came in, store a single copy of the payload (usually json) in some temporary storage and enqueue the id.
  * Any subsequent updates (while the id was enqueued) overwrote that payload and skipped the queue.
So, we were able to ensure we only updated the object once and with the latest state (since the webhook payload always held the entire object state).

We didn't have a great solution for the bootstrap problem though.

I do like the SCROLL proposal, but I wonder about the cost of keeping around log-structured data forever - every log structured DB I know about does compactions for this reason.

Re: The Valley of Webhooks

#93

Post by jmaw »

nektro wrote: post #83

i think a nice middle ground would be the "flip the arrow" but the webhook tells you when the cursor has moved and that's it so you don't hammer the origin server will polls or have N-time stale data.

I agree that this could be a good middle ground. This causes the webhook to be a push notification with the eventId/cursor, that then causes the consumer to pull the actual data (along with other recent events if desired).

Re: The Valley of Webhooks

#94

Post by Xirdus »

__MatrixMan__ wrote: post #87
Xirdus wrote: post #24

Earlier quoted context omitted.

99% of the time (and all 3 times in the blog post), there is only one source of truth for any piece of data, and state transitions are completely arbitrary. Blockchain is almost always the wrong solution.

That might be true for 99% of data, but I find that it's the remaining 1% of that data occupies the majority of the time. If you don't have a problem that is solved by a blockchain don't use one, but if you're saying things like this: > I do not trust the copy I built, and I have no way to know when it’s wrong, so I will re-derive it from scratch every night, forever. Then your life would probably be better if you ju…

Not 99% of data - 99% of applications. In 99% of applications, 100% of data is such that any piece of this 100% has only one authoritative source. Only in 1% of applications, the percent of data that doesn't have one authoritative source is less than 100%. So the absolute upper bound of when blockchain is even appropriate at all is 1% of applications.

> Then your life would probably be better if you just had to consume block-at-a-time and not the whole dataset every night.

Yes, exactly, that's the whole point of the article, the data should be an ordered stream, not asynchronous events. As long as it's an ordered stream, the author's life is peachy. Blockchain gives you an ordered stream, yes. But so does SCROLL. And if you choose SCROLL, you don't have to deal with the plethora of blockchain-specific problems.

Whenever you have a choice between SCROLL and a blockchain, you should always pick SCROLL and never a blockchain. Only if SCROLL won't work for your use case - for example, you actually need a consensus mechanism - you should consider a blockchain.

And no, blindly copying another database and overwriting every discrepancy with their version is NOT a consensus protocol! It's not meant to build a consensus! It's meant to copy data from authoritative source! There's no consensus to be had!

Re: The Valley of Webhooks

#95

Post by Xirdus »

__MatrixMan__ wrote: post #85
Terr_ wrote: post #34

Earlier quoted context omitted.

> Blockchain is almost always the wrong solution. Especially since in most of the cases where it's not-totally-insane to use, the right solution is still the classic distributed database which already existed. In those, the ledger is kept among a predefined/controlled node-membership... as opposed to a bloated mass of workarounds and limitations to make it barely survive being ungovernable. I've seen some boosters pi…

I'm not sure what a private blockchain is, but a permissioned blockchain is one where only certain parties have keys that allow them to write blocks. You end up with a message queue optimized to eliminate anything that would lead to the inconsistency nonsense that this article is talking about as soon as it is detected. It then becomes the writer's problem to retransmit in a way that doesn't cause a problem next time…

What if the keys leak and need to be rotated?

Re: The Valley of Webhooks

#96

Post by RyoSaeba89 »

weli wrote: post #91
dzonga wrote: post #67

Earlier quoted context omitted.

the solution is good but what tends to happen is webhook providers are not gonna work on such a solution - why - because it puts 'work' on them. hell this is without the proposal for a new protocol - just a 'GET' stream or paginated one like the author said. whereas with web hooks - a consumer has to do all the work - as the article above outlined.

Doesn't webhooks put even more work on them? You need to create a webhook dashboard, sometimes with multiple environments and administration endpoints, a way to set HMAC secrets, retriability and deliverability mechanisms, a local tunnel CLI for development, etc. With scroll if you already own a event log system the only work you need to do is exposing it to the client securely, same API keys as the rest of your exis…

[flagged]

Re: The Valley of Webhooks

#97

Post by weli »

marklar423 wrote: post #92

This is a great article and exactly mirrors my experience consuming webhooks. My team built almost the same workarounds as the author. The solution my team eventually settled on to the dedup, buffering, and race problems was: * When a webhook came in, store a single copy of the payload (usually json) in some temporary storage and enqueue the id. * Any subsequent updates (while the id was enqueued) overwrote that payl…

Compactions are supported and heavily recommended! https://github.com/welidev/scroll/issues/1

Re: The Valley of Webhooks

#98

Post by mrkeen »

I usually preach that a big blob of mutable state in the middle of your system (the db) is the problem, and events are the solution, and this seems like its most obvious case.

All your problems start when you try to modify your current state representation in response to hearing the USER_ADDED or USER_BLOCKED events. Just don't. Leave them as events. Any time you have a new stupid edge case (double send, out-of-order, add-then-delete-then-add), this becomes one new unit test, where you can soberly decide what it means, and update your read path to understand it.

If you bake the nonsense into the current db state every time a new event arrives, your first step in any debug or reasoning scenario is to unbake it: what events led to this mess? If you don't throw away the events, your debugging is done for you.

Re: The Valley of Webhooks

#99

Post by mrkeen »

abrookewood wrote: post #84

Could be completely off base, but isn't this the equivalent of polling a Provider's commit log or distributed log? So accessing a third party's Apache Kafka?

Yep! And since your partner will be unlikely to build a new architecture on the request of one customer, you should 'do the Kafka thing' on your end, capture everything (double-sends and all), then smartly query your events, rather than try to program correct double-send-friendly db updates.
Post reply on HN