Yes, you can have exactly-once delivery
blog.rongarret.info
Yes, you can have exactly-once delivery
1–10 of 140 posts
Re: Yes, you can have exactly-once delivery
#2Over 20 years ago I worked with some specialized commercial software, Cyclone, that did guaranteed delivery of files. Guaranteed as in the legal sense. If the server sent back a ticket number to the sending client, it was a LEGAL assurance that the file was received, because there was a contract in place with financial penalties. The time stamps were particularly important in the legal contract. So, there are a lot of ways you can have exactly-once delivery, especially when talking about the application level of the 7 layer burrito model.
Re: Yes, you can have exactly-once delivery
#3[deleted]
Re: Yes, you can have exactly-once delivery
#4I don't see how you can guarantee at least one delivery, but maybe I've seen too many things disappear off busses, never to be seen again.
Re: Yes, you can have exactly-once delivery
#5I don't see how you can guarantee at least one delivery, but maybe I've seen too many things disappear off busses, never to be seen again.
You keep sending it until you get an acknowledgement back. As the article points out, this does assume nobody cut the wire.
Re: Yes, you can have exactly-once delivery
#6I appreciate the insights here, but I am struggling to understand how “exactly one” can equate to “eliminate duplicates”. Let’s say someone arrived at my house and cut my grass, and I failed to confirm they had done so, so the company sent someone over to cut my grass again, maybe multiple times. It seems silly to claim my grass was cut exactly once, despite it consistently remaining at the same height. Obviously it was cut multiple times, just not with much effect after the first. The point of exactly-once is that the server and client don’t need to expend pointless effort on duplicates… right?
Re: Yes, you can have exactly-once delivery
#7Yes of course you can have exactly once delivery if you clearly define who/what the receiver is. You can easily deduplicate messages on the receiving end based on UUIDs created by the sender for example... Since they are duplicates of the exact same message, it doesn't matter which one is 'the original'.
It makes sense to worry about this only if you're worried about wasting bandwidth in the event of network instability (since the same message may sometimes traverse the network multiple times) but that's not generally something engineers should worry about.
It's ironic how some people use this to try to talk down to 'junior' developers.
Anyone can memorise hearsay about distributed systems but few can speak from experience.
Re: Yes, you can have exactly-once delivery
#8I appreciate the insights here, but I am struggling to understand how “exactly one” can equate to “eliminate duplicates”. Let’s say someone arrived at my house and cut my grass, and I failed to confirm they had done so, so the company sent someone over to cut my grass again, maybe multiple times. It seems silly to claim my grass was cut exactly once, despite it consistently remaining at the same height. Obviously it…
In particular what exactly once delivery implies that I do not have to worry about it in my processing logic. I can build a `count += 1` and it will always be exactly correct.
The notion that there is no distinction between exactly once delivery and exactly once processing is very odd to me. In practice my processing needs to accommodate duplicates to be correct. If I had exactly once delivery my processing could be much simpler. If I could get exactly once delivery for free I would always choose it in a heartbeat.
Re: Yes, you can have exactly-once delivery
#9Well pretty confused. You can't get exactly once delivery using proposed solution in general case because you can not have infinite memory for dedupe.
Re: Yes, you can have exactly-once delivery
#10At the end of the day the author and those they are arguing with mostly agree, they simply disagree on what the word "delivery" means. Given the author's background, I wonder if the issue is that they're focused mainly on lower levels of the stack, while those who disagree mainly work with traditional applications that do things like send email, respond to webhooks, update databases, etc.
The reason I think it's important to be pedantic about distinguishing between "delivery" and "processing" is that I have seen plenty of higher level systems that have incorrectly not implemented idempotency and had bugs as a result. I have seen many folks be confused by Kafka's "Exactly-Once Semantics" feature and introduce major bugs into message processing pipelines. The author, who clearly understands these fundamental design challenges, is not my problem. It's everyone else who struggles to design safe, idempotent exactly-once systems.