Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yes, you can have exactly-once delivery

blog.rongarret.info

Post reply on HN
Print view View original

121–130 of 140 posts

Re: Yes, you can have exactly-once delivery

#121

Post by lisper »

stackghost wrote: post #82
lisper wrote: post #78

Earlier quoted context omitted.

As you have described it, you are right, it's not. But presumably the thing discarding the duplicates then does something with the non-duplicates, like deliver then to something else. That is exactly-once delivery.

You wrote in the post: >This post was intended to be about human communication more than distributed systems or network protocols But resorting to technical minutiae as you have done doesn't seem germane to "human communication". Honestly, seeing such mental gymnastics just to avoid losing face on an internet argument is really disappointing.

What you call "technical minutiae" I call "the meanings of words", which is very germane to the matter of human communication.

Re: Yes, you can have exactly-once delivery

#122

Post by lisper »

mlyle wrote: post #90
lisper wrote: post #78

Earlier quoted context omitted.

As you have described it, you are right, it's not. But presumably the thing discarding the duplicates then does something with the non-duplicates, like deliver then to something else. That is exactly-once delivery.

To discard the duplicates, you need to update your state based on the first message and consult that state for all subsequent messages. Someone must be delivered those messages for this to happen. If you say that someone can then pass you messages exactly once over a perfectly reliable channel, well.. sure. But the system still has to deal with duplicate delivery and update its state based on that. (And potentially m…

> To discard the duplicates, you need to update your state based on the first message and consult that state for all subsequent messages. Someone must be delivered those messages for this to happen.

Yes, that's right.

> If you say that someone can then pass you messages exactly once over a perfectly reliable channel, well.. sure.

So you concede that you were wrong?

> But the system still has to deal with duplicate delivery and update its state based on that.

Well, part of the system does. But that part can be hidden behind an abstraction layer so that "you" can have exactly-once delivery doe some value of "you".

> (And potentially must durably persist this state. And, then, if your application doesn't have exactly the same persistence/transactional boundaries as what's doing the deduping, you have problems...).

Sure, but as I pointed out to you in the other thread in which we are engaged, that is irrelevant to the question of whether or not exactly-once delivery is possible.

Re: Yes, you can have exactly-once delivery

#123

Post by mlyle »

lisper wrote: post #122
mlyle wrote: post #90

Earlier quoted context omitted.

To discard the duplicates, you need to update your state based on the first message and consult that state for all subsequent messages. Someone must be delivered those messages for this to happen. If you say that someone can then pass you messages exactly once over a perfectly reliable channel, well.. sure. But the system still has to deal with duplicate delivery and update its state based on that. (And potentially m…

> To discard the duplicates, you need to update your state based on the first message and consult that state for all subsequent messages. Someone must be delivered those messages for this to happen. Yes, that's right. > If you say that someone can then pass you messages exactly once over a perfectly reliable channel, well.. sure. So you concede that you were wrong? > But the system still has to deal with duplicate de…

No, considering something at the end of a fault-free channel as the element of the distributed system being delivered to is silly.

> we are engaged

We are no longer engaged, because I've given up on talking to you. You can use "delivered" however you like by yourself, as you're eliminating everyone else's desire to listen to you.

I think it's ridiculous that you're back to continue to poke this a day later.

Find a hobby. Or read a textbook. I suggest Steen and Tanenbaum, which explains exactly-once semantics in distributed systems in a careful and thorough way (though this particular book doesn't say "delivery", unlike many others.

Re: Yes, you can have exactly-once delivery

#124

Post by lisper »

mlyle wrote: post #123
lisper wrote: post #122

Earlier quoted context omitted.

> To discard the duplicates, you need to update your state based on the first message and consult that state for all subsequent messages. Someone must be delivered those messages for this to happen. Yes, that's right. > If you say that someone can then pass you messages exactly once over a perfectly reliable channel, well.. sure. So you concede that you were wrong? > But the system still has to deal with duplicate de…

No, considering something at the end of a fault-free channel as the element of the distributed system being delivered to is silly. > we are engaged We are no longer engaged, because I've given up on talking to you. You can use "delivered" however you like by yourself, as you're eliminating everyone else's desire to listen to you. I think it's ridiculous that you're back to continue to poke this a day later. Find a ho…

> considering something at the end of a fault-free channel as the element of the distributed system being delivered to is silly.

That may be, but that's a different discussion. "Silly" and "impossible" are not synonyms.

Once you concede that you are wrong about EOD being impossible we can go on to discuss whether or not it is silly. And note that all I have to do to show that it isn't silly is to present one realistic scenario where it might be useful.

> I've given up on talking to you.

Manifestly not.

Re: Yes, you can have exactly-once delivery

#125

Post by mlyle »

lisper wrote: post #124
mlyle wrote: post #123

Earlier quoted context omitted.

No, considering something at the end of a fault-free channel as the element of the distributed system being delivered to is silly. > we are engaged We are no longer engaged, because I've given up on talking to you. You can use "delivered" however you like by yourself, as you're eliminating everyone else's desire to listen to you. I think it's ridiculous that you're back to continue to poke this a day later. Find a ho…

> considering something at the end of a fault-free channel as the element of the distributed system being delivered to is silly. That may be, but that's a different discussion. "Silly" and "impossible" are not synonyms. Once you concede that you are wrong about EOD being impossible we can go on to discuss whether or not it is silly. And note that all I have to do to show that it isn't silly is to present one realisti…

I am not interested in talking to you anymore. When you reply to my comments it gives me a notification, and I am unable to block you here. Please go away.

If I have to write tooling to make it stop, I will.

Re: Yes, you can have exactly-once delivery

#126

Post by lisper »

mlyle wrote: post #125
lisper wrote: post #124

Earlier quoted context omitted.

> considering something at the end of a fault-free channel as the element of the distributed system being delivered to is silly. That may be, but that's a different discussion. "Silly" and "impossible" are not synonyms. Once you concede that you are wrong about EOD being impossible we can go on to discuss whether or not it is silly. And note that all I have to do to show that it isn't silly is to present one realisti…

I am not interested in talking to you anymore. When you reply to my comments it gives me a notification, and I am unable to block you here. Please go away. If I have to write tooling to make it stop, I will.

> I am not interested in talking to you anymore.

Then why do you keep doing it?

Re: Yes, you can have exactly-once delivery

#127

Post by mlyle »

lisper wrote: post #126
mlyle wrote: post #125

Earlier quoted context omitted.

I am not interested in talking to you anymore. When you reply to my comments it gives me a notification, and I am unable to block you here. Please go away. If I have to write tooling to make it stop, I will.

> I am not interested in talking to you anymore. Then why do you keep doing it?

I've installed the block user extension and written an email filtering rule for you. You may now have the last word, but I will never see it.

You prompted this by continuing the discussion and notifying me in other threads past when this discussion was flagged and I had stopped responding to you... and then even beyond when I had asked you to stop. In seven years of prior discussion on HN, this has never been necessary.

Re: Yes, you can have exactly-once delivery

#128

Post by lisper »

brigadier132 wrote: post #102
lisper wrote: post #86

Earlier quoted context omitted.

I'm not asking for a debate. I'm simply asking you to tell me what "exactly once delivery" actually means in a way that isn't circular and doesn't beg the question of whether or not it is possible, i.e. the definition that is used in literally all of distributed systems research. So far no one has been able to do that.

Exactly once delivery is when a message is delivered to any computer at any layer of abstraction exactly once. It doesn't matter if TCP abstracts or any other layer abstracts this away, if a message can be delivered more than once it does not have the property of exactly once delivery. You might disagree with this definition but you would be wrong. There is negative value in redefining exactly once delivery to mean w…

But I don't disagree with that definition. The only thing in dispute is whether or not it is possible to achieve. It seems clear to me that it is possible, but for some reason that I still don't understand people keep insisting that it isn't.

Re: Yes, you can have exactly-once delivery

#129

Post by lisper »

Filligree wrote: post #101
lisper wrote: post #86

Earlier quoted context omitted.

I'm not asking for a debate. I'm simply asking you to tell me what "exactly once delivery" actually means in a way that isn't circular and doesn't beg the question of whether or not it is possible, i.e. the definition that is used in literally all of distributed systems research. So far no one has been able to do that.

Delivery is what happens when the signals cross from the wire into the network interface card. Depending on the layer you’re operating on, you might instead say it’s the call to recv, or the DMA transfer. The point is that it’s logically a memcpy with no further processing. Just a memcpy. The physical data transfer. It’s easy to build a reliable message-passing system on top of that, but any such system will either i…

> Delivery is what happens when the signals cross from the wire into the network interface card.

If you look at this sibling comment [1] you will find someone who disagrees with your definition.

[1] https://news.ycombinator.com/item?id=41601562

On your definition, yes, exactly-once delivery is not possible. But I think your definition is neither reasonable nor authoritative.

Re: Yes, you can have exactly-once delivery

#130

Post by two_handfuls »

lisper wrote: post #108
qaq wrote: post #98

Earlier quoted context omitted.

You certainly do need to rehash it or provide a link to it being solved without constraints on time window, msgs rates and msgs sizes. You also most certainly want to build it and provide it as service because no existing product claims this capability in unbounded case.

> solved without constraints on time window, msgs rates and msgs sizes But that is not what I am claiming.

The question was for the general case, so yes that is exactly what you are claiming in this thread.

Here is the question again: "Well pretty confused. You can't get exactly once delivery using proposed solution in general case because you can not have infinite memory for dedupe."

Post reply on HN