Live data from Hacker News

Google pub/sub released an ordering feature

cloud.google.com

41–50 of 114 posts

Re: Google pub/sub released an ordering feature

#41
post #18

Earlier quoted context omitted.

What's a use case where strict ordering is critically important?

Consider processing two events without guaranteed ordering: - Create A - Delete A In one ordering, A is created and then deleted as expected, in the other, the delete fails but then A is created and remains.

But when does the delete enter the queue and when does the Create enter the queue ;)

Re: Google pub/sub released an ordering feature

#42
post #21
post #18

Earlier quoted context omitted.

Consider processing two events without guaranteed ordering: - Create A - Delete A In one ordering, A is created and then deleted as expected, in the other, the delete fails but then A is created and remains.

Does adding timestamps not handle this case?

With a single producer and consumer, yes - but of course that's seldom the case.

With multiple producers and consumers, clock skew would be an issue, with the time on different machines being off from each other slightly.

One option is to use a single source for generating IDs, but that introduces another failure point, and comes at a hefty performance cost.

Re: Google pub/sub released an ordering feature

#44
post #18

Earlier quoted context omitted.

What's a use case where strict ordering is critically important?

Consider processing two events without guaranteed ordering: - Create A - Delete A In one ordering, A is created and then deleted as expected, in the other, the delete fails but then A is created and remains.

That doesn’t necessarily require that the entire queue is totally ordered, but the alternatives (such as Virtual Synchrony) are still considered arcane / research topics.

Re: Google pub/sub released an ordering feature

#45
This kind of confuses me a bit. Why would you want to turn Pub/Sub into a queue?

How badly does this affect message acknowledgment and retries? I assume just a huge hit to latency. This seems like a horrible idea for anyone expecting to use multiple subscribers or expecting to chunk multiple messages per request.

Services relying on Pub/Sub should be idempotent anyway. If you need to work around that for some reason, you are better off dumping messages from your subscriber into RabbotMQ or Redis for processing and use a Subscriber/Scheduler/Worker pattern.

Re: Google pub/sub released an ordering feature

#46
post #21

Earlier quoted context omitted.

Does adding timestamps not handle this case?

That won’t work in all cases. For instance, if you get messages from devices which can be reimaged they may have clock skew in a period of time before they’re synchronized again.

But in any case you can't rely on the order of message ingress to your system to represent anything meaningful either? It would have to ensure that the key for defining order would have some hard logical ordering purpose for which time is not relevant or useful.

Re: Google pub/sub released an ordering feature

#48
post #21
post #18

Earlier quoted context omitted.

Consider processing two events without guaranteed ordering: - Create A - Delete A In one ordering, A is created and then deleted as expected, in the other, the delete fails but then A is created and remains.

Does adding timestamps not handle this case?

> Does adding timestamps not handle this case?

If you have one message source (a single thread or some kind of coordination), and the messages have lower frequency than the timestamp resolution, yes.

The farther you get from that, the more the answer is no.

Re: Google pub/sub released an ordering feature

#49
post #26

Earlier quoted context omitted.

I understand. I’m just saying that in my my professional experience, nothing good came out of religously avoiding lock-in but the opposite.

With respect, clearly you cannot extrapolate that particular experience very far. Otherwise, eg, FOSS is "nothing good".

I'm not talking about "that" particular experience, but in general.
Post reply on HN