111–114 of 114 posts
Earlier quoted context omitted.
There's no ordering in HTTP. If you could send a whole HTTP request as a UDP packet you'd get exactly the same protocol (obviously sans WebSockets - but you could work around that).
How would you process an unordered response?
This looks like a naive implementation of adding ordering keys on one side, and waiting until all messages arrive on the other in order, without improving the underlying delivery transport to support ordering. From what I can tell, the publisher is responsible for now maintaining arbitrary ordering IDs and passing this in the publish method, see https://github.com/googleapis/nodejs-pubsub/blob/master/samp... . Given…
> the publisher is responsible for now maintaining arbitrary ordering IDs and passing this in the publish method Ah, thanks for pointing this out. The "ordering ID" bit is actually a remnant of how to publish messages before we had officially ordering key support (and only shows up in Node, but not our other samples https://cloud.google.com/pubsub/docs/publisher#using_orderin... ). We'll be fixing this sample shortly…
Out of interest then, what is the reasoning behind the warning about additional latency. Is Google Pub/Sub doing some kind of reconstruction on the receiving end to piece together an unordered stream, or has Google Pub/Sub changed how the underlying streaming primitives work to natively support ordering.
What's the TPS rate limit?
As my colleague, kamalaboulhosn, posted elsewhere in this thread: "The throughput limitation is 1MB/s per ordering key. The number of unique ordering keys allowed on a topic is limited only by what can be represented by a 1KB string, so very high throughput on a topic is still possible." So, it's more of a throughput limit, not a message limit, per key. With small messages, you can get high "TPS". Disclaimer: I work…
This is definitely a surprising, if welcome, development from GCP. We used to be pretty significant users of Pub/Sub but migrated to Cloud Tasks after several discussions with our account manager indicated this wasn't the direction they wanted to go with PubSub. The implementation here also seems to be somewhat unusual at first glass - in particular that a retry of any given tasks appears to also retry any subsequent…
The ordering keys feature supports a large number of keys (though since the throughput limit is 1 MB/sec per key, many applications shouldn't have issues scaling up on a given key). Imagine you have an order processing system where you have to 1) write to a database 2) write to a metrics log 3) and send an email to the customer. You can publish a message with the ordering key being the user who initiated the order. T…