Google pub/sub released an ordering feature

#1

Post by santhoshkumar3 » Fri, Oct 02, 2020, 2:19 AM UTC

Google pub/sub released an ordering feature
cloud.google.com

Re: Google pub/sub released an ordering feature

#2

Post by rubayeet » Fri, Oct 02, 2020, 3:18 AM UTC

Whoa! This is a game changer! I was looking into both Kafka and Google Pub/Sub for a event-oriented system my team was designing. Google Pub/Sub looked very promising, but no guarantee of ordering was a deal breaker for us. I’ll consider this more strongly for the next system we build.

Re: Google pub/sub released an ordering feature

#3

Post by edoceo » Fri, Oct 02, 2020, 3:20 AM UTC

rubayeet wrote: post #2

Whoa! This is a game changer! I was looking into both Kafka and Google Pub/Sub for a event-oriented system my team was designing. Google Pub/Sub looked very promising, but no guarantee of ordering was a deal breaker for us. I’ll consider this more strongly for the next system we build.

What did you pick? I'm interested in non-G solutions.

Re: Google pub/sub released an ordering feature

#4

Post by doh » Fri, Oct 02, 2020, 3:24 AM UTC

rubayeet wrote: post #2

Whoa! This is a game changer! I was looking into both Kafka and Google Pub/Sub for a event-oriented system my team was designing. Google Pub/Sub looked very promising, but no guarantee of ordering was a deal breaker for us. I’ll consider this more strongly for the next system we build.

We used Pub/Sub very extensively (50B messages a day) but moved to Pulsar [0]. It performs equally well and has some nice features. And also no vendor lock-in.

[0] https://kesque.com/billions-of-events-a-day-without-breaking...

Re: Google pub/sub released an ordering feature

#5

Post by erikerikson » Fri, Oct 02, 2020, 3:35 AM UTC

I'm floored by this. Last I spoke with that team the response was a few (but not everyone) in leadership getting it and everyone else asking why we would care. How far to come. GCP just got substantially more interesting.

Edit: by this I only mean to say well done to the Google team.

Re: Google pub/sub released an ordering feature

#6

Post by hulahoof » Fri, Oct 02, 2020, 3:42 AM UTC

"Receiving messages in order might increase latency."

It would be good to know what sort of overhead we are looking at here; very interested in this feature

Re: Google pub/sub released an ordering feature

#7

Post by [deleted] » Fri, Oct 02, 2020, 3:43 AM UTC

[deleted]

Re: Google pub/sub released an ordering feature

#8

Post by fastest963 » Fri, Oct 02, 2020, 3:45 AM UTC

hulahoof wrote: post #6

"Receiving messages in order might increase latency." It would be good to know what sort of overhead we are looking at here; very interested in this feature

And do messages without an ordering key incur the same overhead?

Re: Google pub/sub released an ordering feature

#9

Post by jeffbee » Fri, Oct 02, 2020, 3:48 AM UTC

Head of line blocking as a service!

Re: Google pub/sub released an ordering feature

#10

Post by hongalex » Fri, Oct 02, 2020, 3:50 AM UTC

erikerikson wrote: post #5

I'm floored by this. Last I spoke with that team the response was a few (but not everyone) in leadership getting it and everyone else asking why we would care. How far to come. GCP just got substantially more interesting. Edit: by this I only mean to say well done to the Google team.

By the way, in addition to ordered messages, Pub/Sub has recently gotten a number of new features including message filtering, dead letter queues, and retry policy (some GA, some beta). You can find out more here: https://cloud.google.com/pubsub/docs/release-notes

Disclaimer: I work on Pub/Sub