11–20 of 42 posts
What's going on with Amazon recently? We're seeing a torrent of new technologies and platform offerings. Are we finally catching a glimpse of Bezos's grand scheme?
What's going on with Amazon recently? We're seeing a torrent of new technologies and platform offerings. Are we finally catching a glimpse of Bezos's grand scheme?
Amazon's reinvent conference[0] has been going on over the last few days, it's an obvious time/place to announce. [0] https://reinvent.awsevents.com/
I'm really excited about this - data streaming has been a crucial missing piece for building large-scale apps on AWS. If the performance and pricing are right it's going to relieve a lot of headaches in terms of infrastructure management.
I'm really excited about this - data streaming has been a crucial missing piece for building large-scale apps on AWS. If the performance and pricing are right it's going to relieve a lot of headaches in terms of infrastructure management.
Forgive my ignorance, but what would this potentially replace? Kafka/Storm/Something else?
Seems like a useful reworking of SQS, but all the hard work is being done in the client: "client library automatically handle complex issues like adapting to changes in stream volume, load-balancing streaming data, coordinating distributed services, and processing data with fault-tolerance." Unfortunately, there's no explanation of the mechanics of coordination and fault tolerance, so the hard part appears to be vapo…
> Unfortunately, there's no explanation of the mechanics of coordination and fault tolerance, so the hard part appears to be vaporware. I think it's unfair to call it vaporware - Amazon doesn't tend to release vaporware. You can also be fairly confident this has been in private beta for some time, so we'll probably see a few blog posts about it from some of their privileged (big spending) clients - typically someone…
Its less ideal from a maintenance perspective, because there will need to be feature-rich clients in Java and C (with dynamic language bindings). Applications will be running many many versions of the clients. Also, for coordination, the clients will need to communicate, so there may be configuration and/or firewall issues for the app to resolve.
It will be interesting to see Amazon make this tradeoff for what I believe is the first time.
I'm really excited about this - data streaming has been a crucial missing piece for building large-scale apps on AWS. If the performance and pricing are right it's going to relieve a lot of headaches in terms of infrastructure management.
Forgive my ignorance, but what would this potentially replace? Kafka/Storm/Something else?
This is essentially a hosted Kafka ( http://kafka.apache.org/ ). Given the complexity of operating a distributed persistent queue, this could be a compelling alternative for AWS-centric environments. (We run a large Kafka cluster on AWS, and it is one of our highest-maintenance services.)
We are choosing Kafka over other solutions like RabbitMQ because we like the persistent txn-log-style messages and how cheap consumers are.