Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
1–10 of 33 posts
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#2What are some typical use cases for Amazon Kinesis streams, on the web?
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#3Just so I'm understanding, is this a Kafka competitor?
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#4What are some typical use cases for Amazon Kinesis streams, on the web?
Analytics event or IoT event ingestion, for example. Then, you can hook up AWS Lambda to the Kinesis stream to actually do any processing/aggregation.
Or, if you're using the new Firehose product, you could also use Lambda and attach it to the S3 event source using the bucket that the Firehose dumps into to perform batch processing on these records.
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#5Just so I'm understanding, is this a Kafka competitor?
Kinesis is basically Kafka as a service. IIUC, Kinesis Firehose is just a way to ingest data into S3 buckets/Redshift and Firehose would give you the goodies like monitoring/fault tolerance etc.
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#6At Asana, we've been beta testing Kinesis Firehose. It's been quite convenient not having to manage much, and having the data end up in S3. We're also using Kinesis streams, and have a simple KCL app to pull from streams and write to Firehose. We're looking forward to when streams can be as easy to manage, or when we KCL apps can read from Firehose.
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#7Just so I'm understanding, is this a Kafka competitor?
The closest thing I could compare it to would be services that make it easier to get data into Redshift or S3. E.g. segment.io's redshift product: https://segment.com/redshift.
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#8Hmmmm, I certainly hope dang or another admin consolidate all these stories into a single "Amazon" thread. Can't have multiple stories from a single company eating up space on the front page.
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#9Just so I'm understanding, is this a Kafka competitor?
This looks like an AWS version of Apache Flume and Kinesis looks like an AWS version of Apache Storm. Does AWS have a Kafka equivalent i.e. a pub/sub message queue?
Re: Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
#10This is basically kafka + https://github.com/linkedin/camus which is pretty cool.