It's got to be a rough day for the team at https://www.influxdata.com/ . This could become serious competition for their InfluxCloud hosted offering.
We've been expecting this for two years. It was just a matter of when. It validates our space. AWS did this to Elastic, they have competing products with NewRelic, Splunk, SumoLogic, and countless others. All of whom still have healthy businesses. Our goal remains the same: build the best possible product that optimizes for developer productivity and happiness. And open source as much as we possibly can while maintai…
Amazon Timestream – Fast, scalable, fully managed time series database
101–110 of 131 posts
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#102Earlier quoted context omitted.
At what point are open source projects going to change their licensing to prevent the major cloud providers from just stealing their products? I highly doubt AWS built this from scratch. Amazon, Google, and Microsoft are going to choke the life out of these projects Redis and MongoDB at least seem to have woken up https://www.geekwire.com/2018/open-source-companies-consider...
> At what point are open source projects going to change their licensing to prevent the major cloud providers from just stealing their products? There's a lot to unravel in there. I prefer 'free software' to 'open source' as it has a clearer meaning, especially in this context. Even so, no one can steal free / open source software (or as you say, product -- though that turn strongly implies a commercial offering). By…
Open source is software that through the license enforces the source code to remain open.
I'm not a fan of RMS or his attitudes on most things, but am a strong OSS fan as it is the best way to develop and maintain software.
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#103This is not cheap for the "DevOps" use case. Imagine you have 1000 servers submitting data to 100 timeseries each minute. That's 100,000 writes a minute (unless they support batch writes across series) At $0.50 per million writes that's $72 a day or $26k a year. Now imagine you want to alert on that data. Say you have 100 monitors that each evaluate 1GB of data once a minute. At $10 per TB of data scanned, that's $1,…
Well, that depends on what you consider cheap. Hiring someone to manage a time series system like graphite or prometheus is going to cost you a whole lot more than $26k a year
Self-hosting isn't the only option though. For example, that hypothetical 1000 server scenario would cost $180k a year at list pricing on Datadog or SignalFX.
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#104I wonder how this compares to KDB
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#105This is not cheap for the "DevOps" use case. Imagine you have 1000 servers submitting data to 100 timeseries each minute. That's 100,000 writes a minute (unless they support batch writes across series) At $0.50 per million writes that's $72 a day or $26k a year. Now imagine you want to alert on that data. Say you have 100 monitors that each evaluate 1GB of data once a minute. At $10 per TB of data scanned, that's $1,…
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#106This is not cheap for the "DevOps" use case. Imagine you have 1000 servers submitting data to 100 timeseries each minute. That's 100,000 writes a minute (unless they support batch writes across series) At $0.50 per million writes that's $72 a day or $26k a year. Now imagine you want to alert on that data. Say you have 100 monitors that each evaluate 1GB of data once a minute. At $10 per TB of data scanned, that's $1,…
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#107At my day job, I build a lot of machine learning systems that require data to be fed in a time series manner[1]. Often this means building systems to analyze terabytes of logs [semi]-realtime. All I have to say is - thank god! This is going to make my job a lot easier, and likely empower us to remove our current infrastructure setup. I know at one point we actually considered building our own time series database. In…
I wouldn't jump the gun on this. I've been working within Amazon cloud for years and every year they make massive claims about new services at re:invent. Not saying this isn't going to be a good product, just saying it will probably take a while to be as useful as you're hoping.
They make insane promises but, the promises don't like up to expectations.
Kinesis analytics for example, can aggregate data across a time window (sliding window) from a stream (Kinesis). A huge issue that isn't document or stated is that when kinesis analytics restarts due to the process dies (being migrated, binpacked, etc.) the ENTIRE time window has to get re-aggregated. So your count drops to 0.
Really unacceptable if you're using it to generate KPIs which you alert on. We ended up switching to a system which pushes the stream data into influx and running the aggregations there via queries.
Dealing w/ AWS during this entire process was a huge pain.
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#108At my day job, I build a lot of machine learning systems that require data to be fed in a time series manner[1]. Often this means building systems to analyze terabytes of logs [semi]-realtime. All I have to say is - thank god! This is going to make my job a lot easier, and likely empower us to remove our current infrastructure setup. I know at one point we actually considered building our own time series database. In…
Have you considered columnal databases like 1010data?
https://www.reddit.com/r/apljk/comments/42uf2f/not_exactly_a...
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#109So what will this compare wrt boundary, signalfx, stackdriver, etc types of previous services... Ill have to go look into this, because if aws historic pricing for any large volume stream, quickly becomes untennable. Its very easy to have gobs and gobs of time series points... aws might make using this way too expensive for anything at relative scale for a small startup?
Re: Amazon Timestream – Fast, scalable, fully managed time series database
#110Earlier quoted context omitted.
I might be mistaken but wouldn't Data Explorer be more similar to AWS CloudWatch which has been around for a long time.
Azure Data Explorer/ Kusto is more of a database that is optimized for the log use case than a service. There is a front end tool and a lot of the use-cases are around log management, but it is database you can do general SQL or KQL things with. Time series is one of the core use-cases for it also but it has less marketing around it.