Live data from Hacker News

Amazon Timestream – Fast, scalable, fully managed time series database

aws.amazon.com

101–110 of 131 posts

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#101
post #73
post #30

It's got to be a rough day for the team at https://www.influxdata.com/ . This could become serious competition for their InfluxCloud hosted offering.

We've been expecting this for two years. It was just a matter of when. It validates our space. AWS did this to Elastic, they have competing products with NewRelic, Splunk, SumoLogic, and countless others. All of whom still have healthy businesses. Our goal remains the same: build the best possible product that optimizes for developer productivity and happiness. And open source as much as we possibly can while maintai…

Can't wait to see your new cloud offering :)

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#102
post #97

Earlier quoted context omitted.

At what point are open source projects going to change their licensing to prevent the major cloud providers from just stealing their products? I highly doubt AWS built this from scratch. Amazon, Google, and Microsoft are going to choke the life out of these projects Redis and MongoDB at least seem to have woken up https://www.geekwire.com/2018/open-source-companies-consider...

> At what point are open source projects going to change their licensing to prevent the major cloud providers from just stealing their products? There's a lot to unravel in there. I prefer 'free software' to 'open source' as it has a clearer meaning, especially in this context. Even so, no one can steal free / open source software (or as you say, product -- though that turn strongly implies a commercial offering). By…

Free software and open source aren't always the same thing though. Free Software is software that through the license enforces a philosophy.

Open source is software that through the license enforces the source code to remain open.

I'm not a fan of RMS or his attitudes on most things, but am a strong OSS fan as it is the best way to develop and maintain software.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#103
post #84

This is not cheap for the "DevOps" use case. Imagine you have 1000 servers submitting data to 100 timeseries each minute. That's 100,000 writes a minute (unless they support batch writes across series) At $0.50 per million writes that's $72 a day or $26k a year. Now imagine you want to alert on that data. Say you have 100 monitors that each evaluate 1GB of data once a minute. At $10 per TB of data scanned, that's $1,…

Well, that depends on what you consider cheap. Hiring someone to manage a time series system like graphite or prometheus is going to cost you a whole lot more than $26k a year

You're ignoring half of my example scenario. $26k writing the data, $525k for querying it just for alerting, plus whatever it costs to store and to query ad-hoc. That's over half a million dollars. Even if you hire someone for $250k, you can self-host your time series system for cheaper than that.

Self-hosting isn't the only option though. For example, that hypothetical 1000 server scenario would cost $180k a year at list pricing on Datadog or SignalFX.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#105
post #84

This is not cheap for the "DevOps" use case. Imagine you have 1000 servers submitting data to 100 timeseries each minute. That's 100,000 writes a minute (unless they support batch writes across series) At $0.50 per million writes that's $72 a day or $26k a year. Now imagine you want to alert on that data. Say you have 100 monitors that each evaluate 1GB of data once a minute. At $10 per TB of data scanned, that's $1,…

It's priced for government contracts.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#106
post #84

This is not cheap for the "DevOps" use case. Imagine you have 1000 servers submitting data to 100 timeseries each minute. That's 100,000 writes a minute (unless they support batch writes across series) At $0.50 per million writes that's $72 a day or $26k a year. Now imagine you want to alert on that data. Say you have 100 monitors that each evaluate 1GB of data once a minute. At $10 per TB of data scanned, that's $1,…

That's convoluted... from you example, there are 100K writes/minute, while you assume 1GB data evaluated per minute per alarm. That is you're assuming 10K/item/timeseries for each alarm, while reality is going to be closer to 10-100 bytes/item/time-series, which cuts down the expense by two or three orders of magnitude.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#107
post #36

At my day job, I build a lot of machine learning systems that require data to be fed in a time series manner[1]. Often this means building systems to analyze terabytes of logs [semi]-realtime. All I have to say is - thank god! This is going to make my job a lot easier, and likely empower us to remove our current infrastructure setup. I know at one point we actually considered building our own time series database. In…

I wouldn't jump the gun on this. I've been working within Amazon cloud for years and every year they make massive claims about new services at re:invent. Not saying this isn't going to be a good product, just saying it will probably take a while to be as useful as you're hoping.

I agree w/ this.

They make insane promises but, the promises don't like up to expectations.

Kinesis analytics for example, can aggregate data across a time window (sliding window) from a stream (Kinesis). A huge issue that isn't document or stated is that when kinesis analytics restarts due to the process dies (being migrated, binpacked, etc.) the ENTIRE time window has to get re-aggregated. So your count drops to 0.

Really unacceptable if you're using it to generate KPIs which you alert on. We ended up switching to a system which pushes the stream data into influx and running the aggregations there via queries.

Dealing w/ AWS during this entire process was a huge pain.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#108

At my day job, I build a lot of machine learning systems that require data to be fed in a time series manner[1]. Often this means building systems to analyze terabytes of logs [semi]-realtime. All I have to say is - thank god! This is going to make my job a lot easier, and likely empower us to remove our current infrastructure setup. I know at one point we actually considered building our own time series database. In…

Have you considered columnal databases like 1010data?

I recall why this looks familiar. Their chief scientist put up a job posting on the subreddit for array languages. It sounds like they know how to use some pretty powerful tools.

https://www.reddit.com/r/apljk/comments/42uf2f/not_exactly_a...

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#109

So what will this compare wrt boundary, signalfx, stackdriver, etc types of previous services... Ill have to go look into this, because if aws historic pricing for any large volume stream, quickly becomes untennable. Its very easy to have gobs and gobs of time series points... aws might make using this way too expensive for anything at relative scale for a small startup?

It appears that ingestion alone is more expensive than the commercial metric services. Might not matter for small scale.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#110
post #24

Earlier quoted context omitted.

I might be mistaken but wouldn't Data Explorer be more similar to AWS CloudWatch which has been around for a long time.

Azure Data Explorer/ Kusto is more of a database that is optimized for the log use case than a service. There is a front end tool and a lot of the use-cases are around log management, but it is database you can do general SQL or KQL things with. Time series is one of the core use-cases for it also but it has less marketing around it.

Sounds more like AWS CloudWatch Log Insights launched yesterday.
Post reply on HN