Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Timestream – Fast, scalable, fully managed time series database

aws.amazon.com

Post reply on HN
Print view View original

71–80 of 131 posts

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#71

Post by stingraycharles »

dominotw wrote: post #55
sciurus wrote: post #30

It's got to be a rough day for the team at https://www.influxdata.com/ . This could become serious competition for their InfluxCloud hosted offering.

and for timescaledb, streamlio, two sigma, kdb+ , quasardb ( and perhaps pipelinedb)

QuasarDB employee chiming in - we’re not actually worried, our clients are typically operating at a scale that would make costs very prohibitive for this AWS product.

Having said that, I can definitely see this be an interesting product for people doing less than 10k inserts per second.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#72

Post by dominotw »

stingraycharles wrote: post #71
dominotw wrote: post #55

Earlier quoted context omitted.

and for timescaledb, streamlio, two sigma, kdb+ , quasardb ( and perhaps pipelinedb)

QuasarDB employee chiming in - we’re not actually worried, our clients are typically operating at a scale that would make costs very prohibitive for this AWS product. Having said that, I can definitely see this be an interesting product for people doing less than 10k inserts per second.

> make costs very prohibitive for this AWS product.

Competing with AWS on just cost sounds worrying to me.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#73

Post by pauldix »

sciurus wrote: post #30

It's got to be a rough day for the team at https://www.influxdata.com/ . This could become serious competition for their InfluxCloud hosted offering.

We've been expecting this for two years. It was just a matter of when. It validates our space. AWS did this to Elastic, they have competing products with NewRelic, Splunk, SumoLogic, and countless others. All of whom still have healthy businesses.

Our goal remains the same: build the best possible product that optimizes for developer productivity and happiness. And open source as much as we possibly can while maintaining a healthy business.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#74

Post by tonyedgecombe »

ravedave5 wrote: post #38
willlll wrote: post #26

I'm actually impressed at how incredibly expensive they made this. $0.50 per million 1KB writes, which is 20x what aurora charges, since aurora allows 8KB writes. And Aurora is already expensive if you actually read/write to it.

I get the feeling this is for important data (banking etc) so I have a feeling this is 200x cheaper than whatever else is available.

That's not what is says in the release:

"With Timestream, you can easily store and analyze log data for DevOps, sensor data for IoT applications, and industrial telemetry data for equipment maintenance."

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#75

Post by danso »

cheeze wrote: post #35
danso wrote: post #19

Earlier quoted context omitted.

It's AWS re:Invent day/week. I've never been, but I get the impression that it's like Apple's keynote, or Google I/O, in which big product announcements are made. On those days, you'll see multiple submissions about the respective conferences too.

This is exactly what re:Invent is. Most teams dream of launching a new AWS product at re:Invent (and not missing their date and launching at a later time)

It'd be an interesting blog post topic to look at how AWS:Invent (or I/O, or F8, etc) product threads on HN compare to actual product impact. I remember Rekognition getting decent discussion 2 years ago [0], but not along the angles or magnitude of how Rekognition is usually discussed in recent months. OTOH, other things I've been interested in as a data geek, I've barely heard of since reading about them on HN -- e.g. Athena [1]

[0] https://news.ycombinator.com/item?id=13072956

[1] https://news.ycombinator.com/item?id=13072245

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#76

Post by ilaksh »

temuze wrote: post #25
ilaksh wrote: post #22

Earlier quoted context omitted.

Sure you can do if you're really sure that you won't need to group by something else later. You wouldn't want to store more granularity than necessary but you can't go back in time to get a data point you didn't store.

In that case, can you just store each data point in a data lake somewhere and do a batch-job? Apache Flink supports this use case as well as real-time.

Yes but I guess the point of using time series DB rather than just a lake that doesn't necessarily have a time structure is that if you know time is going to be important then you probably want to organize and query it that way.

What I am doing with one program could almost be called a data lake because it is just a bunch of JSONL files that have really varied data in them. But it's organized by date and hour per day as well as predefined keys, since I know I will need to query it that way.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#77

Post by mathattack »

citilife wrote: post #20

At my day job, I build a lot of machine learning systems that require data to be fed in a time series manner[1]. Often this means building systems to analyze terabytes of logs [semi]-realtime. All I have to say is - thank god! This is going to make my job a lot easier, and likely empower us to remove our current infrastructure setup. I know at one point we actually considered building our own time series database. In…

Have you considered columnal databases like 1010data?

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#78

Post by coredog64 »

I'm wondering if this shares any technology with the CloudWatch metrics backend. They've been making improvements there all year, and most of them generally align with what's announced here.

CloudWatch metrics are also very expensive for what you get, so that's another similarity to Timestream ;)

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#79

Post by StreamBright »

lykr0n wrote: post #43
citilife wrote: post #20

At my day job, I build a lot of machine learning systems that require data to be fed in a time series manner[1]. Often this means building systems to analyze terabytes of logs [semi]-realtime. All I have to say is - thank god! This is going to make my job a lot easier, and likely empower us to remove our current infrastructure setup. I know at one point we actually considered building our own time series database. In…

Have you looked at Clickhouse for timeseries data? It's the one database I've found that can scale and can query in near-realtime. I've loaded a 100 Billion Rows in into a 5 shard database and can do full queries across the whole dataset in under 10 seconds. It also natively consumes multiple kafka topics.

Do you have a schema available publicly? I would like to build a similar system using custom software + S3 + Parquet + Athena for this task and see if it works.

Re: Amazon Timestream – Fast, scalable, fully managed time series database

#80

Post by barake »

willlll wrote: post #26

I'm actually impressed at how incredibly expensive they made this. $0.50 per million 1KB writes, which is 20x what aurora charges, since aurora allows 8KB writes. And Aurora is already expensive if you actually read/write to it.

The pricing lines up with CloudWatch Logs, 50 cents/GB, 3 cents/GB/month.

Curious to see what the query language is for this, wonder if they're just exposing the backing store for CloudWatch as a service now.

Post reply on HN