Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TimescaleDB vs. Amazon Timestream

blog.timescale.com

Post reply on HN
Print view View original

81–90 of 203 posts

Re: TimescaleDB vs. Amazon Timestream

#81

Post by sradman »

nautilus12 wrote: post #60

It doesn't matter, look at kinesis. People are willing to use it over kafka (even though AWS has a hosted kafka solution), just because the percieved convenience and interconnectivity to other AWS services. If timescale starts to make serious money, then AWS can use this to mobilize their engineering and make it fast and take timescale out. Amazon's goal is literally to be the only kid on the block for everything for…

Amazon Kinesis is based on Apache Flink. One complaint might be the branding but keep in mind that the Apache Foundation has a strict policy of enforcing trademarks. The Apache Foundation itself doesn't choose winners and many equivalent projects live under the same umbrella, just like Flink and Kafka.

Performance is often transient and it isn't the only criteria used when choosing a vendor. Some enterprise customers prefer a single vendor, i.e., "one throat to choke". Others have very specific constraints: greenfield decision making is a luxury.

My impression is that Amazon responds to customer requests to solve specific problems and they make a decent effort to continuously improve the performance and other aspects of their services. At one point people said "no one gets fired for choosing IBM" and this switched to Microsoft and, rightly or wrongly, Amazon AWS now wears this crown.

Re: TimescaleDB vs. Amazon Timestream

#82

Post by eloff »

SmokeyHamster wrote: post #79
eloff wrote: post #57

If you take one thing away from this it's do not use AWS timestream, and if you are using it, get rid of it quick. It's unfit for purpose. That first graph tells you everything you need to know. Time to insert a billion events: TimescaleDB: 5 min AWS Timestream: ~2 weeks! I think the team at AWS that built it should be reassigned and contractors or an A team brought in to try and salvage it. Because they've built the…

Yeah, differences like that are why I ditched MySQL years ago in favor of PostgreSQL. I was using MySQL at the time, and did a performance comparison of how long it took each database to dump and load a snapshot of a 100GB database. MySQL took 3 days. PostgreSQL took 30 minutes. I posted on a MySQL forum asking why the drastic difference, and they tried to hand-wave it away by saying I had a lot of indexes. Well yeah…

That's surprising to me since Postgres does less efficient writes compared to MySQL in order to optimize for read queries[1] - a sensible tradeoff since most of the time reads are more common than writes in OLTP workloads. TimescaleDB essentially solves this problem for the specific case of time-series inserts.

  [In Postgres] if we have a table with a dozen indexes defined on it, an update to a field that is only covered by a single index must be propagated into all 12 indexes to reflect the ctid for the new row.[1]
[1] https://eng.uber.com/postgres-to-mysql-migration/

Re: TimescaleDB vs. Amazon Timestream

#83

Post by momothereal »

mfreed wrote: post #74
momothereal wrote: post #68

OT: Every time I see Timescale on here I'm excited to use it. After some preliminary testing I always run into the same issue: can't create a hypertable if there's a unique index / primary key that doesn't include the 'time' field. For example my data looks like (row_id, user_id, time, data), where row_id is a unique ID, and time is a non-unique time field. Timescale will refuse to create a hypertable like this becau…

You can create a hypertable with a UNIQUE composite key on (row_id, time). Otherwise, you are correct in that we require your partitioning keys to be at least _part_ of unique constraints; otherwise, we'd need to build global indexes across all your chunks (which would inhibit scalability)...this would only be worse with multi-node =)

Wouldn't that just make the pair unique? That would still allow you to insert rows with the same row_id, but different time.

Re: TimescaleDB vs. Amazon Timestream

#84

Post by avthar »

oblio wrote: post #43
rattray wrote: post #21

Full title includes a summary of benchmark results for the lazy: Timescaledb achieved 6000x higher inserts, 5-175x faster queries, 150x-220x cheaper Benchmarks open-source, methodology in post.

If I were them I wouldn't rest on my laurels. v1 from AWS is an MVP. It will keep getting improved. I'd like to see those numbers 2-3 years from now.

Timescale employee here. We generally update our benchmark blogs every year to keep them fresh. No doubt you'll see the updated results here on HN as well!

Re: TimescaleDB vs. Amazon Timestream

#85

Post by mavelikara »

LukeEF wrote: post #41
tester34 wrote: post #11

Earlier quoted context omitted.

Working at AWS must be fun You basically reinvent all the stuff - from databases to all avaliable tooling normal companies dont do that, so there' an opportunity to get deep into some specific branch of applied informatics.

But why do they do it so badly? Seems like they have the engineering muscle to put out a great reinvention of a document db, a time series db, a graph db... but each is relatively poor to the other services on AWS. Is it the combination of MVP-culture and lock-in?

The value they sell is really operational convenience. Over time they have figured out how to run large services at scale, and monetize that. The features of those services are often weaker than competition, as noted in this thread.

Re: TimescaleDB vs. Amazon Timestream

#86

Post by ahachete »

ryanbooz wrote: post #77
ahachete wrote: post #70

I enjoy trying to guess how cloud services like Amazon Timestream are internally built. I have already bet in the past that DocumentDB was built on top of (Aurora) PostgreSQL (this I also know was possible as I founded https://torodb.com ). My bet for Timestream is that is built on top of DynamoDB. There are many potential indicators (1KB writes, throttling) and some clear ones(pricing follows the exact proportion up…

This is a likely option, too. Your observations on pricing are keen. The one thing that threw me was querying. The limitations felt more Athena-like than PartiQL. And the billing based on scans felt almost like Redshift Spectrum. I mean, S3 is infinitely scalable, right? It's just impossible to say right now, eh?

Note that DynamoDB's PartiQL support is not an architectural pattern but rather a "simple" translation layer on top of the current three read operations. It doesn't say much to me tbh.

I wouldn't say is impossible to say. Maybe impossible with 100% certainty, obviously, but for me it's quite clear ;)

Re: TimescaleDB vs. Amazon Timestream

#87

Post by dumbfounder »

ameyv wrote: post #64
dumbfounder wrote: post #58

Earlier quoted context omitted.

That is not the mantra at our company. We want serverless and managed services whenever possible because getting resources to setup and maintain anything is next to impossible. We are now trying to use some competing products to AWS products that are full SaaS, so theoretically great, but because it isn't tacked on to our AWS bill it is very hard to get it approved and we are waiting around for weeks turning into mon…

I'm not saying you should not! But lets just say you want ELK stack, you can try ELK cloud from Elastic which is hosted on AWS or Azure Or GCP as managed services But Elastic company manages it...That also help those folks support themselves.

Hosted on AWS does not mean it gets wrapped up in the AWS bill. Which means we need to sign a separate contract, which is painful to make happen.

Re: TimescaleDB vs. Amazon Timestream

#88

Post by ctvo »

xyst wrote: post #42

What an embarrassment for AWS - getting smoked by an open source project. Should just fire the management team overseeing the timestream project and just provide managed timescaledb instances.

Unfortunately that company will double down even if there's a hint of traction, and in 3 years it'll finally be decent and TimescaleDB's cloud offering will be in trouble.

They have a lot of money and a bit of time.

It is very embarrassing that Timestream was delayed for so long only to lose so badly.

Re: TimescaleDB vs. Amazon Timestream

#89

Post by nwsm »

On topic:

While the comparison seems terrible for Timestream, as a customer who does not want to manage my databases I would love a similar GCP option, if the product had better tradeoffs.

It's also interesting that Timescale attributes this AWS product to their own licensing. They had some much discussed [0] developments on that front, and if it did in fact force AWS to build their own implementation, that seems like a win for opensource, but not so much for serverless users.

Slightly off topic:

I have a side project that generates ~10 daily metrics for ~350 entities. Around 150k readings currently (the number of metrics gathered per entity per day has increased since I started).

I'm pretty interested in these timeseries databases, but I think my use case is still on the side of using MySQL/Postgres out of the box. Not to mention that I can get by with archiving data > 1 year old.

Does anyone have any quick checklist or metrics to make decisions like this? When does it make sense to evolve from a vanilla RDB to a timeseries one?

[0] https://news.ycombinator.com/item?id=24579905

Post reply on HN