TimescaleDB vs. Amazon Timestream

I'm not a database expert in any way, but I've had to evaluate database options for both existing and new applications, and my "methodology" is as follows:

- Understand the workload

Is the database workload read-intensive, write-intensive, or both? Does the application require strong consistency? Does it require replication? What kind of replication? Can writes be batched, or are processed ad-hoc? What kind of read operations will you do, and what is the desired response times for those operations (ex. is 2s tolerable? 500ms? 20ms?)

- Scouting

Select a set of available options to benchmark using whatever criteria you may see fit, but following the previous step constraints.

- Benchmark

Create a somewhat simple model that fits your workload and models your real case, using the language you'll be using to build your application; Create benchmark code for the most critical/complex operations; Run it at least 5 times on similar hardware of what you expect production to be (this is important; eg. storage (both memory and disk) latency varies wildly between "my laptop" and a cloud provider) and record the results; Be sure to keep an eye on memory consumption, cpu and disk usage when running the benchmarks;

- Decide :)

Usual criteria:

* Price;

* Features (of course);

* Easiness of deploy or cloud provider availablity;

* Documentation quality;

* Driver quality for your application language (it may happen you already excluded a candidate on the previous step due to this);

* Query language familiarity;

* Performance;

* Adoption (translated to: can I expect a new developer to have the basic skills on this tech?);

That's a knee-jerk assessment. You don't know the first thing about Timestream's development and we are only a few years into the development process of what I believe is a novel architecture for a timeseries database. Give it another couple of years and then we can take a look at how Timestream is doing.

One of Amazon's advantages is the ability to plan and execute on a longer time scale than their competitors. They can take approaches that take longer to bear fruit but are better in the long term.

Timestream is a scalable timeseries database with separated compute and storage built for extremely high volumes (or that's my guess given the architecture offloads cold data to magnetic storage). Whereas Timescale seems to be timeseries functionality added to Postgres - which means they are probably going to hit scaling (perf/cost) issues once they need to work at petabyte scale. And my guess would be that their coupling to Postgres is going to make it painful to build for that scale. They also claim to offer separated compute and storage, but based on the pricing model it seems to be that you can change your CPU+Storage configuration, not an ephemeral design where you only pay for compute when it's needed - very different from Timestream's ephemeral pricing model. This is a pure guess, but I would imagine that Timescale being built on top of Postgres is going to make a truly serverless SaaS (i.e. pay for only what you use) very difficult to build.

Time will tell, but this is sort of if MySQL benchmarked writes against Snowflake 5 years ago and then thinking that everyone at Snowflake should be fired.

Not sure about "comparably provisioned re: IOPS" given that you don't know about that at all with AWS Timestream, but our blog post reports the performance/cost of a suitably provisioned Timescale Forge instance (8vCPU / 1TB storage).

100GB ingest then query benchmarks took far less than 1 hour with Timescale Forge @ $2.18/hour.

The "consumption based pricing" for AWS Timestream for same benchmark took $336.40.

> 1 remote client machine, 1 database server, both in the same cloud datacenter

> Disk Size: 4.8TB of disk in a raid0 configuration (EXT4 filesystem)

Both those statements lead me to believe it's a single server with locally attached SSDs in a RAID0. Which is it?

I know benchmarking is hard, and it's difficult to test certain aspects of Amazon Timestream due to it being a managed service, but I really think these details need to be firmed up to make sure you are comparing apples to apples. TimescaleDB seems like a cool product.

Another suggestion I'd make is to run the Amazon Timestream clients across multiple AZs if you aren't already. The blog post doesn't mention whether all the t3 instances are in the same AZ or not.