Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TimescaleDB vs. Amazon Timestream

blog.timescale.com

Post reply on HN
Print view View original

151–160 of 203 posts

Re: TimescaleDB vs. Amazon Timestream

#151

Post by djk447 »

momothereal wrote: post #146
djk447 wrote: post #144

Earlier quoted context omitted.

Why not create an index on user_id, time? Why do you need a unique reference back to the row? We find that they're often (though certainly not always) kind of meaningless and/or unnecessary for timeseries workloads.

Yes that's what the reply suggested. However the business requirement is that the row_id should be unique, such that looking up a row by ID is guaranteed to have 0-1 results. A unique constraint on (row_id, time) doesn't satisfy that.

Sorry, the previous reply suggested it was on row_id, time I thought, anyway. I think the question is why should row_id be unique? What looks it up by row_id? And can they look it up by user_id/time instead? It may be impossible. On the other hand, if row_id is really what you're searching by, not by time, then just use our partitioning on row_id, assuming it's a bigint or something like that, you can just partition on that...

Re: TimescaleDB vs. Amazon Timestream

#152

Post by kenhwang »

cheesgrateron wrote: post #104
zachd1_618 wrote: post #73

Earlier quoted context omitted.

I think AWS is early to offer a not-yet-matured product just to get it in front of customers and start gathering real usage feedback. I’d consider dynamo a fantastic tier 1 service, but it wasn’t always that way. This is inline with the Amazon philosophy in general. As for “making something for the sake of having it”, I believe the key reason for building their own applications is that they can build it on the same m…

>I think AWS is early to offer a not-yet-matured product just to get it in front of customers and start gathering real usage feedback. It's not just this. In many cases, the services launched are essentially MVPs that were only created to serve a handful of customers' needs. If a huge whale of a customer says they want hosted Jupyter notebooks on AWS (just a theoretical example I picked because I saw someone criticiz…

I used to work for one of AWS's biggest customers. This was 100% how we operated, we told AWS what we wanted them to solve, and they would build it for us. When we're done closed beta testing it and getting changes in, it gets released as a public service a year later or so.

Re: TimescaleDB vs. Amazon Timestream

#153

Post by misterbwong »

Serious question: Why doesn't AMZ just buy TimeScaleDB instead of building their own? Time series db's in general seem to fit their vibe pretty well: it's a new hot tech, it's usually deeply embedded into infrastructure, and requires loads of bandwidth & storage.

Re: TimescaleDB vs. Amazon Timestream

#154

Post by momothereal »

djk447 wrote: post #151
momothereal wrote: post #146

Earlier quoted context omitted.

Yes that's what the reply suggested. However the business requirement is that the row_id should be unique, such that looking up a row by ID is guaranteed to have 0-1 results. A unique constraint on (row_id, time) doesn't satisfy that.

Sorry, the previous reply suggested it was on row_id, time I thought, anyway. I think the question is why should row_id be unique? What looks it up by row_id? And can they look it up by user_id/time instead? It may be impossible. On the other hand, if row_id is really what you're searching by, not by time, then just use our partitioning on row_id, assuming it's a bigint or something like that, you can just partition…

No problem, my fault I misread your comment. `row_id` is a unique identifier for the row, for example my API will need it when the user wants to delete the specific row. Since it's possible for multiple rows to have the same `time`, even for the same `user_id`, I cannot assume uniqueness there.

Partitioning on the row_id by making it an ordinal instead of a UUID could work, however I feel I would be missing out on TimescaleDB's advantages for querying based on the `time` filter?

Consider that my main queries are:

* DELETE FROM t WHERE row_id = xyz AND user_id = xyz

* INSERT INTO t (...)

* SELECT FROM t WHERE user_id = xyz AND time > xyz and time < xyz

Re: TimescaleDB vs. Amazon Timestream

#155

Post by ahachete »

jeremycarter wrote: post #111
ahachete wrote: post #70

I enjoy trying to guess how cloud services like Amazon Timestream are internally built. I have already bet in the past that DocumentDB was built on top of (Aurora) PostgreSQL (this I also know was possible as I founded https://torodb.com ). My bet for Timestream is that is built on top of DynamoDB. There are many potential indicators (1KB writes, throttling) and some clear ones(pricing follows the exact proportion up…

I might take toro for a spin this weekend.

It's no longer maintained :(

Re: TimescaleDB vs. Amazon Timestream

#156

Post by jamesblonde »

deanCommie wrote: post #145
superzamp wrote: post #9

Earlier quoted context omitted.

I remember the announcement of Sagemaker at an AWS summit, a moment of profound embarrassment - the whole thing was barely a Jupyter notebook glued to some EC2 stuff with duct tape. You're right, they're definitely doing things for the sole purpose of locking you in.

...have you looked at it recently? It's a MASSIVE suite of services, tools, and capabilities... https://aws.amazon.com/sagemaker/

You still have to compile your own Docker image and then "run it" from a Jupyter notebook if you are not using their Estimators (who uses those?). What is the point of just launching Docker images from Jupyter notebooks and looking a different window to see the log?

Serving is ok on Sagemaker, i will admit. The new ones: feature store, pipelines, and data wrangler - we will see, all rushed (some more than others).

Re: TimescaleDB vs. Amazon Timestream

#157

Post by netik »

Timestream hasn't even really been generally released as best I can tell. We wanted to use it, went to TimescaleDB, and then ended up going to Druid to improve performance.

Seriously, If you need to do this at scale, use Druid. It's much more efficient for time-series data.

Re: TimescaleDB vs. Amazon Timestream

#158

Post by arminiusreturns »

hardwaresofton wrote: post #54
ants_a wrote: post #46

Earlier quoted context omitted.

I did some time series data benchmarking recently. Most large data is "time series" data, but I will not digress on terminology right now. For the usecase I was looking at my results for InfluxDB did not qualitatively disagree the above blogpost. Timescale got better compression efficiency, faster query results, more constrained memory usage, but lower ingest speed at high concurrencies blocked by WAL insert locking…

I know the first thing that shocked me was how they could get so close to something that was purpose built for time series. Even being within spitting distance is really great in my opinion for an off the shelf, general tool. > However none of the databases tested above is anywhere close to the efficiency of a column store database that can do vectorized execution over batches of rows. ClickHouse is a good example of…

I'm out of my depth here (not a db guy but curious), so forgive the interruption, but have either of you messed with either SciDB or VoltDB? I have found them to be quite interesting especially given Stonebraker himself being involved in the design. Druid and timescale have also been on my radar too, and I think these comments have finally given me the push to play around with them in my sandbox.

Re: TimescaleDB vs. Amazon Timestream

#159

Post by chmod775 »

Ah yes. The horizontal scalability endgame.

Amazon designed a system that is perfectly horizontally scalable. The only issue is that they now need 33,000 machines to achieve the throughput of TimescaleDB running on a single node.

I would call the results this new school of software development produces horrifying, but I already get lynched every time I tell a startup that their horrifyingly complex horizontally 'scalable' AWS setup that costs them $10k a month could be replaced by a handful of ducktape running on a rPI.

The best part is how the scaling never really works and just adding a load balancer and a second rPI would've worked better.

"We didn't think we'd get that many users. I mean 10k! Whew! It just wasn't designed for this!"

"Oh. That sucks. My rPI is serving 2 million monthly visitors right now and I'm still waiting for CPU usage to dip into the double digits."

I'm not saying people should design their stuff to handle millions of users on underpowered hardware, but I am suggesting they not lose their sense of perspective while building their whatever.

Because if you do, you may end up thinking you did well when your DB maxes out at 500 inserts/second - Another DB running on a Nintendo DS might just outperform it.

Re: TimescaleDB vs. Amazon Timestream

#160

Post by a_gopher »

pstuart wrote: post #110
shin_lao wrote: post #12

Earlier quoted context omitted.

In a recent webinar for capital markets, AWS didn't even try to advertise Timestream, they talked about the three most popular tsdb in finance and that's it. Source: https://pages.awscloud.com/awsmp-h2-fin-time-series-database...

And those 3 are?

OneTick, Kx, and QuasarDB
Post reply on HN