Hey! I've been working on ListenBrainz [0] for the past 3-ish years. Happy to answer questions if anyone has any. [0]: https://listenbrainz.org
I read that whole page and I have no idea what the project does.
It's last.fm.
ListenBrainz moves to TimescaleDB
51–60 of 100 posts
Re: ListenBrainz moves to TimescaleDB
#52Hey! I've been working on ListenBrainz [0] for the past 3-ish years. Happy to answer questions if anyone has any. [0]: https://listenbrainz.org
I read that whole page and I have no idea what the project does.
Yeah, we need to fix that landing page. It's basically an open repository of your music listening history.
Re: ListenBrainz moves to TimescaleDB
#53Earlier quoted context omitted.
Can you elaborate on those issues?
It would be too long. To quickly summarize, from the pain of backups (unless you setup a WAL replica, the load may take your database down), the large size of the data on disk (timescale does offer some compression now, but it's still too much), the low performance of large queries, the memory requirements - it's death by a thousand papercuts! Don't get me wrong, timescale is a great way to get started with time seri…
Timescale has added their own layer of compression and columnar layouts to the Postgres row storage. That will get you to around 70% of the performance of using a dedicated column-oriented data warehouse, with the rest depending on how complex and selective your queries are.
It won't match the pure scan and computation speed of Clickhouse but the continuous aggregation feature is the recommended approach for querying large datasets (similar to Clickhouse table engines like AggregatingMergeTree).
Re: ListenBrainz moves to TimescaleDB
#54Earlier quoted context omitted.
This is actually pretty straightforward. For example, `DELETE FROM "foo" WHERE time >= now() - 2d AND time < now() - 1d`
Ok, we had an issue deleting data from a series. The delete just hung and never completed. I assumed this was not really possible on Influx. Perhaps this is an edge case.
(Solution architect at InfluxData here) Out of curiosity...what was the need for this DELETE? Deleting (not dropping) being somewhat of a "second class citizen" was a design choice to make room for more pressing time series needs. In my experience, `DELETE`ing is rarely necessary.
Re: ListenBrainz moves to TimescaleDB
#55I'm in the process of moving from InfluxDB to TimescaleDB myself and can't wait to get rid of the hoops I have to jump through to get InfluxDB to answer some basic questions, mostly stemming from the fact that InfluxQL doesn't support boolean expressions. Something like 'SELECT MAX(temperature) > 10 FROM...' doesn't work.
(InfluxData solution architect here)
Boolean is supported. You query it in the WHERE clause.
Try `SELECT MAX(temperature) FROM ... WHERE temperature > 10`. That said, I'm not sure why you'd run a query like that in InfluxQL as it's the same as `SELECT max(temperature)`. :).
Re: ListenBrainz moves to TimescaleDB
#56I'd love to hear more about how your data ingestion works. I'm thinking of implementing TimescaleDB myself, but in my initial read of the docs, the focus seemed to be managing the database, not getting data into the database...
Re: ListenBrainz moves to TimescaleDB
#57I'm in the process of moving from InfluxDB to TimescaleDB myself and can't wait to get rid of the hoops I have to jump through to get InfluxDB to answer some basic questions, mostly stemming from the fact that InfluxQL doesn't support boolean expressions. Something like 'SELECT MAX(temperature) > 10 FROM...' doesn't work.
(InfluxData solution architect here) Boolean is supported. You query it in the WHERE clause. Try `SELECT MAX(temperature) FROM ... WHERE temperature > 10`. That said, I'm not sure why you'd run a query like that in InfluxQL as it's the same as `SELECT max(temperature)`. :).
It's not the same thing. 'SELECT MAX(temperature) > 10 FROM ...' gets you all the datapoints, with a value of true or false. Moving it to the WHERE clause only gets you the datapoints where temperature is > 10. Yes, you can fill with 0 after the GROUP BY, but if all datapoints are less than 10, you get nothing back from the database. That's confusing: were all datapoints less than 10, or was there nothing registered in the time window I'm querying? Impossible to tell the difference. Not to mention some user interfaces just bailing: no data. I need to show a chart with a timeline of this condition being true or false.
Plus my actual use case is even more complex, not only do I need something like MAX(temperature) > 10, I need (MAX(temperature)) > 10 && (MAX(temperature) - MAX(dewpoint)) > 4.5).
Re: ListenBrainz moves to TimescaleDB
#58I'd love to hear more about how your data ingestion works. I'm thinking of implementing TimescaleDB myself, but in my initial read of the docs, the focus seemed to be managing the database, not getting data into the database...
same way you'd insert data into postgres.
Re: ListenBrainz moves to TimescaleDB
#59I'm in the process of moving from InfluxDB to TimescaleDB myself and can't wait to get rid of the hoops I have to jump through to get InfluxDB to answer some basic questions, mostly stemming from the fact that InfluxQL doesn't support boolean expressions. Something like 'SELECT MAX(temperature) > 10 FROM...' doesn't work.
(InfluxData solution architect here) Boolean is supported. You query it in the WHERE clause. Try `SELECT MAX(temperature) FROM ... WHERE temperature > 10`. That said, I'm not sure why you'd run a query like that in InfluxQL as it's the same as `SELECT max(temperature)`. :).
Blame the customer. Nice solution, architect!
Re: ListenBrainz moves to TimescaleDB
#60Fun fact: TimescaleDB exists because we were using InfluxDB + Postgres for a previous IoT project and also found it unworkable (developer experience, query language, reliability, scalability and performance, operations, etc). We first built TimescaleDB as "Postgres for time-series" for our own needs and then decided to open-source it for others. :-)
Have you been following ZHeap and do you think Timescale will benefit from a storage engine like that (less write amplification)?