ListenBrainz moves to TimescaleDB
blog.metabrainz.org
ListenBrainz moves to TimescaleDB
1–10 of 100 posts
Re: ListenBrainz moves to TimescaleDB
#2+1 on escaping measurement names. Quoting from their source code:
def get_escaped_measurement_name(user_name): # ... comment omitted
return '"\\"{}\\""'.format(user_name.replace('\\', '\\\\\\\\').replace('"', '\\"').replace('\n', '\\\\\\\\n'))Re: ListenBrainz moves to TimescaleDB
#3+1 on escaping measurement names. Quoting from their source code: def get_escaped_measurement_name(user_name): # ... comment omitted return '"\\"{}\\""'.format(user_name.replace('\\', '\\\\\\\\').replace('"', '\\"').replace('\n', '\\\\\\\\n'))
That looks somehow worse than regex in C literal.
Re: ListenBrainz moves to TimescaleDB
#4> If you ever write bad data to a measurement in InfluxDB, there is no way to change it
Correct me if I'm wrong, but I'm fairly certain you can just write data with the same timestamp again and it gets updated. Deleting is also easily possible.
Re: ListenBrainz moves to TimescaleDB
#5+1 on escaping measurement names. Quoting from their source code: def get_escaped_measurement_name(user_name): # ... comment omitted return '"\\"{}\\""'.format(user_name.replace('\\', '\\\\\\\\').replace('"', '\\"').replace('\n', '\\\\\\\\n'))
Do you have a URL? Thanks!
Re: ListenBrainz moves to TimescaleDB
#6+1 on escaping measurement names. Quoting from their source code: def get_escaped_measurement_name(user_name): # ... comment omitted return '"\\"{}\\""'.format(user_name.replace('\\', '\\\\\\\\').replace('"', '\\"').replace('\n', '\\\\\\\\n'))
Do you have a URL? Thanks!
Re: ListenBrainz moves to TimescaleDB
#7+1 on escaping measurement names. Quoting from their source code: def get_escaped_measurement_name(user_name): # ... comment omitted return '"\\"{}\\""'.format(user_name.replace('\\', '\\\\\\\\').replace('"', '\\"').replace('\n', '\\\\\\\\n'))
There are some edge cases in InfluxDB where escaping queries becomes a dark art.
Re: ListenBrainz moves to TimescaleDB
#8> If you ever write bad data to a measurement in InfluxDB, there is no way to change it Correct me if I'm wrong, but I'm fairly certain you can just write data with the same timestamp again and it gets updated. Deleting is also easily possible.
You can't for example delete data from a time period to another, to the best of my knowledge.
Re: ListenBrainz moves to TimescaleDB
#9Earlier quoted context omitted.
Do you have a URL? Thanks!
https://github.com/metabrainz/listenbrainz-server/blob/b0846...
Thanks. I love developer comments -- we've all written things like "this is hacky and ugly" in our code :)
# Note: influxdb-python first replaces each backslash in the username with two backslashes
# and because in influx queries, we have to escape each backslash, overall each backslash
# must be replaced by 4 backslashes. Yes, this is hacky and ugly.Re: ListenBrainz moves to TimescaleDB
#10> If you ever write bad data to a measurement in InfluxDB, there is no way to change it Correct me if I'm wrong, but I'm fairly certain you can just write data with the same timestamp again and it gets updated. Deleting is also easily possible.
ListenBrainz dev here. We wanted stuff like the ability to do stuff like "DELETE FROM measurement where field = blah" and that support didn't exist last time we looked. [0]
Another thing that's not mentioned in the post but was a pain point for us was that it's not easy to query for fields with "null" values [1].
I figure a lot of our pain might be because we're not as good at Influx as we are at PostgreSQL. We've been running MusicBrainz[2] for ~18 years on PostgreSQL, that knowledge will hopefully transfer over a little with Timescale.
[0]: https://github.com/influxdata/influxdb/issues/3210
[1]: https://github.com/influxdata/docs.influxdata.com/issues/717