Multi-node TimescaleDB is now free • Hacker News Board

Re: Multi-node TimescaleDB is now free

#71

Post by akulkarni » Fri, May 22, 2020, 5:21 PM UTC

tpetry wrote: post #57

akulkarni wrote: post #3

Hi, I authored this post, but the credit really goes to the Timescale database team. Multi-node TimescaleDB is the result of a massive amount of engineering effort over two years, as can be seen in this +67,000 line PR: https://github.com/timescale/timescaledb/pull/1923 We're thrilled to make this free so that more developers can use it.

Am i the only one who thinks it is really cool that this will be free but hesitant to use it? I have seen so many distributed data storages fail in a multitude of ways that i just dont trust anyone anymore. After 2-3 years they may have ironed out most bugs and i can evaluate again whether i do trust their implementation to store my data safely.

This is why we built this on top of Postgres. It allows us to inherit Postgres reliability.

While I can't guarantee there won't be bugs ;-), we have found that building on Postgres has enabled a much higher level of reliability than other time-series databases.

Re: Multi-node TimescaleDB is now free

#72

Post by pritambaral » Fri, May 22, 2020, 5:22 PM UTC

xyzzy_plugh wrote: post #29

CameronNemo wrote: post #15

Earlier quoted context omitted.

> I think we're too hung up on OSI open source licenses. I disagree. The free software criteria were defined as they are for a reason. AWS and other cloud vendors are taking advantage, but that is not a good reason to give up on the ideals of the movement. I would be much more comfortable contributing to timescaledb if the license had a date at which it expired to AGPL or some other OSI/DFSG/fourfreedoms license.

In principle, yes I agree, but in practice: no! AGPL is a minefield, it's a shame it is, but it is. The spirit of GPL (and AGPL) was that you could do whatever you want but you have to release the source, but they have continually been handicapped by the evolution of software. Existing licenses are not appropriate for SaaS businesses, they are either too liberal or too restrictive. What you're seeing here is the righ…

> The spirit of free software is very much alive in this decision, methinks.

From the text of the Timescale License, clause 2.1 (d): "A license to prepare, compile, and test Derivative Works of the TSL Licensed Software Source Code solely in a Non-Production Environment". Further along, in section 2.2, the following prohibition is laid out: "You agree not to, except as expressly permitted in Section 2.1(d), prepare Derivative Works of any TSL Licensed Software"

That removes the freedom to run your own modifications in production. Pretty incompatible with the spirit of free software.

Re: Multi-node TimescaleDB is now free

#73

Post by malisper » Fri, May 22, 2020, 5:23 PM UTC

mfreed wrote: post #55

malisper wrote: post #47

> All of these capabilities are being released under the Timescale License, our source-available license that permits broad usage, except for where organizations are providing TimescaleDB-as-a-service. Maybe someone can give clarification on this, but the line between using TimescaleDB to build a product and providing TimescaleDB-as-a-service seems incredibly blurry. If I have a product that in some way let's you que…

Hi @malisper, we totally appreciate concerns around potential uncertainty what a "Value Added Service" means. In fact, when we were looking at Timescale licensing, we took careful look at what a lot of other like company licenses did here (Confluent, Redis, etc), and what later became the Polyform License. Most of them left this definition pretty vague -- because frankly, legal language is never as precise (and perha…

> We went a step further, and tried to define this more precisely about what it means to "offer" TimescaleDB

I don't understand how the bit you posted helped make things more concrete? Section 3.21, the section you referenced lists three conditions, all of which have to be true for your product to be considered "Value Added". I agree the third condition, the one you quoted, is pretty clear. But the second condition, the one I quoted seems really vague so the definition of "Value Added" as a whole becomes really vague.

> What that means is that if you've defined the Heap schema, you have built the indexes and tables, and then are offering a SaaS product on this, you're fine.

FWIW, Heap would automatically create new tables for customers as they sign up and would also automatically create new indexes for customers as needed. For that reason alone, I'm pretty sure Heap would violate the Timescale license.

I agree that it's pretty difficult to be specific about what "value added" means. I'm not sure what the right solution is. I would still want to go over with the Timescale License with an IP lawyer pretty thoroughly before I were to use TimescaleDB.

Re: Multi-node TimescaleDB is now free

#74

Post by heipei » Fri, May 22, 2020, 5:25 PM UTC

Does TimescaleDB support automated downsampling using various functions (min/max/mean/avg) and then during querying automatically picking the correct downsampled data? This is the biggest issue that I and others have with InfluxDB, that it doesn't do that, so the only convenient way to use it is just to expire all data outside the retention policy. Ticket here: https://github.com/influxdata/influxdb/issues/7198

Re: Multi-node TimescaleDB is now free

#75

Post by teraflop » Fri, May 22, 2020, 5:26 PM UTC

thesausageking wrote: post #41

What's the difference between Timescale's license ("free to everyone except cloud providers") and GPLv3?

The differences are pretty substantial.

The GPL puts no restrictions whatsoever on how you can use software that falls under it. Timescale's license, on the other hand, gives you very limited usage rights. You can use unmodified versions of the software, but you can't allow clients to make schema changes, nor can you use it to provide any service that is "primarily [a] database storage or operations product or service".

In addition, Timescale's license is much more restrictive about allowing derivative works. The GPL lets you create modified versions and/or reuse code in other products, no matter how extensive your changes, as long as the results are also GPL-licensed. Timescale's license lets you create modified versions, but you're not allowed to:

* make any changes that bypass "usage restrictions"

* use your changes in production

* distribute your changes in any way, except for assigning all the rights back to Timescale

Re: Multi-node TimescaleDB is now free

#76

Post by mfreed » Fri, May 22, 2020, 5:26 PM UTC

tpetry wrote: post #57

akulkarni wrote: post #3

Hi, I authored this post, but the credit really goes to the Timescale database team. Multi-node TimescaleDB is the result of a massive amount of engineering effort over two years, as can be seen in this +67,000 line PR: https://github.com/timescale/timescaledb/pull/1923 We're thrilled to make this free so that more developers can use it.

Am i the only one who thinks it is really cool that this will be free but hesitant to use it? I have seen so many distributed data storages fail in a multitude of ways that i just dont trust anyone anymore. After 2-3 years they may have ironed out most bugs and i can evaluate again whether i do trust their implementation to store my data safely.

One thing to recognize that the lowest-level storage guts of TimescaleDB is Postgres, which really provides a super-stable, reliable foundation. This obviously doesn't avoid all distributed bugs, but it's a huge benefit.

It's also the case that TimescaleDB provides real benefit and scale even in "single-node" form, which allows for traditional primary/replica replication (for fault tolerance / HA / read replicas and scaling), especially when coupled with our native compression.

So we have users storing 100s of billions of rows in hypertables in the non-distributed version of TimescaleDB as well, including in our fully managed cloud service.

Re: Multi-node TimescaleDB is now free

#77

Post by tylerfontaine » Fri, May 22, 2020, 5:27 PM UTC

psankar wrote: post #50

Are there any tools to migrate from elasticsearch to timescale ? We are considering a switch from our es and are evaluating options. Timeseries is also one of the contenders. We are not looking for text search just some nested queries on a timeseries data.

Disclosure: I work for Timescale, previously worked for Elastic

Pretty much any ETL tool you like could do this, as long as it speaks to elasticsearch and postgres.

Logstash (if you're using the ELK stack) can write to CSV or other formats as well as do any processing, but it doesn't have a JDBC output plugin, so you'd have to ingest with something else. Conversely, fluentd for example can output to Postgres, but doesn't have an elasticsearch input (at least that I could find), so you'd have to export from es with something else.

So it might be a couple of steps, though there are rich clients for most major programming languages for both elasticsearch and postgres. If your schema is fairly simple, this might not be too bad to roll your own.

That said, the hardest part is likely massaging your data, if your elasticserch schema is complex. Because you have to totally denormalize things for es (generally), you might have to unravel some of that going back into a relational database.

Re: Multi-node TimescaleDB is now free

#78

Post by pritambaral » Fri, May 22, 2020, 5:31 PM UTC

tomnipotent wrote: post #45

eitland wrote: post #20

Earlier quoted context omitted.

> That sounds like open source to me, I bet they're just being really conservative about saying "open source" because people there's been so much backlash at MongoDB/Cockroach/etc for similar restrictions. Open Source has a very defined meaning. Please read up on the history of open source and source available licenses before saying it is all the same. We've been defending it against a number of attacks and we will p…

> Please read up on the history of open source Not this patronizing bullshit again. The term "open source" was marketing to take advantage of Netscape releasing their source code. Since since, everyone seems keen on trying to usurp its definition for whatever personal perspective they have that week. To the rest of the community outside of the OSI and FSF (which is 99%+ of the software community), this is a perfectly…

> To the rest of the community outside of the OSI and FSF (which is 99%+ of the software community), this is a perfectly acceptable example of "open source"

Please review clause 2.1 (d) and section 2.2. The freedom to run your own modifications in production is not granted. This is a big deal, and rightly a deal-breaking omission for something to be acceptable as either open source or free (as in freedom).

> The Timescale license checks almost all the boxes ...

_Almost_ all, but not all. Some things work only when all of them work, like freedoms.

Re: Multi-node TimescaleDB is now free

#79

Post by mfreed » Fri, May 22, 2020, 5:31 PM UTC

gigatexal wrote: post #65

atanasovskib wrote: post #52

Earlier quoted context omitted.

The Hypertables and Distributed Hypertables can be used to store any kind of data, but works best as long as it has a monotonously increasing partitioning key (e.g. time), with high ingest load, few data modifications (preferable bulked) The beauty of TimescaleDB being built on Postgres is you can have your regular Postgres tables (OLTP schema) and time-series data (Hypertables) live side by side. Use 1 language (1 m…

This sounds like the perfect fit to a write only event log table we stored in postgres at a previous employer. I pushed to move it to BigQuery but this sounds like it would have been fine.

Here is a community post on storing logs in TimescaleDB:

https://www.komu.engineer/blogs/timescaledb/timescaledb-for-...

Re: Multi-node TimescaleDB is now free

#80

Post by teraflop » Fri, May 22, 2020, 5:32 PM UTC

tomnipotent wrote: post #45

eitland wrote: post #20

Earlier quoted context omitted.

> That sounds like open source to me, I bet they're just being really conservative about saying "open source" because people there's been so much backlash at MongoDB/Cockroach/etc for similar restrictions. Open Source has a very defined meaning. Please read up on the history of open source and source available licenses before saying it is all the same. We've been defending it against a number of attacks and we will p…

> Please read up on the history of open source Not this patronizing bullshit again. The term "open source" was marketing to take advantage of Netscape releasing their source code. Since since, everyone seems keen on trying to usurp its definition for whatever personal perspective they have that week. To the rest of the community outside of the OSI and FSF (which is 99%+ of the software community), this is a perfectly…

> The Timescale license checks almost all the boxes of the OSI definition

By my reading, it fails most of the interesting ones, particularly points 1, 3, 4, 6, and 9, due to the field-of-use restrictions and the prohibition on distributing modified versions.