Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

InfluxDB 1.0 GA Released: A Retrospective and What’s Next

influxdata.com

Post reply on HN
Print view View original

61–70 of 83 posts

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#61

Post by leventov »

vegabook wrote: post #55
zenlot wrote: post #24

While talking about InfluxDB people should not forget about their beta release of clustering to attract more users and then making it just enterprise. And no, it's not an argument that this is the only way to get paid in open source. Users should be careful while adopting InfluxDB as Influxdata does not clearly elaborate their plans on the product.

This is becoming problematic across the open source DB world. Witness the graph capabilities of Datastax Enterprise on top of Cassandra, and Riak TS also took quite a while to open source the "TS" version, hoping I guess to monetize without having to open source. This makes me worried about adopting in case I later get gotcha'd. I am really looking forward to a truly open source time series database. It's not that I…

What about Druid? It is largely supported by Metamarkets and Yahoo, companies which are not trying to monetise the db itself.

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#62

Post by caniszczyk »

alexandre_m wrote: post #42
pauldix wrote: post #34

Earlier quoted context omitted.

It's worth noting that Prometheus isn't distributed either. So OSS InfluxDB or that are the same in terms of distributed capabilities.

Prometheus can be scaled with sharding and there are different solutions coming out to address this, like a proxy layer for queries. See the PromCon 2016 videos for latest news on this.

The video is here: https://www.youtube.com/watch?list=PLoz-W_CUquUlCq-Q0hy53Tol...

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#63

Post by nwmcsween »

As per the article the issue influx tries to solve does so by trying to jam non-ts data into a ts database. You gain much more by simplifying things with say a constant increment mirror mmap file with another simple KV that maps offsets for tags and such. I would bet this would be immensely faster except for contrived update-tag-on-every-entry.

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#64

Post by jrv »

pauldix wrote: post #34
deadbunny wrote: post #33

Earlier quoted context omitted.

Yeah, that's kinda a dick move. I was waiting for 1.0 before giving it another real go (after their decision to rewrite everything for 0.9). Looks like I'll be checking out something else now, maybe Prometheus[1] or DalmatinerDB[2] will fit the bill. 1. https://prometheus.io/ 2. https://dalmatiner.io/

It's worth noting that Prometheus isn't distributed either. So OSS InfluxDB or that are the same in terms of distributed capabilities.

While Prometheus's built-in local storage is not distributed, we are in the process of implementing generic interfaces to allow Prometheus to talk to decoupled distributed, horizontally scalable, durable, remote storage (or even other non-storage systems like queues etc.). There will be several implementations sitting behind this interface in the end (see already Vulcan from DO and Frankenstein from Weaveworks, both experimental right now), depending on what you'd rather run and what fits your use case best.

Most importantly, Prometheus will never decide to exclude a feature for commercial reasons. As it's a 100% open-source and independent project not controlled by any single company, we don't have those kinds of incentives. We just want to build the best monitoring system. Note that we still say "no" to a lot of features, but that's only for technical reasons, to avoid bloat, or to keep the project maintainable.

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#65

Post by dismantlethesun »

jnordwick wrote: post #60
vegabook wrote: post #55

Earlier quoted context omitted.

This is becoming problematic across the open source DB world. Witness the graph capabilities of Datastax Enterprise on top of Cassandra, and Riak TS also took quite a while to open source the "TS" version, hoping I guess to monetize without having to open source. This makes me worried about adopting in case I later get gotcha'd. I am really looking forward to a truly open source time series database. It's not that I…

Why does the open sources world struggle with timeseries / tick databases so much? I'm a very big KDB fan, but I thought there would be some competition from the open source people at some point, but it seems like every attempt fails. KDB does so well because of its simplicity. Can the OS people not do simple (this is a possible argument), or is it that, as you point out, whenever something is about to be released in…

It seems to me that specialised tools will usually be released as a product, because they're only for a specialised niche.

Relational databases are treated like they can store any kind of data, and for small data sets, it really doesn't matter if your data-model does not fit the relational type.

Once you have a larger data set, you're no longer a general purpose user.

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#66

Post by Rapzid »

We are using influxDB to calculate SLO performance. Currently, among other things, we process about 30M ELB logs entries into influxdb per day; it handles this easily of course. Here are some musings for those interested based on 0.9:

Wins:

* The new storage engine is very, very cool. Would love to work on this thing. It's fast and space efficient.

* Built in support for time bucketing GROUP's

* Grafana integration is pretty good

* Writes come back after the data is stored; makes it easy to create durable, idempotent processing pipelines.

Woahs

* Unable to combine measurements in the same query; needs ETL with continuous queries or external tools

* No support for subqueries; more ETL

* Stream processing is a little lacking -> can't group on values and high cardinality tags make the DB explode; high cardinality is being worked on but IDK how high it will go, plus I mean the storage engine serves up streams of time-sorted data so samza that stuff up.

* Random crashes but the DB gets along fine when it comes back up

* Compactions use LOTS of RAM. Supposedly this can be tweaked and has been improved for 1.0

* Backfill queries with lots of points seem to use a crazy amount of RAM when bucketing on narrow time windows

Overall it's chugging along quite well. Most of the query limitations we are able to solve with a combination of continuous queries and AWS lambda functions kicked off by CloudWatch Events.

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#67

Post by Rapzid »

We are using influxDB to calculate SLO performance. Currently, among other things, we process about 30M ELB logs entries into influxdb per day; it handles this easily of course. Here are some musings for those interested based on 0.9:

Wins:

* The new storage engine is very, very cool. Would love to work on this thing. It's fast and space efficient.

* Built in support for time bucketing GROUP's

* Grafana integration is pretty good

* Writes come back after the data is stored; makes it easy to create durable, idempotent processing pipelines.

Woahs

* Unable to combine measurements in the same query; needs ETL with continuous queries or external tools

* No support for subqueries; more ETL

* Stream processing is a little lacking -> can't group on values and high cardinality tags make the DB explode; high cardinality is being worked on but IDK how high it will go, plus I mean the storage engine serves up streams of time-sorted data so samza that stuff up.

* Random crashes but the DB gets along fine when it comes back up

* Compactions use LOTS of RAM. Supposedly this can be tweaked and has been improved for 1.0

* Backfill queries with lots of points seem to use a crazy amount of RAM when bucketing on narrow time windows

Overall it's chugging along quite well. Most of the query limitations we are able to solve with a combination of continuous queries and AWS lambda functions kicked off by CloudWatch Events.

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#68

Post by bogomipz »

vegabook wrote: post #55
zenlot wrote: post #24

While talking about InfluxDB people should not forget about their beta release of clustering to attract more users and then making it just enterprise. And no, it's not an argument that this is the only way to get paid in open source. Users should be careful while adopting InfluxDB as Influxdata does not clearly elaborate their plans on the product.

This is becoming problematic across the open source DB world. Witness the graph capabilities of Datastax Enterprise on top of Cassandra, and Riak TS also took quite a while to open source the "TS" version, hoping I guess to monetize without having to open source. This makes me worried about adopting in case I later get gotcha'd. I am really looking forward to a truly open source time series database. It's not that I…

What is KX KDB?

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#69

Post by jimmcslim »

jnordwick wrote: post #60
vegabook wrote: post #55

Earlier quoted context omitted.

This is becoming problematic across the open source DB world. Witness the graph capabilities of Datastax Enterprise on top of Cassandra, and Riak TS also took quite a while to open source the "TS" version, hoping I guess to monetize without having to open source. This makes me worried about adopting in case I later get gotcha'd. I am really looking forward to a truly open source time series database. It's not that I…

Why does the open sources world struggle with timeseries / tick databases so much? I'm a very big KDB fan, but I thought there would be some competition from the open source people at some point, but it seems like every attempt fails. KDB does so well because of its simplicity. Can the OS people not do simple (this is a possible argument), or is it that, as you point out, whenever something is about to be released in…

KX seems to have a story that is very convincing specifically for financial markets applications.

All the other OSS TSDBs seem to have very good stories for storing server statistics and web clicks, or IoT data; but there are few stories, case studies, best practices for using these in financial applications?

Re: InfluxDB 1.0 GA Released: A Retrospective and What’s Next

#70

Post by snowwindwaves »

jnordwick wrote: post #60
vegabook wrote: post #55

Earlier quoted context omitted.

This is becoming problematic across the open source DB world. Witness the graph capabilities of Datastax Enterprise on top of Cassandra, and Riak TS also took quite a while to open source the "TS" version, hoping I guess to monetize without having to open source. This makes me worried about adopting in case I later get gotcha'd. I am really looking forward to a truly open source time series database. It's not that I…

Why does the open sources world struggle with timeseries / tick databases so much? I'm a very big KDB fan, but I thought there would be some competition from the open source people at some point, but it seems like every attempt fails. KDB does so well because of its simplicity. Can the OS people not do simple (this is a possible argument), or is it that, as you point out, whenever something is about to be released in…

Here [0] is a good blog post and spreadsheet comparing the various open source time series databases.

[0]: https://blog.dataloop.io/top10-open-source-time-series-datab...

Post reply on HN