Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

InfluxDB has taken its open-source business to Silicon Valley

technical.ly

Post reply on HN
Print view View original

11–20 of 23 posts

Re: InfluxDB has taken its open-source business to Silicon Valley

#11

Post by pauldix »

shanemhansen wrote: post #10

I really want to love influxdb because I think the world needs a better answer to time series databases that doesn't include java (OpenTSDB, Cassandra). The underlying storage engine (leveldb/rocksdb) is quite solid. I'm currently running 3 nodes in production (for collecting stats) and doing a few thousand writes/s. I'm not using any of the clustering features, I probably won't even evaluate that until 0.9. I'm curr…

Hi Shane, thanks for the encouragement and sorry you're having a few problems with the current 0.8.8 release.

We're heads down working on 0.9.0 and won't be doing any more releases in the 0.8.x line (except to create a migration path to 0.9.0). So we are merging PRs, but only those that apply to 0.9.0 (which includes the admin UI).

Re: InfluxDB has taken its open-source business to Silicon Valley

#12

Post by pauldix »

eik3_de wrote: post #7
pauldix wrote: post #5

Earlier quoted context omitted.

We're busy at work on the production ready version. We're targeting March, but we won't release until it's ready (even if that means slipping our target). Remember, in software development there are lies, damn lies, and delivery estimates. We'll get it out as quickly as possible, sorry for any delays.

can you say something about the upgrade path, will that be possible to do live?

We haven't built the migration tool yet, but mostly likely it will involve running a new version in parallel with the old version while the upgrade runs.

For a guide on how to design your schema for a clean migration see here: http://influxdb.com/docs/v0.8/advanced_topics/schema_design....

Re: InfluxDB has taken its open-source business to Silicon Valley

#14

Post by gtrubetskoy »

Incidently I wrote a blog on it last week: http://grisha.org/blog/2015/03/20/influxdb-data/

The site says "production ready in March" - it seemed to me like there's at least 3 months of work there given that most of the clustering features (e.g. how to rebuild a fialed node, how to expand the cluster, distributed queries) are not there.

My other concern with InfluxDB is that it doesn't follow the fate of FoundationDB - get acquired by a giant corporation and disappear.

Re: InfluxDB has taken its open-source business to Silicon Valley

#15

Post by infinotize »

Another InfluxDB user here. I'd done some evaluations with OpenTSDB and the Graphite suite, and while I had some concerns with stability and maturity the main things that sold me on it were:

* No dependencies. Compare this with setting up HDFS/HBase and Graphite which is a real pain in the neck to manage, especially since my tsdb has to run on an arbitrary machine pool in a sandbox.

* Active development. This is a big one. Releases have been coming steadily and Paul & co. do a good job of having a real roadmap and chipping away at it; this is probably my tipping point over Graphite.

* Clustering. Maybe it's not there yet, but see above. Most tools in this space are not elastic at all.

* Grafana integration - seems like there is a good bit of momentum in that project in general which is promising.

PS Reading this it almost sounds like an ad, no I'm not affiliated with influx.

PPS logfile configuration for rotation/cleanup would be a nice-to-have enhancement ;)

Re: InfluxDB has taken its open-source business to Silicon Valley

#16

Post by pauldix »

gtrubetskoy wrote: post #14

Incidently I wrote a blog on it last week: http://grisha.org/blog/2015/03/20/influxdb-data/ The site says "production ready in March" - it seemed to me like there's at least 3 months of work there given that most of the clustering features (e.g. how to rebuild a fialed node, how to expand the cluster, distributed queries) are not there. My other concern with InfluxDB is that it doesn't follow the fate of FoundationDB…

Hi Grisha, I saw that post, thanks for writing it! The coming features you're talking about are the work we're focused on for finishing this release. The three you mention should drop in an RC within two weeks.

The distributed queries part isn't a large amount of work beccause of how we've designed things. Under the covers the query engine already represents each query as a MapReduce job to be run.

For cluster expansion, work is starting on that today. Again it's just a matter of wiring some things up. Node replacement is also starting today.

We may miss the March goal but it won't be by anything close to 3 months. Glad you're paying attention to the project though :)

For the Foundation problem, I thought they were never open source. Just free for 5 nodes or less, no?

I think the key to avoiding this fate is to build an active community of contributors outside the company. Luckily we have people submitting PRs every week. We'll be trying to document more of the code and make it easier for outsiders to get involved as we go along.

That way if the worst happens, at least the community can fork and keep the project going forward. I'd love nothing more than for Influx to become bigger than this company.

Re: InfluxDB has taken its open-source business to Silicon Valley

#17

Post by pauldix »

infinotize wrote: post #15

Another InfluxDB user here. I'd done some evaluations with OpenTSDB and the Graphite suite, and while I had some concerns with stability and maturity the main things that sold me on it were: * No dependencies. Compare this with setting up HDFS/HBase and Graphite which is a real pain in the neck to manage, especially since my tsdb has to run on an arbitrary machine pool in a sandbox. * Active development. This is a bi…

Thanks, we're working hard on getting the clustering features complete so we have a real answer for HA, failover, and scalability (up to a point based on current design).

For logfile rotation our recommended solution is to use logrotate. We'll be updating the install to include a config. See https://github.com/influxdb/influxdb/issues/1943

Re: InfluxDB has taken its open-source business to Silicon Valley

#19

Post by gtrubetskoy »

pauldix wrote: post #16
gtrubetskoy wrote: post #14

Incidently I wrote a blog on it last week: http://grisha.org/blog/2015/03/20/influxdb-data/ The site says "production ready in March" - it seemed to me like there's at least 3 months of work there given that most of the clustering features (e.g. how to rebuild a fialed node, how to expand the cluster, distributed queries) are not there. My other concern with InfluxDB is that it doesn't follow the fate of FoundationDB…

Hi Grisha, I saw that post, thanks for writing it! The coming features you're talking about are the work we're focused on for finishing this release. The three you mention should drop in an RC within two weeks. The distributed queries part isn't a large amount of work beccause of how we've designed things. Under the covers the query engine already represents each query as a MapReduce job to be run. For cluster expans…

Thanks Paul! So you're saying it's all a SMOP :)

Another thing that I think might be a critical (or at least interesting) characteristic is back-filling optimization, i.e. when you need to load a trillion data points of historical data - this y/t explains it pretty well and talks about how OpenTSDB addresses it: https://www.youtube.com/watch?v=SgD3RD2Shg4

Anyhow - keep up the good work, I very much believe that in the next couple of years "Time Series" is going to become a resume-must-include buzzword :)

Re: InfluxDB has taken its open-source business to Silicon Valley

#20

Post by pauldix »

gtrubetskoy wrote: post #19
pauldix wrote: post #16

Earlier quoted context omitted.

Hi Grisha, I saw that post, thanks for writing it! The coming features you're talking about are the work we're focused on for finishing this release. The three you mention should drop in an RC within two weeks. The distributed queries part isn't a large amount of work beccause of how we've designed things. Under the covers the query engine already represents each query as a MapReduce job to be run. For cluster expans…

Thanks Paul! So you're saying it's all a SMOP :) Another thing that I think might be a critical (or at least interesting) characteristic is back-filling optimization, i.e. when you need to load a trillion data points of historical data - this y/t explains it pretty well and talks about how OpenTSDB addresses it: https://www.youtube.com/watch?v=SgD3RD2Shg4 Anyhow - keep up the good work, I very much believe that in th…

Cool, I'll have to take a look at that talk. We've had people ask about backfilling large amounts of data so it's something we'll have to figure out.
Post reply on HN