Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

InfluxDB – Open-source distributed time-series, events, and metrics database

influxdb.org

Post reply on HN
Print view View original

11–20 of 79 posts

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#11

Post by pauldix »

vosper wrote: post #6

This looks like the exact feature-set we need at my company; we're in the middle of moving to Redshift but I'll be keeping an eye on Influx. I know it's early days but I didn't see any information about cluster management - how does one setup an Influx cluster, can it be resized, what kind of hardware does it prefer?

We're building out that portion right now. There will be a web interface for managing the cluster. We'll benchmark it on cloud configs on different sizes with regular spinning disks, EBS, and SSDs.

The goal with the cluster stuff is that it should be possible to add nodes to the cluster, but the storage part of it isn't highly elastic. Meaning, you won't be adding and removing instances from it frequently. So adding nodes will require you to go into the admin interface, activate them, then wait up to half a day for rebalancing to be complete (but the cluster will be available for reads and writes during this time). However, we will be optimizing for the case of replacing a failed or soon to be shut down node.

If you're serious about giving it a try when we have the clustered version available, shoot me an email: paul@pauldix.net. Would definitely like to hear more about your use case.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#12

Post by pauldix »

chubot wrote: post #9
pauldix wrote: post #2

I'm one of the committers. The project is still early stage. At this point we're looking for feedback on the API, which we're planning on finalizing this month. Would love to hear about anything you'd like changed or added to the API.

I wouldn't use this bastardized SQL dialect. SQL comes from relational databases, which comes from relational algebra, which is an exceptionally poor model for time series data. It's going to end up a mess and confuse people. SQL is already a mess by itself. I would simply use functions and operators over time series or data frame types. Perhaps take a look at the R zoo library for examples of more advanced things pe…

We opted for the bastardized SQL because we thought it would be easier to understand. Of course, that may not be the case which is why we'd like to hear what people think of it. We based our decision on people using our API in Errplane and not understanding some parts of it without a lot of additional explanation. One of our users said about one part of it "oh, that's just like group by".

I'm curious, did you find the SQL dialect readable and understandable?

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#13

Post by vasilipupkin »

chubot wrote: post #9
pauldix wrote: post #2

I'm one of the committers. The project is still early stage. At this point we're looking for feedback on the API, which we're planning on finalizing this month. Would love to hear about anything you'd like changed or added to the API.

I wouldn't use this bastardized SQL dialect. SQL comes from relational databases, which comes from relational algebra, which is an exceptionally poor model for time series data. It's going to end up a mess and confuse people. SQL is already a mess by itself. I would simply use functions and operators over time series or data frame types. Perhaps take a look at the R zoo library for examples of more advanced things pe…

Actually, it's nice to have both. you may have something already that is doing a lot of queries and this way you can swap this database in and not have to change the queries much.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#14

Post by pauldix »

genericacct wrote: post #7

sounds interesting; i am currently using hbase for similar purposes. Do "tables" have to be created explicitly, or can I just store a value into a timeseries, and if the ts doesn't exist yet it will be created?

You can just write data in on the fly. Time series get created when you write the first point. You also can create new columns on the fly. And there's no enforcement of a data type across all values for a given column. That's on the user.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#15

Post by vosper »

siliconc0w wrote: post #10

I was actually looking at a bunch of open source time series databases and settled on kairosdb but this looks pretty nice. I think there is a hackernews rule someplace that a more interesting tech alternative shows up right when you decided to go with something else. For reference here is the list I created when researching these: http://opentsdb.net/overview.html Built on HBASE http://www.gocircuit.org/vena.html Bui…

Thanks for sharing all these links! I'm always trying to find new and interesting DB technologies, especially time-series.

Can you share anything of your experiences with kairosdb so far - what's the use case and how has it performed?

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#16

Post by alexk »

pauldix wrote: post #2

I'm one of the committers. The project is still early stage. At this point we're looking for feedback on the API, which we're planning on finalizing this month. Would love to hear about anything you'd like changed or added to the API.

This project looks really, really promising, thanks for working on it!

How robust/scalable in your opinion the backend is at this stage? I'm just trying to set my expectations properly when checking it out.

Thanks, Sasha

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#17

Post by hoopism »

vosper wrote: post #15
siliconc0w wrote: post #10

I was actually looking at a bunch of open source time series databases and settled on kairosdb but this looks pretty nice. I think there is a hackernews rule someplace that a more interesting tech alternative shows up right when you decided to go with something else. For reference here is the list I created when researching these: http://opentsdb.net/overview.html Built on HBASE http://www.gocircuit.org/vena.html Bui…

Thanks for sharing all these links! I'm always trying to find new and interesting DB technologies, especially time-series. Can you share anything of your experiences with kairosdb so far - what's the use case and how has it performed?

Thanks! We also went through a similar assessment and came away with kairosdb being the favorable option. Have not started any integration so early thoughts would be helpful. We def favored cassandra over HBASE and the little work we did with opentsdb went well.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#18

Post by pauldix »

alexk wrote: post #16
pauldix wrote: post #2

I'm one of the committers. The project is still early stage. At this point we're looking for feedback on the API, which we're planning on finalizing this month. Would love to hear about anything you'd like changed or added to the API.

This project looks really, really promising, thanks for working on it! How robust/scalable in your opinion the backend is at this stage? I'm just trying to set my expectations properly when checking it out. Thanks, Sasha

We're writing the clustered portion of it right now. That won't be available until December, but we'll have performance benchmarks on a variety of configurations.

The single node performance at this point for writes is tens of thousands of points per second if batched, and for reads we haven't optimized yet. Queries that only have to go through a few hundred thousand points should return in For now we're focused on creating a developer friendly API and building out clustering.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#20

Post by jwatte »

siliconc0w wrote: post #10

I was actually looking at a bunch of open source time series databases and settled on kairosdb but this looks pretty nice. I think there is a hackernews rule someplace that a more interesting tech alternative shows up right when you decided to go with something else. For reference here is the list I created when researching these: http://opentsdb.net/overview.html Built on HBASE http://www.gocircuit.org/vena.html Bui…

You didn't look at istatd? It does 150000 counters at three different retention intervals every ten seconds, with average, total, standard deviation, mom and max for all metrics.

We looked at other options (including open tsdb and graphite) before building this.

Https://github.com/imvu-open/istatd

Post reply on HN