Live data from Hacker News

InfluxDB – Open-source distributed time-series, events, and metrics database

influxdb.org

31–40 of 79 posts

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#31
There's a lot to be learned from the non-open source players in this space. Specifically, kdb+ has always provided everything I needed. It's built for HFT, and does millions of data points per second with analytics, with history going back years.

It is, however, rather expensive.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#32
post #2

I'm one of the committers. The project is still early stage. At this point we're looking for feedback on the API, which we're planning on finalizing this month. Would love to hear about anything you'd like changed or added to the API.

I'll second what Chubot said that I'm not liking the SQL much.

The interface I'd like would be something close to numpy (or matlab/r, if those are more familiar). Let me do vectorized operations and write functions in code. Let me load a few different timeseries into a dataframe. Most likely the easiest thing to do would just be to either embed numpy into your engine, or create simple wrappers to load data to it.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#33
post #27

The sandbox appears to be down, but I'm a little concerned about the security. Can database users be created that only have read access?

You can limit read and write access on users. It's documented on this page: http://influxdb.org/docs/api/http.html . We haven't implemented the specific column limits part of that API yet so feedback would be great. Does that take care of the use case you were thinking of? However, security is probably not something to bother with in sandbox since it's not HTTPS. We're looking at it now, and should have it back up in…

Yes, that would handle it. I look forward to playing with the sandbox.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#34
post #31

There's a lot to be learned from the non-open source players in this space. Specifically, kdb+ has always provided everything I needed. It's built for HFT, and does millions of data points per second with analytics, with history going back years. It is, however, rather expensive.

The stuff built for finance is definitely interesting. John and I (two of the people working on this) previously worked at a fintech startup and worked with a closed source time series db called OneTick. It was super fast, but its API work for analytics/events use cases. Great for fast moving market data though.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#35
post #2

I'm one of the committers. The project is still early stage. At this point we're looking for feedback on the API, which we're planning on finalizing this month. Would love to hear about anything you'd like changed or added to the API.

I'll second what Chubot said that I'm not liking the SQL much. The interface I'd like would be something close to numpy (or matlab/r, if those are more familiar). Let me do vectorized operations and write functions in code. Let me load a few different timeseries into a dataframe. Most likely the easiest thing to do would just be to either embed numpy into your engine, or create simple wrappers to load data to it.

The numpy style interface might be interesting. The problem is that it would probably kill performance to marshal into a Python process. Unless there's some way to embed numpy?

Supporting custom functions is definitely something we want to do.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#36
post #34
post #31

There's a lot to be learned from the non-open source players in this space. Specifically, kdb+ has always provided everything I needed. It's built for HFT, and does millions of data points per second with analytics, with history going back years. It is, however, rather expensive.

The stuff built for finance is definitely interesting. John and I (two of the people working on this) previously worked at a fintech startup and worked with a closed source time series db called OneTick. It was super fast, but its API work for analytics/events use cases. Great for fast moving market data though.

I was trying to recall the name of OneTick ... It does work well for market data (especially with corrections and stuff), but is not a convenient general database.

kdb+, on the other hand, is a time series database that works perfectly well as a general database, with a query language that is at the same time infinitely simpler than SQL and yet much faster, more expressive and useful. There's a learning curve, it is steep, but it is well worth it.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#38

Whenever I see these new DBs I promise myself to try them in a side-project but I almost never get around to thinking of one. Can someone be kind enough to hit up a few ideas for side-project where this DB would shine?

I wouldn't say InfluxDB (or Graphite, or whatever) is something you'd develop a side project around (though you could probably implement some novel data visualization), but rather, they provide a backend for your side projects to collectively aggregate metrics.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#39
post #27

Earlier quoted context omitted.

You can limit read and write access on users. It's documented on this page: http://influxdb.org/docs/api/http.html . We haven't implemented the specific column limits part of that API yet so feedback would be great. Does that take care of the use case you were thinking of? However, security is probably not something to bother with in sandbox since it's not HTTPS. We're looking at it now, and should have it back up in…

Yes, that would handle it. I look forward to playing with the sandbox.

http://play.influxdb.org is back up

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#40
Every now and then I see a new opensource distributed and whatnot database pop out, now, I'm totally naive in terms of databases and distributed systems. Do we really need all this Databases? What's special about this one? Can someone give me a summary of the main ones (Mongo, Redis, Rethink, Riak, etc.) ?

Now not discouraging InfluxDB or anything, as a systems programming fan it's great to see more things like this coming, and as a Gopher too.

Post reply on HN