Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

InfluxDB – Open-source distributed time-series, events, and metrics database

influxdb.org

Post reply on HN
Print view View original

61–70 of 79 posts

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#61

Post by yummyfajitas »

waterlion wrote: post #59
yummyfajitas wrote: post #41

Earlier quoted context omitted.

Why would it kill performance? All you need to do is this: double* values = (double*)malloc(sizeof(double)*num_of_values); (If you are unfamiliar with numpy, it's just a python wrapper around raw blocks of memory.)

You still have to get the bytes out of the server and into the Python process.

Assuming Python is not running in the same process as InfluxDB, amend my code snippet:

    double* data = shmget(key, sizeof(double)*num_data_pts, whatever_flag);
But directly embedding a Python interpreter is pretty easy in C, so I imagine it should also be fairly easy in Go as well.

(Of course, given that my code snippets are C, you can probably deduce I've never written any Go, so take my comments on how easy it is with a grain of salt.)

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#62

Post by waterlion »

yummyfajitas wrote: post #61
waterlion wrote: post #59

Earlier quoted context omitted.

You still have to get the bytes out of the server and into the Python process.

Assuming Python is not running in the same process as InfluxDB, amend my code snippet: double* data = shmget(key, sizeof(double)*num_data_pts, whatever_flag); But directly embedding a Python interpreter is pretty easy in C, so I imagine it should also be fairly easy in Go as well. (Of course, given that my code snippets are C, you can probably deduce I've never written any Go, so take my comments on how easy it is wi…

Go interop with C is very doable (although I've never tried it). Not sure about shared memory WRT garbage collection though.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#63

Post by Loic »

nornagon wrote: post #46

This is built as a round-robin DB, yes?

Looking at the code[1], LevelDB is used for the datastore. This is using the LevelDB Go bindings (glue in C).

[1]: https://github.com/influxdb/influxdb/blob/master/src/datasto...

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#64

Post by pauldix »

morgante wrote: post #56

Have you looked into StatsD support? At the very least, a backend for StatsD (to write into InfluxDB) would make adoption a lot easier.

It's definitely high up on the todo list. First finalize the API and release production worthy builds, then all those additional little add ons!

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#65

Post by taf2 »

beagle3 wrote: post #54
taf2 wrote: post #51

Earlier quoted context omitted.

I really like SQL - as I was learning SQL, I thought it was terrible, but now after years of using it really makes getting to data easy, I'm very excited to see you choose SQL instead of JSON or something less query like to query...

... but that's sort of a stockholm syndrome; it's because you know SQL and it makes you feel comfortable, and you can think of worse options (like JSON). But SQL is really a bad option in the moodern world, especially since (I estimate) about 90% of SQL statements are built programmatically; Thus, a query language that is easier to construct from code makes a lot more sense. (And not, that's not JSON - some form of a…

I disagree - it's not stockholm syndrome.. for me it was realizing that SQL actually has solved a lot of common and useful ways of expressing the process of getting to the data you want in the order process you want... simply because we write code to generate SQL does not make it bad... I suspect you also think HTML is bad? Being able to say select x,y,z from table where x=1 is very simple and very clear IMO... I'm just giving you my opinion though... you clearly see things differently :D

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#66

Post by pauldix »

fit2rule wrote: post #58

I don't seem to be able to log in to their playground - anyone else able to register a new account? I just get "invalid username/password" no matter what I enter. Other than that, I look forward to evaluating this .. maybe its a solution for a problem I have recently where I'm collecting massive log files of operation systems, and need to navigate/parse/analyze .. so I guess I import the logs into InfluxDB, and put a…

It's back up now, a deploy had reset the password on it.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#67

Post by pauldix »

andyhmltn wrote: post #60
fit2rule wrote: post #58

I don't seem to be able to log in to their playground - anyone else able to register a new account? I just get "invalid username/password" no matter what I enter. Other than that, I look forward to evaluating this .. maybe its a solution for a problem I have recently where I'm collecting massive log files of operation systems, and need to navigate/parse/analyze .. so I guess I import the logs into InfluxDB, and put a…

I get that. But, what's more worrying is that my password came through in the URL string (a GET request)

We're no longer putting the password in the URL, but play and sandbox aren't over HTTPS. And the password still gets sent. As their name implies, they're for playing around, not for real data. On a real installation you'll want to use SSL. We'll have that built into the prod releases or you can always have your load balancer/proxy handle that for you.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#68

Post by pauldix »

ankit84 wrote: post #52

WOW.. the sandbox show passwords in URL !!! http://sandbox.influxdb.org:9062/#/?username=ankit&password=...

We're no longer putting the password in the URL, but play and sandbox aren't over HTTPS. And the password still gets sent. As their name implies, they're for playing around, not for real data. On a real installation you'll want to use SSL. We'll have that built into the prod releases or you can always have your load balancer/proxy handle that for you.

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#69

Post by jjonte »

pauldix wrote: post #2

I'm one of the committers. The project is still early stage. At this point we're looking for feedback on the API, which we're planning on finalizing this month. Would love to hear about anything you'd like changed or added to the API.

This looks awesome. Interestingly, I started writing something similar to this myself not too long ago. I was using DynamoDB as the backer though.

I think some more aggregate functions would be useful:

- Count (not listed on the Functions page, but used in the query examples?) - Sum - Standard Deviation - First - Last

Re: InfluxDB – Open-source distributed time-series, events, and metrics database

#70

Post by pauldix »

otterley wrote: post #50

I like the fact that data is dimensional. What's the scalability model? It's not clear from the documentation.

We're working on the clustered version now. The short answer is that data points are sharded across the cluster and replicated based on a replication factor on a per database basis. Queries hit the # nodes / RF to answer any given query. So writes scale horizontally and queries balance across the cluster.
Post reply on HN