IOx: InfluxData’s New Storage Engine
influxdata.com
IOx: InfluxData’s New Storage Engine
1–10 of 32 posts
Re: IOx: InfluxData’s New Storage Engine
#2>Unbounded cardinality
This has been the largest criticism of InfluxDB in the past. Kudos to the team for acknowledging and solving it!
> IOx supports SQL natively and our cloud customers can connect using Postgres-compatible clients like psql, Grafana’s Postgres data source, and BI tools like PowerBI and Tableau.
Initially InfluxDB had InfluxQL, a SQL like language for querying data. Then they transitioned to Flux, indicating it was superior to writing complex SQL queries over time series data. Now they are highlighting native SQL support. Since this was only announced today, hopefully there will be clear messaging on which query languages will be supported going forward.
It’s also worth noting that queries can also be executed over an HTTP API that platforms like PowerBI can consume today.
>First introduced in 2020 as the open source project InfluxDB IOx, the new storage engine is the product of sustained development by InfluxData and considerable contribution from the InfluxDB open source developer community. Today, the new engine based on IOx arrives first in InfluxData’s multi-tenant InfluxDB Cloud service, available to developers worldwide.
Will this later be available in an OSS package for self-hosting?
Re: IOx: InfluxData’s New Storage Engine
#3>Unbounded cardinality This has been the largest criticism of InfluxDB in the past. Kudos to the team for acknowledging and solving it! > IOx supports SQL natively and our cloud customers can connect using Postgres-compatible clients like psql, Grafana’s Postgres data source, and BI tools like PowerBI and Tableau. Initially InfluxDB had InfluxQL, a SQL like language for querying data. Then they transitioned to Flux,…
The SQL support is likely because they're using DataFusion which already has pretty good SQL support, so it's sort of "free".
https://arrow.apache.org/datafusion/user-guide/sql/sql_statu...
Re: IOx: InfluxData’s New Storage Engine
#4>Unbounded cardinality This has been the largest criticism of InfluxDB in the past. Kudos to the team for acknowledging and solving it! > IOx supports SQL natively and our cloud customers can connect using Postgres-compatible clients like psql, Grafana’s Postgres data source, and BI tools like PowerBI and Tableau. Initially InfluxDB had InfluxQL, a SQL like language for querying data. Then they transitioned to Flux,…
Hi, post author and founder of InfluxDB here. We're supporting Flux (our scripting and query language), InfluxQL (our original SQL like language), and SQL (specifically the Postgres dialect as that's what DataFusion supports). The query engine is DataFusion, which is part of the Apache Arrow project. We contribute to it significantly. So that's what's built in natively. We support Flux and InfluxQL through separate Go processes that use an API to connect to the core DB. Although we're working on native InfluxQL support (it's a Rust based InfluxQL parser that will yield DataFusion logical query plans).
Right now we're focused on our cloud offering. We'll have official open source releases and documentation in the future.
Re: IOx: InfluxData’s New Storage Engine
#5>Unbounded cardinality This has been the largest criticism of InfluxDB in the past. Kudos to the team for acknowledging and solving it! > IOx supports SQL natively and our cloud customers can connect using Postgres-compatible clients like psql, Grafana’s Postgres data source, and BI tools like PowerBI and Tableau. Initially InfluxDB had InfluxQL, a SQL like language for querying data. Then they transitioned to Flux,…
The SQL support is likely because they're using DataFusion which already has pretty good SQL support, so it's sort of "free". https://arrow.apache.org/datafusion/user-guide/sql/sql_statu...
Author here -- it is "free" in the sense that all the effort we put into DataFusion flows directly into IOx. But we do put a lot of effort into DataFusion
Re: IOx: InfluxData’s New Storage Engine
#6I evaluated InfluxDB with the Prometheus Kube Stack chart.
It didn’t really work.
I’m not stupid and I can read docs.
My feeling was it’s like Elastic. Default configuration is so flawed and inscrutable, on purpose, you can forget about using it yourself.
I use Thanos now. At least it fucking works.
I suppose if I need fast queries, I’ll use Postgres.
You guys need to focus on making stuff that works. It’s competitive out there and you don’t have the insights into people who try and wind up hating your guts for being annoying.
Re: IOx: InfluxData’s New Storage Engine
#7Earlier quoted context omitted.
The SQL support is likely because they're using DataFusion which already has pretty good SQL support, so it's sort of "free". https://arrow.apache.org/datafusion/user-guide/sql/sql_statu...
Author here -- it is "free" in the sense that all the effort we put into DataFusion flows directly into IOx. But we do put a lot of effort into DataFusion
I didn't mean to imply it's free as in no effort goes into. Just that the underlying library provides it so it's less effort on top of the already significant effort going into DataFusion itself.
Re: IOx: InfluxData’s New Storage Engine
#8Will InfluxDB IOx eventually replace InfluxDB v2?
Re: IOx: InfluxData’s New Storage Engine
#9Earlier quoted context omitted.
Author here -- it is "free" in the sense that all the effort we put into DataFusion flows directly into IOx. But we do put a lot of effort into DataFusion
I didn't mean to imply it's free as in no effort goes into. Just that the underlying library provides it so it's less effort on top of the already significant effort going into DataFusion itself.
Ah -- got it! This is the beauty of aligning ourselves with technologies like Arrow, Parquet and DataFusion. We can share as well as benefit from the efforts of the broader community
Re: IOx: InfluxData’s New Storage Engine
#10Just want to say congratulations to the team!
2 years and 9,500+ commits is a hell of a feat.