Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Influxdb made the switch from Go to Rust

old.reddit.com

Post reply on HN
Print view View original

101–110 of 162 posts

Re: Influxdb made the switch from Go to Rust

#101

Post by pauldix »

x86a wrote: post #86
tetha wrote: post #34

Earlier quoted context omitted.

We're migrating off of InfluxDB due to that rollercoaster, honestly. It's hard enough to find time to maintain the monitoring stack at work. Casually dropping "Oh, and now you get to rebuild the entire grafana to change the query language" on that doesn't help. And apparently, version 3 does the same thing, except backwards. Sorry, but at that point, we've decided to rebuild the entire metric visualization once on Ti…

We are influxdb enterprise customers and looking to do the same thing. They've kept their enterprise offering on 1.x, which has kept us mostly happy, but seeing what's going on in their OSS stuff is horrifying and we're looking to avoid the crash and burn at the end of the tunnel.

We announced the availability of the v3 successor to Enterprise v1. It supports the v1 API. We're still building data migration tooling, but if you're interested in testing it out just email support or your sales rep.

Re: Influxdb made the switch from Go to Rust

#102

Post by borninator »

mardifoufs wrote: post #7

I love influx but damn do they like moving (too?) fast and quickly changing stuff. In a way, it's pretty cool since it means that they don't get stuck with bad decisions for backwards compatibility reasons, but it's a bit of a roller coaster for users. Not sure what's the best solution though. Having a "stable" but fundamentally limited product (I guess influxdb v1) or breaking stuff in hopes of ending up with a way…

We moved from Influxdb to Prometheus for this reason. Influxdb is far more powerful, but ain't nobody got time to fix all the graphs in Grafana or learn the very mathematical like QL.

If we had dedicated personell to manage our monitoring we might have stuck with it.

Re: Influxdb made the switch from Go to Rust

#103

Post by louwrentius »

ilyt wrote: post #62
louwrentius wrote: post #38

As a side note, the Flux language that they introduced in v2 never seems to have taken off as there are a few (2) public Grafana dashboards made with it, whereas the older influx language has around 1345 currently. Unfortunately I was stupid enough to build my dashboard on flux, which I’m really sorry to say I dislike quite a bit, while still wanting to be respectful for the people who build the stuff. All that said,…

I tried it and it was more complex to do anything compared to PromQL. Like, it looks super powerful for complex queries I will never need to make...

Yes, and in another HN post the Influx founder stated (me paraphrasing) it’s and it’s all about SQL compatibility.

Re: Influxdb made the switch from Go to Rust

#104

Post by say_it_as_it_is »

pauldix wrote: post #79
d3w4s9 wrote: post #73

This is really surprising for a commercial company. Not about the choice of language itself, but the fact that they prioritize such rewrites over features, similar to the concerns from other commenters. They mention "performance" and "garbage collector" and "error handling" which are almost technical details. A company can make good money for quite a while as long as the performance is "good enough", and usually focu…

In this case, the features we kept getting asked for by our customers necessitated a change in underlying database architecture. I talk about that quite a bit in the reddit thread. I totally agree that a rewrite is risky. It's not something I'd choose to do again, but at the time we didn't really see any way around rewriting the bulk of the database (even if we kept it implemented in Go). Using Rust and the Arrow eco…

What is meant by separating compute from storage? This keeps being mentioned as if it were some new paradigm shift so I assume there's a non-obvious situation.

Re: Influxdb made the switch from Go to Rust

#105

Post by jerf »

pauldix wrote: post #65
techn00 wrote: post #4

tl;dr - No garbage collector - Fearless concurrency (thanks Rust compiler) - Performance - Error handling - Crates - they thought they were gonna use C++ and wanted interop (ended up not using C++?) - ecosystem: Apache Arrow DataFusion - "I thought that if we're going to rewrite most of the database anyway, we might as well do it in the best language choice in 2020" But the real reason might be: "Rust good, Go bad" /…

I still love Go, but I think Rust is a better fit for very performance sensitive systems software like a database.

You can write a database in Go, sure, but if you want to compete at the very top end, you need all the control. Go is nice in the way that Go is nice (if you even agree with that statement) precisely because it doesn't have that control. Unless it is a terminal goal for a database to be in Go simply to be in Go (BoltDB seems to fit into this category) or you don't care to compete at the very top end, I think it's a mistake to even start in Go.

To be honest, most of the "we started in Go and switched to Rust" stories read to me as "you should have always known that you should have started in Rust" (or C++ or something, though I'd choose Rust out of the viable set of languages here too). It's IMHO always been obvious that Go was not really viable, I mean, sure, you can get farther than you could with starting with Python, but, it's not something Go was even trying to solve.

Re: Influxdb made the switch from Go to Rust

#107

Post by [deleted] »

Patrickmi wrote: post #32
Patrickmi wrote: post #16

People are not talking that InfluxData is NO more just a time series database, this isn’t just a language change but feature additions and with the level of massive dependence on C++ libraries it’s pretty foolish to continue using Go

Down-voters, could you please explain your disagreement to the statement

[deleted]

Re: Influxdb made the switch from Go to Rust

#108

Post by d3w4s9 »

voiper1 wrote: post #78
d3w4s9 wrote: post #73

This is really surprising for a commercial company. Not about the choice of language itself, but the fact that they prioritize such rewrites over features, similar to the concerns from other commenters. They mention "performance" and "garbage collector" and "error handling" which are almost technical details. A company can make good money for quite a while as long as the performance is "good enough", and usually focu…

The first comment is from the Cofounder and he lists the _features_ that motivated a rewrite: >Then there's the question of why we did a rewrite at all. We wanted to get at some important requirements: >Unlimited cardinality >Analytics queries against time series at the performance of a columnar DB >Use object store as the durability layer for historical data (i.e. separate compute from storage) >SQL and broader ecos…

Sure, the write definitely could have helped releasing these new features, but the language transition itself takes non-trivial effort and could have been used for adding additional features instead. I haven't seen any evidence that such a language transition is desperately needed. Also I said deprioritize adding features, not stop adding features, mind you. The core question here is what kind of benefit we are seeing by changing the language.

If you are a fan of Rust and want to rewrite all your just because of that and you are the CTO, whatever, what can I say. But the title here is a bit clickbait-y and I really don't see much reflection on the languages themselves and how to balance the business needs for such a transition.

Re: Influxdb made the switch from Go to Rust

#109

Post by atombender »

the_duke wrote: post #64
tetha wrote: post #34

Earlier quoted context omitted.

We're migrating off of InfluxDB due to that rollercoaster, honestly. It's hard enough to find time to maintain the monitoring stack at work. Casually dropping "Oh, and now you get to rebuild the entire grafana to change the query language" on that doesn't help. And apparently, version 3 does the same thing, except backwards. Sorry, but at that point, we've decided to rebuild the entire metric visualization once on Ti…

Fair warning, I had serious scaling issues with Timescale. Solutions like Grafana Mimir, Victoria Metrics, Clickhouse, or yes, the new Influx implementation, are much more scalable and will give you much fewer headaches. ClickhouseDB is realy brilliant, btw, it's a powerhouse. Especially with the fairly recent additions that enable hybrid local + S3 option, pushing older metrics to S3 for cheap long-term storage.

The cloud storage option for CH looks like a game changer for time-based data. Any concerns there about accidentally causing "trashing" when cold data is needed? I believe the MergeTree system works by splitting the table into parts during insertion that then later get merged together, so you have to careful that during merging, only the "hot" data is touched, otherwise you'll start pulling in cloud storage data that's supposed to be effectively read-only.

Re: Influxdb made the switch from Go to Rust

#110

Post by mfreed »

pphysch wrote: post #63
tetha wrote: post #51

Earlier quoted context omitted.

We're much rather looking at reducing the number of technologies we have and exchanging one specialized one-use database for another one doesn't sound great. And sure, TimescaleDB is a hefty extension and will require some work to understand it, but things like HA, backups and overall management of Postgres are pretty much solved for us. And beyond that, TimescaleDB works with a few things we have already. We could m…

That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is d…

Can you say more about "dynamic labels"? Do you just mean that as you evolve, you want to add a new type of "key-value" pair?

The most common approach here is just to store the step of "dynamic" labels in JSON, which can be evolved arbitrarily.

And we've found that this type of data actually compresses quite well in practice.

Also regarding compression, Timescale supports transparent mutability on compressed data, so you can directly INSERT/UPDATE/UPSERT/DELETE into compressed data. Under the covers, it's doing smart optimizations to manage how it asynchronously maps individual mutations into segment level operations to decompress/recompress.

(Timescale cofounder)

Post reply on HN