Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Influxdb made the switch from Go to Rust

old.reddit.com

Post reply on HN
Print view View original

61–70 of 162 posts

Re: Influxdb made the switch from Go to Rust

#61

Post by ilyt »

vb-8448 wrote: post #14
GauntletWizard wrote: post #5

Earlier quoted context omitted.

I am about the biggest Prometheus stan that you can find, but I will not mock or denigrate influx db. They are a runner-up, but there is room in this market for runner-ups, and their feature set does match some that Prometheus is not good at.

Don't get me wrong, I don't denigrate Influx DB. They have some interesting features that prometheus doesn't have, I just don't think that this features can lead to a mass adoption(and money). Let's put this way: is there any killer feature that can ditch most of the Prometheus installations in their favor?

And it *has* to be feature, not just "it's faster", because they are (much) faster time-series databases that support PromQL and are near-drop-in to your infrastructure.

Re: Influxdb made the switch from Go to Rust

#62

Post by ilyt »

louwrentius wrote: post #38

As a side note, the Flux language that they introduced in v2 never seems to have taken off as there are a few (2) public Grafana dashboards made with it, whereas the older influx language has around 1345 currently. Unfortunately I was stupid enough to build my dashboard on flux, which I’m really sorry to say I dislike quite a bit, while still wanting to be respectful for the people who build the stuff. All that said,…

I tried it and it was more complex to do anything compared to PromQL.

Like, it looks super powerful for complex queries I will never need to make...

Re: Influxdb made the switch from Go to Rust

#63

Post by pphysch »

tetha wrote: post #51
pphysch wrote: post #40

Earlier quoted context omitted.

Are you running the "OLAP" TimescaleDB on the same instance as your regular OLTP Postgres? This is the only reason I would entertain TimescaleDB, if I had a strict "1 server" requirement. I briefly deployed and looked into it and there were a lot of footguns like with compression. If not, I would suggest looking at a proper OLAP DB. VictoriaMetrics has been great and was easy to set up.

We're much rather looking at reducing the number of technologies we have and exchanging one specialized one-use database for another one doesn't sound great. And sure, TimescaleDB is a hefty extension and will require some work to understand it, but things like HA, backups and overall management of Postgres are pretty much solved for us. And beyond that, TimescaleDB works with a few things we have already. We could m…

That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution.

My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is designed to achieve great compression while also allowing flexibility of series. Hopefully Timescale will/has fixed that without adding another drastic perf tradeoff, but given how Postgres is architected for OLTP I would be surprised.

Re: Influxdb made the switch from Go to Rust

#64

Post by the_duke »

tetha wrote: post #34
mardifoufs wrote: post #7

I love influx but damn do they like moving (too?) fast and quickly changing stuff. In a way, it's pretty cool since it means that they don't get stuck with bad decisions for backwards compatibility reasons, but it's a bit of a roller coaster for users. Not sure what's the best solution though. Having a "stable" but fundamentally limited product (I guess influxdb v1) or breaking stuff in hopes of ending up with a way…

We're migrating off of InfluxDB due to that rollercoaster, honestly. It's hard enough to find time to maintain the monitoring stack at work. Casually dropping "Oh, and now you get to rebuild the entire grafana to change the query language" on that doesn't help. And apparently, version 3 does the same thing, except backwards. Sorry, but at that point, we've decided to rebuild the entire metric visualization once on Ti…

Fair warning, I had serious scaling issues with Timescale.

Solutions like Grafana Mimir, Victoria Metrics, Clickhouse, or yes, the new Influx implementation, are much more scalable and will give you much fewer headaches.

ClickhouseDB is realy brilliant, btw, it's a powerhouse. Especially with the fairly recent additions that enable hybrid local + S3 option, pushing older metrics to S3 for cheap long-term storage.

Re: Influxdb made the switch from Go to Rust

#65

Post by pauldix »

techn00 wrote: post #4

tl;dr - No garbage collector - Fearless concurrency (thanks Rust compiler) - Performance - Error handling - Crates - they thought they were gonna use C++ and wanted interop (ended up not using C++?) - ecosystem: Apache Arrow DataFusion - "I thought that if we're going to rewrite most of the database anyway, we might as well do it in the best language choice in 2020" But the real reason might be: "Rust good, Go bad" /…

I still love Go, but I think Rust is a better fit for very performance sensitive systems software like a database.

Re: Influxdb made the switch from Go to Rust

#66

Post by howinteresting »

techn00 wrote: post #4

tl;dr - No garbage collector - Fearless concurrency (thanks Rust compiler) - Performance - Error handling - Crates - they thought they were gonna use C++ and wanted interop (ended up not using C++?) - ecosystem: Apache Arrow DataFusion - "I thought that if we're going to rewrite most of the database anyway, we might as well do it in the best language choice in 2020" But the real reason might be: "Rust good, Go bad" /…

While one can bring a lot of nuance to the conversation, most imperative languages do not have the &/&mut separation that Rust does and are therefore bad. &/&mut is basically essential to writing correct code in an imperative language.

Re: Influxdb made the switch from Go to Rust

#67

Post by pauldix »

brabel wrote: post #30
zzzeek wrote: post #9

> So this isn't the approach I'd recommend for this kind of project, but we started fresh from scratch. wow, the from scratch rewrite. I can't even imagine that for a major piece of software

The author was looking for an excuse to use Rust for something since 2018: https://www.influxdata.com/blog/rust-can-be-difficult-to-lea... The rewrite started in 2020... they rationalize now, but it's pretty clear they just really wanted Rust and found the reasons they list later as a post-decision justification. Nothing wrong with that, if you don't mind risking the future of your business on a risky rewrite... thou…

To be fair, we didn't drop everything and do a rewrite. Over the last 3.5 years (the length of time for this project), our total engineering team has ranged from 50-90 people. For the first year it was me and two other people. Then for the 2 years following that it was 9 people total.

It wasn't until late last year that we made the decision to go all in on the rewrite and made that the focus of everyone in engineering. And we did that because we had 4 years of experience trying to get v2 and Flux to be successful, with modest results.

Most of the time we were developing this version, we were spending massively more engineering effort on developing v2 or maintaining v1 for our customers.

Re: Influxdb made the switch from Go to Rust

#68

Post by pauldix »

zzzeek wrote: post #9

> So this isn't the approach I'd recommend for this kind of project, but we started fresh from scratch. wow, the from scratch rewrite. I can't even imagine that for a major piece of software

It's definitely not something I'd do again. I'd warn off anyone from taking this approach. But the early results are looking good and I'm very excited about what this enables for the next few years ahead of us.
Post reply on HN