Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Influxdb made the switch from Go to Rust

old.reddit.com

Post reply on HN
Print view View original

91–100 of 162 posts

Re: Influxdb made the switch from Go to Rust

#91

Post by demi56 »

pauldix wrote: post #67
brabel wrote: post #30

Earlier quoted context omitted.

The author was looking for an excuse to use Rust for something since 2018: https://www.influxdata.com/blog/rust-can-be-difficult-to-lea... The rewrite started in 2020... they rationalize now, but it's pretty clear they just really wanted Rust and found the reasons they list later as a post-decision justification. Nothing wrong with that, if you don't mind risking the future of your business on a risky rewrite... thou…

To be fair, we didn't drop everything and do a rewrite. Over the last 3.5 years (the length of time for this project), our total engineering team has ranged from 50-90 people. For the first year it was me and two other people. Then for the 2 years following that it was 9 people total. It wasn't until late last year that we made the decision to go all in on the rewrite and made that the focus of everyone in engineerin…

So what’s gonna happen for v1 and v2 ? cause the number of features you’re offering in v3 is gonna be difficult to maintain compared to v1, v2

Re: Influxdb made the switch from Go to Rust

#94

Post by jimmyspice »

candiddevmike wrote: post #53
mardifoufs wrote: post #7

I love influx but damn do they like moving (too?) fast and quickly changing stuff. In a way, it's pretty cool since it means that they don't get stuck with bad decisions for backwards compatibility reasons, but it's a bit of a roller coaster for users. Not sure what's the best solution though. Having a "stable" but fundamentally limited product (I guess influxdb v1) or breaking stuff in hopes of ending up with a way…

I'm honestly surprised the CTO is still employed.

serious question on behalf of the uniformed, why? I feel that society encourages people to double down and be consistent even if they are armed with better information. we could be better if we didn't have to stick to the one true path, no?

Re: Influxdb made the switch from Go to Rust

#95

Post by pphysch »

hnarn wrote: post #82
pphysch wrote: post #63

Earlier quoted context omitted.

That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is d…

You threw out a database because it didn’t offer compression in your specific use case? That’s it? Just solve compression on the block level, why are you so specific about it happening in the database? It’s probably one of the least interesting feature comparisons when betting on which database to trust.

Not all compression is created equally. Timeseries data has well-defined characteristics that generic block level compression doesn't understand. It's a great example where application-level compression is worthwhile. The proof is in the pudding.

Re: Influxdb made the switch from Go to Rust

#96

Post by PaulWaldman »

pphysch wrote: post #63
tetha wrote: post #51

Earlier quoted context omitted.

We're much rather looking at reducing the number of technologies we have and exchanging one specialized one-use database for another one doesn't sound great. And sure, TimescaleDB is a hefty extension and will require some work to understand it, but things like HA, backups and overall management of Postgres are pretty much solved for us. And beyond that, TimescaleDB works with a few things we have already. We could m…

That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is d…

>My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us.

What is the "dynamic labels" feature? Is it a part of Postgres or Timescale?

Re: Influxdb made the switch from Go to Rust

#97

Post by mardifoufs »

pauldix wrote: post #68
zzzeek wrote: post #9

> So this isn't the approach I'd recommend for this kind of project, but we started fresh from scratch. wow, the from scratch rewrite. I can't even imagine that for a major piece of software

It's definitely not something I'd do again. I'd warn off anyone from taking this approach. But the early results are looking good and I'm very excited about what this enables for the next few years ahead of us.

I'd love a blog or a write up on the rewriting process, lessons learnt etc. It would be awesome!

Re: Influxdb made the switch from Go to Rust

#98

Post by pauldix »

demi56 wrote: post #91
pauldix wrote: post #67

Earlier quoted context omitted.

To be fair, we didn't drop everything and do a rewrite. Over the last 3.5 years (the length of time for this project), our total engineering team has ranged from 50-90 people. For the first year it was me and two other people. Then for the 2 years following that it was 9 people total. It wasn't until late last year that we made the decision to go all in on the rewrite and made that the focus of everyone in engineerin…

So what’s gonna happen for v1 and v2 ? cause the number of features you’re offering in v3 is gonna be difficult to maintain compared to v1, v2

We're going to continue to support our customers on v1 and v2. We're building migration tooling over to v3 for those that want it. We have multiple years of transition ahead of us, we expect to have customers on all 3 versions for quite a while.

Re: Influxdb made the switch from Go to Rust

#99

Post by pphysch »

PaulWaldman wrote: post #96
pphysch wrote: post #63

Earlier quoted context omitted.

That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is d…

>My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. What is the "dynamic labels" feature? Is it a part of Postgres or Timescale?

It's a Timescale feature that allows you to "just insert" metrics without first doing a schema migration to make sure a possible new label schema is supported. I.e. a feature that is taken for granted in native timeseries databases, which don't have to work around RDBMS schemas.

I assume it's doing an automatic ALTER TABLE when necessary, which modifies each row and somehow breaks compression across the sharded tables. Or at least an automatic re-compression would cause massive latency on insert that they wanted to avoid.

Re: Influxdb made the switch from Go to Rust

#100

Post by j16sdiz »

Patrickmi wrote: post #32
Patrickmi wrote: post #16

People are not talking that InfluxData is NO more just a time series database, this isn’t just a language change but feature additions and with the level of massive dependence on C++ libraries it’s pretty foolish to continue using Go

Down-voters, could you please explain your disagreement to the statement

Haven't downvote, but I can't make sense of your comment in the context of this article.

Maybe you have some context or background info I don't have?

Post reply on HN