Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Influxdb made the switch from Go to Rust

old.reddit.com

Post reply on HN
Print view View original

121–130 of 162 posts

Re: Influxdb made the switch from Go to Rust

#121

Post by the_duke »

atombender wrote: post #109
the_duke wrote: post #64

Earlier quoted context omitted.

Fair warning, I had serious scaling issues with Timescale. Solutions like Grafana Mimir, Victoria Metrics, Clickhouse, or yes, the new Influx implementation, are much more scalable and will give you much fewer headaches. ClickhouseDB is realy brilliant, btw, it's a powerhouse. Especially with the fairly recent additions that enable hybrid local + S3 option, pushing older metrics to S3 for cheap long-term storage.

The cloud storage option for CH looks like a game changer for time-based data. Any concerns there about accidentally causing "trashing" when cold data is needed? I believe the MergeTree system works by splitting the table into parts during insertion that then later get merged together, so you have to careful that during merging, only the "hot" data is touched, otherwise you'll start pulling in cloud storage data that…

The merging happens primarily on hot data, I haven't run into any issues there.

But there are lots of approaches, depending on your needs.

You can (should) define a "cache disk" for S3, which will cache up to X Gb locally to avoid trashing.

Another option is is to move data into separate (purely S3 backed) tables after a certain time to avoid accidentally fetching large amounts of data from S3. You can still easily join the data together if needed.

Re: Influxdb made the switch from Go to Rust

#122

Post by threatofrain »

bachmeier wrote: post #50

This was discussed on HN at the time (2020): https://news.ycombinator.com/item?id=25049253 At some point HN is going to have to decide if it's the Rust subreddit or a news site.

Rust is one of the few languages that have a chance to climb out of the hobbyist/academic/ultra-niche range, so it's interesting for me to hear about developments towards the direction of reaching mainstream status. I'd say the same thing about Zig but with less strength.

After that nobody wants to hear about Java.

Re: Influxdb made the switch from Go to Rust

#123

Post by petee »

tsak wrote: post #6
techn00 wrote: post #4

tl;dr - No garbage collector - Fearless concurrency (thanks Rust compiler) - Performance - Error handling - Crates - they thought they were gonna use C++ and wanted interop (ended up not using C++?) - ecosystem: Apache Arrow DataFusion - "I thought that if we're going to rewrite most of the database anyway, we might as well do it in the best language choice in 2020" But the real reason might be: "Rust good, Go bad" /…

Rust gets you on the frontpage!

Yeah, I'm sure they spent 3 years working on it to get on the front page for only a few hours. I'd get this argument maybe if you're talking about a pet project...

Re: Influxdb made the switch from Go to Rust

#124

Post by manicennui »

GauntletWizard wrote: post #5
vb-8448 wrote: post #3

Can someone explain what's the InfluxData's market? Or how they make/plan to make money? If we speak about metrics, Prometheus just win.

I am about the biggest Prometheus stan that you can find, but I will not mock or denigrate influx db. They are a runner-up, but there is room in this market for runner-ups, and their feature set does match some that Prometheus is not good at.

I think you'd find that Prometheus is way behind a number of solutions in the corporate world, including Graphite, DogStatsD, etc. Popularity on HN does not translate to real world popularity.

Re: Influxdb made the switch from Go to Rust

#126

Post by mmcnl »

kungito wrote: post #120
tetha wrote: post #34

Earlier quoted context omitted.

We're migrating off of InfluxDB due to that rollercoaster, honestly. It's hard enough to find time to maintain the monitoring stack at work. Casually dropping "Oh, and now you get to rebuild the entire grafana to change the query language" on that doesn't help. And apparently, version 3 does the same thing, except backwards. Sorry, but at that point, we've decided to rebuild the entire metric visualization once on Ti…

Same here. I joined current company 3 years ago when Influx v2 was coming out. I was supposed to build some analytics on top of it. It was very painful. Flux compiler was often giving internal errors, docs were unclear and it was hard to write any a bit more complicated code. The dash is subpar to graphana but graphana had just raw support. There was no query builder for flux so I tried building dashboards in influxv…

I'm running InfluxDB 1 and 2 in parallel for a personal project, waiting for v2 to get mature and stable enough to replace v1. It's never happening I guess. v1 still works great for me.

Re: Influxdb made the switch from Go to Rust

#127

Post by lispisok »

efxhoy wrote: post #76
PaulWaldman wrote: post #71

Earlier quoted context omitted.

Can you elaborate on what scaling issues you had with Timescale?

I ran into issues with TS too. Main issue I recall now was maintaining a grand total count of events that were already rolled up into daily counts was not fast since it always looked back at all the data. There was no way in the ts patterns to efficiently express it without handrolling something. The issue was that a grand total count can’t be expressed in terms of a hypertable because there’s no time column. It’s fa…

You tried to use a timeseries database without a time column?

Re: Influxdb made the switch from Go to Rust

#128

Post by vb-8448 »

manicennui wrote: post #124
GauntletWizard wrote: post #5

Earlier quoted context omitted.

I am about the biggest Prometheus stan that you can find, but I will not mock or denigrate influx db. They are a runner-up, but there is room in this market for runner-ups, and their feature set does match some that Prometheus is not good at.

I think you'd find that Prometheus is way behind a number of solutions in the corporate world, including Graphite, DogStatsD, etc. Popularity on HN does not translate to real world popularity.

Most of the companies in the corporate world have more than a monitoring system, at least 3 in my experience: sysadmins have at least one, network guys another one and one for legacy platforms (mainframes, as400 and so on).

Plus, in the last years with the rise of Kubernetes, most corporates have at least one cluster with Prometheus monitoring it.

Always, base on my experience, Prometheus is very popular, but the adoption is not so wide due to its `oss nature`: CTOs want someone to blame when things go wrong.

Re: Influxdb made the switch from Go to Rust

#129

Post by za3faran »

tetha wrote: post #34
mardifoufs wrote: post #7

I love influx but damn do they like moving (too?) fast and quickly changing stuff. In a way, it's pretty cool since it means that they don't get stuck with bad decisions for backwards compatibility reasons, but it's a bit of a roller coaster for users. Not sure what's the best solution though. Having a "stable" but fundamentally limited product (I guess influxdb v1) or breaking stuff in hopes of ending up with a way…

We're migrating off of InfluxDB due to that rollercoaster, honestly. It's hard enough to find time to maintain the monitoring stack at work. Casually dropping "Oh, and now you get to rebuild the entire grafana to change the query language" on that doesn't help. And apparently, version 3 does the same thing, except backwards. Sorry, but at that point, we've decided to rebuild the entire metric visualization once on Ti…

What are you migrating to?

Re: Influxdb made the switch from Go to Rust

#130

Post by hagen1778 »

cyber1 wrote: post #111

This is intriguing. Interesting, how does this new Influx engine compete in terms of performance with VictoriaMetrics (which is written in Go and really fast)? They moved their entire stack from Go to Rust, rewrote the system from the ground, and spent a lot of time on it, I guess this is a big cost. Is it worth it?

If I'm reading this [0] right, there will be no a standalone OS influxdb 3.0 version. So there's no point in comparing. I also wonder if it would be allowed to publish benchmarks of ENT version by 3rd-parties.

[0] https://www.influxdata.com/blog/the-plan-for-influxdb-3-0-op...

Post reply on HN