CERN swaps out databases to feed its petabyte-a-day habit
theregister.com
CERN swaps out databases to feed its petabyte-a-day habit
1–10 of 57 posts
Re: CERN swaps out databases to feed its petabyte-a-day habit
#2[deleted]
Re: CERN swaps out databases to feed its petabyte-a-day habit
#3tl,dr:
Speaking to The Register, Roman Khavronenko, co-founder of VictoriaMetrics, said the previous system had experienced problems with high cardinality, which refers to the level of repeated values – and high churn data – where applications can be redeployed multiple times over new instances.
Implementing VictoriaMetrics as backend storage for Prometheus, the CMS monitoring team progressed to using the solution as front-end storage to replace InfluxDB and Prometheus, helping remove cardinality issues, the company said in a statement.
Re: CERN swaps out databases to feed its petabyte-a-day habit
#4Over 24hr period its more then 11 Gigabytes/second or rounding to 100 gbps. Those shards must be pretty crazy
Re: CERN swaps out databases to feed its petabyte-a-day habit
#5That’s one hell of an endorsement. Marketing team won the jackpot.
Re: CERN swaps out databases to feed its petabyte-a-day habit
#6I also dropped InfluxDB at work due to its terrible performance. VictoriaMetrics is great
I was using Promscale (TimescaleDB) but they EOL'd Promscale which forced us to Victoria. But either way both of these are much faster than Influx
Don't get fooled into the latest InfluxDB rewrite. I think the latest is cloud hosted only too? So stupid
Re: CERN swaps out databases to feed its petabyte-a-day habit
#7This is nothing compared to what dragnet surveillance has to deal with.
Re: CERN swaps out databases to feed its petabyte-a-day habit
#8Over 24hr period its more then 11 Gigabytes/second or rounding to 100 gbps. Those shards must be pretty crazy
The headline is about the data processed on their compute, the amount of data in the monitoring system is considerably smaller (but still not small data):
> But Brij Kishor Jashal, a scientist in the CMS collaboration, told The Register that his team were currently aggregating 30 terabytes over a 30-day period to monitor their computing infrastructure performance.
So 1 TB / day, that's about 10 MB/s.
Re: CERN swaps out databases to feed its petabyte-a-day habit
#9At the end of the article it says
"InfluxDB said in March this year it had solved the cardinality issue with a new IOx storage engine."
Does this mean that in the end it wasn't really necessary to switch to VictoriaMetrics' offering?
Re: CERN swaps out databases to feed its petabyte-a-day habit
#10I really like VictoriaMetrics's architecture
vmagent takes care of all the pesky edge things like emulating prometheus config parsing and various scraping bits. It also does buffering in case you lose network connection for a while, and accept vast spread of different protocols
vminsert/vmselect scale separately from eachother and your queries don't bother your ingest all that much.
vmstorage does just that, storage. Only thing that bothers me (compared to say, Elasticsearch), is that data can't migrate between nodes so you can't "just" start a new one and drain an old one, but a tiny bit ops work in rare cases is IMO price worth paying for straightforwardness of the stack..
PromQL compatibility is also great, tools like Grafana "just work" without anyone having to write support for it.
We started migrating from InfluxDB at work, and on my private stuff I already did. Soo much less memory usage too.