Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't recommend serious use of an all-in-one local Grafana Loki setup

utcc.utoronto.ca

Post reply on HN
Print view View original

91–100 of 100 posts

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#91

Post by striking »

ilyt wrote: post #89
dpkirchner wrote: post #45

Earlier quoted context omitted.

I don't know. Do you really know what you're giving up by signing these? I'd have to study CLAs and hope that I am interpreting them correctly within our respective jurisdictions, or ask a lawyer. All of this stands in the way of contributing. And this is their decision to make, of course, but it is hostile to would-be contributors.

> Do you really know what you're giving up by signing these? Yes. Everything. That's the point of near-every CLA. So the corporation behind it have option to close the code if they want to, taking your contributions with it. Some corps might not ever do it, but any company is one MBA away from "what we can cut from OSS version and move to enterprise to get more customers"?

I mean, this flatly contradicts this line in the CLA:

> Except for the license granted herein to Grafana Labs and recipients of software distributed by Grafana Labs, You reserve all right, title, and interest in and to Your Contributions. [emphasis mine]

You can always use the version of the code your contributions went into. You still own the code you wrote, and still retain the rest of theirs under its original license. You're just not entitled to future versions of the source like a copyleft license without a CLA would grant.

I understand not being excited to serve corporate interests (that's what a CLA does), but posting intellectually uncurious flamebait as a result makes for boring reading.

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#92

Post by isodude »

valyala wrote: post #87

Grafana Loki stores every data chunk in a separate file [1]. Data chunks are created every 2 hours per every stream [2] which receives at least a single log entry during the last 2 hours. This creates 12 * 30 = 360 chunk files per month per every active stream. If Grafana Loki is used for collecting logs from a thousand of services, and each service generates 10 different log streams, then the number of chunk files c…

As noted [1] they are configurable.

  ingester:
    chunk_retain_period: 30s
    chunk_idle_period: 5m0s
    chunk_block_size: 262144
    chunk_target_size: 1572864
    chunk_encoding: gzip
    max_chunk_age: 2h0m0s
[1] https://grafana.com/docs/loki/latest/operations/storage/file...

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#93

Post by ta1243 »

TheCondor wrote: post #70
Spivak wrote: post #11

It really is amazing at just how bad the various log shipping systems are for the simple use case of "I have logs on some servers and I want them to be over here." We somehow peaked at rsyslog and have been struggling ever since. If you don't follow the one-true-architecture you will get bitten in a million ways. * Log ingestion on the host pulls logs from the application/system/whatever, timestamps the logs itself (…

Here is a question, I mean it honestly, I'm relatively old school and have built many apps using syslog. When it comes to doing log mining, I've got a fairly old school utility belt, I poke around with less, I cat through grep (really ripgrep), I cat through grep and pipe to awk and extract things. Sometimes I fire up cut. I get a ton of milage from sort and uniq. Obviously, I fire up zcat in place of cat when needed…

At FOSDEM the talk on loki was described as a modern version of what you and I do with syslog servers

mine come in to an anycast IP on the network, one file per host, the syslog stamps the receive time at the start in "y-m-d-h-m-s+0000" format, in a y/m/d directory struture, bzip2 after a few days

I have a few scripts which I use to parse the logs and pull reports out (BGP drops/recover times for example), but most of the time tail/cat/sort/grep/cut/etc does the job. Where I differ is I use perl rather than awk.

Sure it doesn't scale to millions of terrabytes a second of minable personal information or whatever the average modern LAMP stack generates, but it currently records about 15G a day from 400 different devices just fine.

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#94

Post by SteveNuts »

medellin wrote: post #36
eriksjolund wrote: post #30

It seems Red Hat believes in Loki Red Hat logging product manager says: "We made the decision to move to Loki and Vector" https://www.youtube.com/watch?v=QZ4Hv85lEJ0&t=938s

They also push systemd…

For as much functionality as Systemd replaced (when you think about it it's an extremely ambitious project), it has been an extremely smooth transition.

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#95

Post by tiwarinitish86 »

the_duke wrote: post #65
tiwarinitish86 wrote: post #34

Parseable is an open source Loki alternative. - Single binary - Written in Rust (lightweight, fast and stable) - Use S3 bucket or Mount point - Visualize with Grafana https://github.com/parseablehq/parseable (founder here)

Can Parseable be sharded, or can you just run a single instance?

Not yet, but we have this on roadmap. We'll add this within a few months.

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#96

Post by tiwarinitish86 »

tomrod wrote: post #52
tiwarinitish86 wrote: post #34

Parseable is an open source Loki alternative. - Single binary - Written in Rust (lightweight, fast and stable) - Use S3 bucket or Mount point - Visualize with Grafana https://github.com/parseablehq/parseable (founder here)

Post a Show HN?

Will do soon!

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#97

Post by zaphar »

edapted wrote: post #75

Ed here, (@slim-bean on github), one of the original authors and lead of the Loki project. The criticisms brought by Chris are valid, and it's good feedback. I just wish it was done so in a more constructive manner. I personally know what Loki is capable of, I've watched it grow from something tiny to something I'm incredibly proud of and in awe of. I run Loki on several Raspberry Pi's ingesting 20-100GB a day, I als…

As a counterpoint to the article, I use loki, grafana, and promtail to manage logs for 5 hosts running a number of different services. I have had absolutely no issues for the last year. I haven't run into any of the issues the OP did. I suppose it's possible that he's running at a scale where certain problems happen that I won't see with my setup.

But honestly I sort of assumed that if you are going to manage hundreds of machines you should probably look at the more scalable configurations anyway. But if you are just doing a handful of machines it's more than capable in the local store only version.

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#98

Post by zaphar »

j0ner wrote: post #39

If you use Loki as the default logging driver with Docker and the Loki container shuts down, the rest of your containers will freeze up. This has been an issue for almost 3 years. https://github.com/grafana/loki/issues/2361

Docker has a long history of issues with the daemon causing containers to freeze when dependencies go away of have issues. It's one of the reasons I prefer other container runtimes.

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#99

Post by ChrisCooney »

sph wrote: post #81
berkle4455 wrote: post #19

Grafana is a terribly ran company that once made a pretty decent OSS dashboard system. Stay away from them for anything but some charts.

I'm building a one-man SaaS and I'm currently shopping for a hosted monitoring service. Got the free trial of Grafana Cloud but I'm not a fan of their products in general. I know Prometheus, I have used DataDog in the past but seems crazy expensive spread thin with 1001 features. What's a good all-in-one cloud monitoring solution that might optionally deal with logs as well? Log monitoring is not actually a priority…

Check out Coralogix! It comes in at a fraction of the price of DataDog (typically a 40-70% cost reduction for migrating customers), no service tiering (so you instantly get support, managed onboarding etc) with a complete, end to end, open source experience (even archived logs are stored in Parquet).

For your use case, Coralogix is awesome. It comes with a managed Grafana instance, if you wish to use that interface, or a custom dashboarding solution, metric driven alarms, release tagging, and much more.

You can find out more at https://coralogix.com/platform/metrics/

Re: I can't recommend serious use of an all-in-one local Grafana Loki setup

#100

Post by ChrisCooney »

In defense of Grafana Loki, it's awesome... if you have a lot of information about your telemetry data up front - i.e volumes, use cases, how you wish to query it etc.

A broader solution like Coralogix (https://coralogix.com/) is more appropriate if you're venturing into the unknown and you need more of a data discovery capability. The problem with most of THESE "all in one" platforms is they tried flexibility and feature breadth for cost. Coralogix is a wee bit different, and it gives users an awesome set of cost optimization tools, as well as a simple pricing model, to offer data discovery without a sudden increase in costs (or worse, surprise overages!).

Post reply on HN