Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Nerdlog – Fast, multi-host TUI log viewer with timeline histogram

github.com

Post reply on HN
Print view View original

11–20 of 58 posts

Re: Show HN: Nerdlog – Fast, multi-host TUI log viewer with timeline histogram

#11

Post by mamcx »

tstack wrote: post #4
Zopieux wrote: post #3

journalctl is mentioned once in the landing page and it seems to imply that journalctl is not supported per se, as logs need to be stored plaintext to legacy syslog (?). I do not want to store plaintext logs and use ancient workarounds like logrotate. journald itself has the built-in ability to receive logs from remote hosts (journald remote & gateway) and search them using --merge.

The article makes it sound like it uses various command-line tools (bash/awk/head/tail) to process the logs. So, I imagine it's not a huge leap to extend support to using journalctl to do that work instead.

One small hitch I found is that this kind of tools are fixes in what to process, so for example I can't use them for structured logging. If it has an escape hatch where I can supply my own pipe (for example `process = 'vector ....'`) then it will be enough.

Re: Show HN: Nerdlog – Fast, multi-host TUI log viewer with timeline histogram

#13

Post by lenova »

dimonomid wrote: post #7
Zopieux wrote: post #3

journalctl is mentioned once in the landing page and it seems to imply that journalctl is not supported per se, as logs need to be stored plaintext to legacy syslog (?). I do not want to store plaintext logs and use ancient workarounds like logrotate. journald itself has the built-in ability to receive logs from remote hosts (journald remote & gateway) and search them using --merge.

That's true, as of today nerdlog doesn't use journalctl, and needs plain log files. There were a few reasons of that, primarily related to the sheer amount of logs that we were dealing with. As mentioned in the article, my original use case was: having a fleet of hosts, each printing pretty sizeable amount of logs, e.g. having more than 1-2GB log file on every host on a single day was pretty common. My biggest proble…

I appreciate this response, and want to say I really like your tool's UI over something like lazyjournal. But like the above commentor, I would love to see journald support as well, just because it's the default these days on the distros I use, and seems like the direction the Linux system industry has headed in.

Re: Show HN: Nerdlog – Fast, multi-host TUI log viewer with timeline histogram

#15

Post by openWrangler »

mdaniel wrote: post #9
tomerbd wrote: post #8

Can i view logs from aws cloudwatch?

You'll go broke doing that, as those API calls are not free. Best to configure cloudwatch to dump into some sane place (S3, SigNoz, whatever) so you only pay the api call once and not every time for interactive viewing I went spelunking around in the codebase trying to get the actual answer to your question and it seems it's like many things: theoretically yes with enough energy expended but by default it seems to be…

Seconded - it sounds like compatibility isn't there yet with AWS, but it would be great if there was a way to use nerdlog with other OSS dashboard tools like Signoz or Coroot like you mentioned. Still a really interesting graylog altnerative.

Re: Show HN: Nerdlog – Fast, multi-host TUI log viewer with timeline histogram

#16

Post by dimonomid »

lenova wrote: post #13
dimonomid wrote: post #7

Earlier quoted context omitted.

That's true, as of today nerdlog doesn't use journalctl, and needs plain log files. There were a few reasons of that, primarily related to the sheer amount of logs that we were dealing with. As mentioned in the article, my original use case was: having a fleet of hosts, each printing pretty sizeable amount of logs, e.g. having more than 1-2GB log file on every host on a single day was pretty common. My biggest proble…

I appreciate this response, and want to say I really like your tool's UI over something like lazyjournal. But like the above commentor, I would love to see journald support as well, just because it's the default these days on the distros I use, and seems like the direction the Linux system industry has headed in.

Thanks for the feedback. I'll see what I can do. But for now, do you think the workaround of having to install rsyslog is not good enough?

Re: Show HN: Nerdlog – Fast, multi-host TUI log viewer with timeline histogram

#17

Post by dimonomid »

whalesalad wrote: post #10
dimonomid wrote: post #7

Earlier quoted context omitted.

That's true, as of today nerdlog doesn't use journalctl, and needs plain log files. There were a few reasons of that, primarily related to the sheer amount of logs that we were dealing with. As mentioned in the article, my original use case was: having a fleet of hosts, each printing pretty sizeable amount of logs, e.g. having more than 1-2GB log file on every host on a single day was pretty common. My biggest proble…

can't you just read from stdin? i use lnav in this way all the time: journalctl -f -u service | lnav this is the ethos of unix tooling

Not really, at least not yet, because nerdlog's focus is very different than that of lnav. There is a section about it in the article as well.

In fact nerdlog doesn't even support anything like -f (realtime following) yet. The idea to implement it did cross my mind, but I never really needed it in practice, so I figured I'd spend my time on something else. Might do it some day if the demand is popular, but still, nerdlog in general is not about just reading a continuous stream of logs; it's rather about being able to query arbitrary time periods from remote logs, and being very fast at that.

Re: Show HN: Nerdlog – Fast, multi-host TUI log viewer with timeline histogram

#19

Post by ryanhecht »

I definitely intend on playing around with this later! I see that [gzipped log archives aren't supported](https://dmitryfrank.com/projects/nerdlog/article#depends_on_...), minimizing the use case for me personally. You've at least thought enough about that to bring it up as a limitation you think people will call attention to -- any plans to eventually support it?
Post reply on HN