51–60 of 105 posts
Earlier quoted context omitted.
Could you not view the journal on another system? I'm curious why: journalctl -D / / /var/log/journal wouldn't work in this case
You assume the other system is a linux box with systemd installed. That may be true, or may not be true at all. :)
> Embedded systems don't have the resources! > ... > I'd still use a binary log storage, because > I find that more efficient to write and parse, > but the indexing part is useless in this case. This is yet again a case of a programmer completely misjudging how an actual implementation will perform in the real world. When I wrote the logging system for this thing http://optores.com/index.php/products/1-1310nm-mhz-fdm…
Earlier quoted context omitted.
(Background: I'm not a journal apologist. For a fact I'm finding it challenging to make the mental shifts required to become adept with this new suite of system tools on my myriad Debian boxen.) > That's a lot more information than you could get from a binary log without any tools. Arguably you need a tool to get the information you showed above - a single line from an apache log. The tool may have been grep, cat, vi…
It is not only different. It is also less universal. With a text based logging system, I can take the usb stick with the system that does not boot on my headless homeserver to any computer and read the logs there. I could even boot the original linux system on that server, running a really old kernel and practically no userland tools, and read them there. Cause that server was using journald, that was not possible. S…
I'm sorry, but I don't find the "but I can view text on a machine from the last century" argument convincing. We're not in the past century, and when doing forensics, we usually do that on a reasonable machine, where all the tools we need are available. Otherwise its an exercise in futility.
Yes, grepping logs is terrible if "you have 100Gb of logs a day". I'm not sure why the author is thinking his use case is anything near the norm or why he's shocked in most use cases people prefer text files. I'm also not getting why he just doesn't use scripts to parse the logs and insert them into a database at that point. Why use some ad-hoc logging binary format if you're doing complex queries that SQL would be b…
Logging format and log storage format are two very different things.
Also, I'm not shocked people prefer text files. I'm shocked why they're so much against binary log storage. There's an important distinction between the two: you can prefer text, if that fits your case better, without hating on binary storage.
Earlier quoted context omitted.
The author shows a use case for both a small and a large logging system. The use case is complex queries which spans multiple applications and don't need regex ninja skills but sensible queries.
he does not. His small logging system is not small at all, it spans multiple systems and has requirements that are not at all typical for small systems.
Earlier quoted context omitted.
It is not only different. It is also less universal. With a text based logging system, I can take the usb stick with the system that does not boot on my headless homeserver to any computer and read the logs there. I could even boot the original linux system on that server, running a really old kernel and practically no userland tools, and read them there. Cause that server was using journald, that was not possible. S…
Bringing a few extra tools for forensics should not be a problem. You bring grep, strings, less, and a bunch more to read text logs. Why not bring one more to read the binary dump too? I'm sorry, but I don't find the "but I can view text on a machine from the last century" argument convincing. We're not in the past century, and when doing forensics, we usually do that on a reasonable machine, where all the tools we n…
I think is it more telling about lack of organization rather than wrong technical choice, but sometime you have to deal with legacy systems and it is good to be able to rely on something as universal as text.
> Embedded systems don't have the resources! > ... > I'd still use a binary log storage, because > I find that more efficient to write and parse, > but the indexing part is useless in this case. This is yet again a case of a programmer completely misjudging how an actual implementation will perform in the real world. When I wrote the logging system for this thing http://optores.com/index.php/products/1-1310nm-mhz-fdm…
Again, transport and storage are different. While I prefer binary storage, most of my transports are text (at least in large part, some binary wrapping may be present here and there).
Earlier quoted context omitted.
It is not only different. It is also less universal. With a text based logging system, I can take the usb stick with the system that does not boot on my headless homeserver to any computer and read the logs there. I could even boot the original linux system on that server, running a really old kernel and practically no userland tools, and read them there. Cause that server was using journald, that was not possible. S…
Bringing a few extra tools for forensics should not be a problem. You bring grep, strings, less, and a bunch more to read text logs. Why not bring one more to read the binary dump too? I'm sorry, but I don't find the "but I can view text on a machine from the last century" argument convincing. We're not in the past century, and when doing forensics, we usually do that on a reasonable machine, where all the tools we n…
For one, because it is not packaged for my distribution. For two, because I get exactly nothing in return. All binary logs do for me is forcing me to use an additional tool.
> I'm sorry, but I don't find the "but I can view text on a machine from the last century" argument convincing.
POGO-E02. I really don't know how old this is, but it has USB-2 and I bought it 2 years ago, though it was marked as classic then. Maybe 2009?
> and when doing forensics, we usually do that on a reasonable machine, where all the tools we need are available
I'm normally doing that at my own environment, with the tools I am used to, and on my machine. Nothing of that includes a binary log viewer.
People don't want it because it's binary, not because you can't grep it. * you need to use a new proprietary tool to interact with them * all scripts relating to logs are now broken * binary logs are easy to corrupt, e.g. if they didn't get closed properly. >You can have a binary index and text logs too! / You can. But what's the point? The point is having human-readable logs without having to use a proprietary piece…
And again, there is no need for proprietary tools at all. Everything I want to do is achievable with free software - so much so, that I use only such software in all my systems.
As for compressing - yeah, no. Please try compressing 100Gb of data and tell me the performance cost is nonexistent.
As for LogStash & ES: Guess what: their storage is binary.
Also note that my article explicitly said that the Journal is unfit for my use cases.