81–90 of 105 posts
At best it's a NUL separated database structure where the fields are not compressed, which IS greppable just use \x00 in your regexp. At worst he might mean BER, which is an ASN.1 data encoding structure.
This applies more generally than just to logs. I love Unix, but "everything is text" is not actually great. It's better that Unix utils output arbitrary ASCII than that they output arbitrary binary data, but it's obvious why people don't do serious IPC 'the Unix way.' Imagine if instead of exchanging JSON, or ProtoBufs, or whatever, your programs all exchanged text you had to regex into some sort of adhoc structure.…
Text is not synonymous with unstructured.
Change for the sake of change is anti-engineering. It is anti-productive. Your changes must be improvements, and they must not cost more than they save or generate in a reasonable period of time. Many organizations have a fully functional, well-debugged logging infrastructure. The basic design happened years ago, was implemented years ago, and was expected to be useful basically forever. Growth was planned for. Ongoi…
One interesting problem with almost all of the "advantages" of binary logs, is if they're good reasons today, they would have been really awesome reasons in '93 when I started admining my first linux box. The problem with changing the way I've been doing things is I'm already used to the staggering change in performance from a 40 meg non-DMA PATA drive in '93 to dual raid fractional terabyte SSDs. Its really quite a boost in raw power. Yet what I need to log hasn't changed much. So performance gains have been spectacular. So the comparative appeal is incredibly low. It wasn't a "real problem" in '93. Its maybe a thousandth of that problem level today due to technological improvement.
"Hey, if you change everything in your infrastructure, and all your machines, and all your command lines and procedures and ways of thinking to access logs, you MIGHT be 5% more efficient, well, eventually, in the long term" "Eh so what I remember transitioning from spinning rust to SSD and getting 100x the overall system-wide performance a couple years ago, if I want 5% its more economic just to wait for the next tech boost. Also shrinking basically zero load and effort by half is worthless if there's any cost at all, and unfortunately the cost is absolutely huge."
Earlier quoted context omitted.
Cool tech you have there, but I only understood it once I saw the video. You basically have a very fast laser that can do volumetric scans at a high framerate, did I get this right? What do people typically use it for?
> You basically have a very fast laser that > can do volumetric scans at a high framerate, > did I get this right? Sort of. The laser itself is constantly sweeping its wavelength (over a bandwidth of >100nm). Using it as a light source in a interferometer where one leg is reflected by a fixed mirror and the other leg goes into the sample something interesting happens: The interferometric fringes produced for a certai…
Frequency-sweeping... How are you doing that? Is the laser itself able to frequency sweep? Or are you chirping pulses?
Earlier quoted context omitted.
I know it's pretty weak to say so, but that universality is a result of the stage of the migration -- that particular problem will undeniably reduce over time. But you're right that there are some work flows and use cases where it'll bite you big time. A recent migration to systemd on my Debian lvm-on-dmcrypt laptop caused me some hours of pain, so I'm not unsympathetic. Back in the early 90's I was involved in manag…
Thanks for the sympathy ;) > The migration to Windows 95 introduced the same concerns, with similar responses. For me, that is the second big large negative point, apart from the missing universal access (which like you said might get better over time, maybe). This route of having a binary journal with its dedicated journal viewers feels awful lot like being on windows. It's the same negative feeling I get when I get…
Anyway, memory may be failing, but the big problem was one of configuration data (typically small volumes) that used to be kept in .ini (text) files, now being shuffled into the registry. There wasn't a size or complexity issue that drove that move, unlike the challenge of managing and merging many large log files from disparate services on multiple hosts.
In the particular case the toolkit did eventually catch up, but it took a very long time (3-5 years for us, I think, to recover the same level of deployment, configuration, automation). With Journal, in contrast, the toolkit's already there, and ultimately I'm just not convinced that 'I don't have Journal tools installed on this computer' is a persuasive argument against the tool.
I'm not saying there are no compelling arguments, just that one isn't.
I don't have experience with binary logs. I think the fragility of binary logs is not baseless though. AFAIK there was (is?) a problem in systemd's journal where a local corruption of the log could cause a global unavailability of the logged data. People like text logs because local corruptions remain local. Some lines could be gibberish, but that's all. I'm not suggesting that this couldn't be done with binary logs,…
I was thinking exactly the same, once you want to create a binary efficient format which you can query, you then have the same problems as a database. And if there is something we have learned in the history of computing, it's that databases are hard to design properly, and especially from scratch.
The last few entries of a log file before something catastrophic happens are precisely the entries that are the most important to make sure they aren't lost.
One solution to the problem of too much logging data can be what I call "session-based logging" (also known as tracing). You can enable logging on a single session (e.g. a phone call), and for that call you get a lot of logging data, much more than a typical logging system. This obviously only works when you are trouble shooting a specific issue, not when you need to investigate something that happened in the past (w…
And it invites timing-based heisenbugs (enable tracing, problem goes away).
Still a neat approach, however.
On a slightly unrelated note, as a largely amateur Linux user: have people made systems that instead of grepping for info, use machine learning do detect normal patterns of a log file (like what type of events, similar, at different intervals) and report the anomalous output via email or report to an admin? I was thinking this would be a cool area of research for me to try programming again, but it seems so daunting…
I don't know of any systems that do this. As an software developer, I generally use log levels to indicate severity in my logs. So grepping for ERROR should catch anything I had the foresight to log at the ERROR level. Simple heuristics like the number of WARN level logs a minute may be useful. Beyond that it sounds interesting. It may be hard to do in a general way, so focusing on Apache logs or something common may…
[0] https://github.com/etsy/statsd [1] https://github.com/etsy/skyline
A traditional log with a parallel index would be completely backwards compatible, the query tool should work the same way, and you could even treat the index file as a rebuildable cache which can be useful. The interface presented by a specialized tool doesn't have to depend on any specific storage method.
Really, this recent fad of trying to remove old formats in the believe the old format was somehow preventing any new format from working in parallel reminds me of JWZ's recommendations[1] on mbox "summary files" over the complexity of an actual database. Sometimes you can get the features you want without sacrificing performance or compatibility.