Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You say “cave dweller debugging”, I say debug logging

sicpers.info

Post reply on HN
Print view View original

91–100 of 139 posts

Re: You say “cave dweller debugging”, I say debug logging

#91

Post by nyanpasu64 »

chrsig wrote: post #82
GnarfGnarf wrote: post #39

There is a time and place to step through the code with a debugger, and there are also situations where a log file can show the "big picture" and display information at a higher level. In the latter case, a log can show in minutes what would take hours to ascertain if stepping through line-by-line. My app formats the trace log as HTML tables, and uses color to signal errors or invalid cases. Makes debugging fun. Also…

> Tip: close & re-open the file for each write, so the last and most important tail end of the log doesn't vanish if the app crashes. It sounds like you're looking for fsync[0] [0] https://man7.org/linux/man-pages/man2/fdatasync.2.html

I think it's more important to fflush after logging data, so userspace buffers don't get lost if the app crashes. f[data]sync is only necessary to protect against system crashes.

Re: You say “cave dweller debugging”, I say debug logging

#92

Post by smrtinsert »

geophile wrote: post #48

I have become less and less enamored of gdb (or IDE equivalent) debugging over my many years in software. Debugging is often very confusing with multithreaded code. It only works in one language at a time. If you're doing Java with JNI for example, the code called through JNI is a black box. If you are doing anything involving multiple processes, or distribution, debuggers are useless. But the real problem, and top-l…

Add thread ids to your logs

Re: You say “cave dweller debugging”, I say debug logging

#93

Post by mping »

geophile wrote: post #77
jayd16 wrote: post #71

Earlier quoted context omitted.

> If you are doing anything involving multiple processes, or distribution, debuggers are useless How so? Can't you just attach to each process? Do you mean at production scale?

Even in development. You could, I suppose, but I'm not sure why you would want to. Going back and forth to two debuggers (at least) seems like torture. And with multiple processes, timing and synchronization issues could make interactive debugging a real nightmare. Why? Why would you do this? Interactive debugging doesn't scale in any dimension.

I used intellij to debug 3 processes over ssh, it worked seamlessly. The problem with logging is that you have to know where to log. Debuggers are excellent for exploration, where logging implies you already put the code. If you forget some log you need to do the whole build/deploy dance, with a debugger I just restart. There are even time travel debuggers that allow replays.

Re: You say “cave dweller debugging”, I say debug logging

#94

Post by throwawayboise »

supermatt wrote: post #2

No doubt there is a place (or indeed a need) for debug (or trace) logging in environments where you can’t simply pause execution. However, in development environments you would be foolish not to use the (vastly superior) tooling available. I think many developers shy away from debuggers simply because they haven't taken the time to understand how to use them effectively, whereas everyone knows how to write a log stat…

> in development environments you would be foolish not to use the (vastly superior) tooling available.

I don't know. Learning and using a debugger adds an additional layer of cognitive load. Instead of thinking about the bug, you're thinking about using the debugger and trying to think about the bug. I have found their value to be limited with the code I normally write.

Re: You say “cave dweller debugging”, I say debug logging

#95

Post by floren »

dgan wrote: post #19

when outputting lots of information (like a running game) i found that a "csv-like" format is actually the best compromise between readability and being structured. Just log "header_1;header2;... value_1;value_2;..." you can still grep it, or redirect to a file and parse later

We've settled on RFC5424 syslog messages with structured data elements (https://datatracker.ietf.org/doc/html/rfc5424#section-6.3). You get a message, plus a bunch of key-value pairs of additional info:

1 2022-05-06T21:01:24.54493Z web1.floren.lan webserver 20000066 webserver/search.go:188 [gw@1 searchid="13094420576" uid="2" user="john" query="tag=gravwell" start="2022-05-06T20:01:24Z" end="2022-05-06T21:01:24Z" background="false"] Search launched

Re: You say “cave dweller debugging”, I say debug logging

#96

Post by geophile »

mping wrote: post #93
geophile wrote: post #77

Earlier quoted context omitted.

Even in development. You could, I suppose, but I'm not sure why you would want to. Going back and forth to two debuggers (at least) seems like torture. And with multiple processes, timing and synchronization issues could make interactive debugging a real nightmare. Why? Why would you do this? Interactive debugging doesn't scale in any dimension.

I used intellij to debug 3 processes over ssh, it worked seamlessly. The problem with logging is that you have to know where to log. Debuggers are excellent for exploration, where logging implies you already put the code. If you forget some log you need to do the whole build/deploy dance, with a debugger I just restart. There are even time travel debuggers that allow replays.

> The problem with logging is that you have to know where to log.

I start with basic lifecycle logging, at an INFO level: The FooBar is created, the FooBar is destroyed, and major states in between. If a FooBar manages a set of things, then I might also add DEBUG level logging for operations on those things. But mostly, detailed logging comes later, as I investigate problems. You really can't anticipate too much logging, because you don't know what's going to need it.

Re: You say “cave dweller debugging”, I say debug logging

#97

Post by User23 »

geophile wrote: post #88
GeorgeTirebiter wrote: post #84

Earlier quoted context omitted.

The lure of gdb is that you do not need to write the logging code. But I am reminded of the diddy: "If a system has a Powerful Debugger, what does that say about the apparent need for a Powerful Debugger?" --- I read that as: if you have a kick-a* debugger, you have a really complicated system. Maybe the answer is to make the system less complicated.

That doesn't make sense to me. Complication is mainly a property of the software you're writing, not the language in which the software is written.

It's true that at the end of the day unnecessary complexity is largely a matter of programmer discipline or lack thereof. However, some languages give the programmer considerably more opportunities for unnecessary complexity than others. And some language/runtime combinations make writing correct software impossible. For example back in the day Perl wasn't properly reentrant, so any signal could possibly lead to undefined behavior. There were ways to mitigate the problem, but not to eliminate it.

Re: You say “cave dweller debugging”, I say debug logging

#98

Post by slaymaker1907 »

The thing I like about debug logging is that it forces you to be more deliberate about what you are doing. It also provides good visualization of the program state over time. Plus, you may discover places that really should have permanent logging and improve the production code.

Re: You say “cave dweller debugging”, I say debug logging

#99

Post by slaymaker1907 »

smrtinsert wrote: post #92
geophile wrote: post #48

I have become less and less enamored of gdb (or IDE equivalent) debugging over my many years in software. Debugging is often very confusing with multithreaded code. It only works in one language at a time. If you're doing Java with JNI for example, the code called through JNI is a black box. If you are doing anything involving multiple processes, or distribution, debuggers are useless. But the real problem, and top-l…

Add thread ids to your logs

Context IDs are often even better since the physical thread executing some high-level operation might change (i.e. if you use a thread pool).

Re: You say “cave dweller debugging”, I say debug logging

#100

Post by vkou »

mjevans wrote: post #86
khalladay wrote: post #75

Earlier quoted context omitted.

Don't data breakpoints help with both of these things?

Only if you already know the value(s) being watched. If you know that you can also add trace level debug statements on those items in the code.

You can, but it's usually easier to set a data breakpoint than to find every place where the data is modified, add sufficiently verbose logging statements there + upstream of there, rebuild, and search through logs.
Post reply on HN