Live data from Hacker News

Stop using tail -f (mostly)

brianstorti.com

121–130 of 211 posts

Re: Stop using tail -f (mostly)

#121
post #99

For semi-large file (1/2GB+) less takes seconds if not minutes to count the number of lines (it read the whole file) so you can navigate and search across the file. In that cases tail -f and grep are still way faster than less, though less is a handier tool

You don't have to wait. You can turn that off with '-n'.

When you jump to the end of the file ('>' command) you'll see "Calculating line numbers... (interrupt to abort)" and interrupt (ctrl-C) here will also turn them off.

Re: Stop using tail -f (mostly)

#123
post #99

For semi-large file (1/2GB+) less takes seconds if not minutes to count the number of lines (it read the whole file) so you can navigate and search across the file. In that cases tail -f and grep are still way faster than less, though less is a handier tool

You don't have to wait. You can turn that off with '-n'. When you jump to the end of the file ('>' command) you'll see "Calculating line numbers... (interrupt to abort)" and interrupt (ctrl-C) here will also turn them off.

I didn't know about the -n switch. Usually I need to get to the bottom of the file and find the last occurrence of a string in the logs (G, CTRL+C to stop counting lines, ?string to search) and the slower part is waiting for less to match a string going backwards on the file. Will check if the -n switch solves the problem

Re: Stop using tail -f (mostly)

#124
post #16
post #8

This _almost_ touches on the useful part of this: If you search in less and then put it in follow mode, it continues to highlight the search terms. This is very useful for trapping an exception or webcall in the wild. The downside: less buffers and tail -f prints directly. On a slow printing log this can cause events to show up slowly, and can cause performance issues on a fast moving log. If you're piping through a…

+1 (upvoted). $ less ./somefile /somesearch^M ( (Edit: formatting)

But how do you combine the two on the command line, so less is tailing the file AND highlighting a pattern when launched?

I typically read logfiles with less +F, but it would be nice to create an alias for patterns in specific types of logs, so I don't have to remember them or rely on command history.

Re: Stop using tail -f (mostly)

#126

Earlier quoted context omitted.

Another less-vs-vi question: As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are: - I might write to the log file. That seems like a real worry…

On large files less fires up immediately and vim is quite slow. I think that vim is loading an entire file in the memory while less just loads visible chunk. May be other vi implementations are more efficient with large files. Besides — vim really shouldn't be used on log files. It's editor, not viewer.

With vim, I usually use 'vim -u NONE ' which skips the startup file and makes loading zippy

Re: Stop using tail -f (mostly)

#127

Earlier quoted context omitted.

Another less-vs-vi question: As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are: - I might write to the log file. That seems like a real worry…

On large files less fires up immediately and vim is quite slow. I think that vim is loading an entire file in the memory while less just loads visible chunk. May be other vi implementations are more efficient with large files. Besides — vim really shouldn't be used on log files. It's editor, not viewer.

Vim ships with a command called `view` that will start it in read-only mode, and makes it very much a viewer — not just an editor.

Re: Stop using tail -f (mostly)

#128

Earlier quoted context omitted.

Another less-vs-vi question: As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are: - I might write to the log file. That seems like a real worry…

On large files less fires up immediately and vim is quite slow. I think that vim is loading an entire file in the memory while less just loads visible chunk. May be other vi implementations are more efficient with large files. Besides — vim really shouldn't be used on log files. It's editor, not viewer.

It may also try to do syntax highlighting and other formatting stuff, taking up even more resources and cpu time.

Re: Stop using tail -f (mostly)

#129

Earlier quoted context omitted.

imagine the scenario: "I want to see everything that happens in a single request" Enter method: 1. Press enter a bunch of times 2. Reload browser 3. Press enter a bunch of times and scroll up Your method: 1. Search for last line in output to highlight it? 2. Reload page 3. Try and figure out where stuff starts and ends with loads of visual noise

less method: 1. ma 2. Reload browser 3. 'a less method with two marks: 1. ma 2. Reload browser 3. mb 4. 'a

Unfortunately mark doesn't seem to work while you're following.

Re: Stop using tail -f (mostly)

#130

Earlier quoted context omitted.

Yeah, tail -F is the way to go imo. Also, if you are using a proper ssh client, opening a second ssh connection to manage the reacting to the log files is habit for me at this point. The only time I'm using tail -F is after a new configuration deployment. Otherwise, I'm looking at archival data in ELK.

Have you tried using screen[1]? Might make things a little easier than have two separate SSH connections. [1] https://www.gnu.org/software/screen/

Yes. I use it when I need to resume a SSH session. Given proper configuration, switching SSH tabs/connections is as easy as switching browser tabs...there is no real need for me to use screen for such a purpose.

Thank you for the suggestion tho. :)

Post reply on HN