This is pretty awesome. I have always thought that "tail -f" was a little limited but still useful. Any how, great post!
Stop using tail -f (mostly)
111–120 of 211 posts
Re: Stop using tail -f (mostly)
#112Earlier quoted context omitted.
Another less-vs-vi question: As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are: - I might write to the log file. That seems like a real worry…
On large files less fires up immediately and vim is quite slow. I think that vim is loading an entire file in the memory while less just loads visible chunk. May be other vi implementations are more efficient with large files. Besides — vim really shouldn't be used on log files. It's editor, not viewer.
> Besides — vim really shouldn't be used on log files. It's editor, not viewer.
I think the two concerns I stated are probably legit, but this one is boring to me. Vim is nicer than less for reading files. I have more navigation commands. I can yank part of the file and save it somewhere else. I can pipe a range of lines through awk/etc. I can switch between files. I can split my screen. Etc. Some of these are probably available in less too (more than more(1) supported in 2001), but I doubt all, and I already know the commands in vim. I'm interested in not clobbering my logs and not crashing the server, but if you tell me vim is an editor not a viewer, I'll ask Why?
Btw re-reading my words I don't mean to sound combative. But the point of my original question was to understand. I've been cargo-culting "use less for logs" for 14 years already.
Re: Stop using tail -f (mostly)
#113Earlier quoted context omitted.
Another less-vs-vi question: As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are: - I might write to the log file. That seems like a real worry…
I think your idea of `vi -R` or `vim -R` is a good idea. `less` uses less memory and may load faster but looking at logs for things like valgrind vim will give you syntax highlighting that I am not sure you can get with `less`.
BTW, starting vim as "view" (through e.g. a symlink) is equivalent to "vim -R", precisely for this use case [0].
[0] http://vimdoc.sourceforge.net/htmldoc/starting.html#view
Re: Stop using tail -f (mostly)
#114Earlier quoted context omitted.
Just the other day I was looking for a way to do this in less: yank 10 lines, open a new file, paste the 10 lines, save the file. Is that possible? My Googling last week didn't find a good solution.
If you wanted to grab 10 lines from /var/log/messages, starting at the first line that has "pjungwlr" in it, you could: $ less /var/log/msgs /pjungwlr^M ^G (note line informational line numbers) v (launch vi) [double check your line #s, etc] :d1,. (delete from 1st line, to current line) 10j (go down 10 lines) :d,$ (delete to EOF) :w my_newfile :q
`v` was the missing key for me. Looks like it even preserves my position in the file. Thanks!
Re: Stop using tail -f (mostly)
#115Earlier quoted context omitted.
Another less-vs-vi question: As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are: - I might write to the log file. That seems like a real worry…
On large files less fires up immediately and vim is quite slow. I think that vim is loading an entire file in the memory while less just loads visible chunk. May be other vi implementations are more efficient with large files. Besides — vim really shouldn't be used on log files. It's editor, not viewer.
And an editor should not be used on log files?
Frequently, frequently I am in the position that I need to manipulate an overly-verbose log file to condense out the information I want. I could spend a half hour concocting wizard-like shell invocations, OR I could do it interactively in five minutes with vi...
Re: Stop using tail -f (mostly)
#116> We all have been there: You are watching a file with tail -f, and then you need to search for something in this file, or just navigate up and down. Now you need to exit tail (or open a new shell), and ack this file or open it with vim to find what you are looking for. After that, you run tail again to continue watching the file. There's no need to do that when you are using less. There's also no need to do that if…
Not if you also need to search things that weren't at the end of the file when you started tailing - which is very common with log files.
Generally you wouldn't want auto updating like the less +F or tail -f if the content you wanted was at the start of the file. However you can still work around those rare instances by specifying the -n flag. eg
tail -n 2000 -f
And the best thing about this method is you can still pipe grep (which you couldn't do with less) so if you are writing out ~2000 lines rapidly enough that the content you want wouldn't be at the bottom, you can manage the text stream more efficiently (ie grep -v out stuff you don't want or only grep in the content you do want)
I should add that I am a fan of less on many occasions, but in this instance the piping ability of tail is a deal breaker when working with larger files.
Re: Stop using tail -f (mostly)
#117Earlier quoted context omitted.
Sure it does. If your log lines are distinct (e.g. they have timestamps or unique IDs) then you can use less's search highlighting to provide a visual marker for a specific line, similar to what you can do by manually inserting a bunch of blank lines on the console. This trick doesn't work if your log file has a bunch of identical lines and you want to keep an eye on their rate, though.
imagine the scenario: "I want to see everything that happens in a single request" Enter method: 1. Press enter a bunch of times 2. Reload browser 3. Press enter a bunch of times and scroll up Your method: 1. Search for last line in output to highlight it? 2. Reload page 3. Try and figure out where stuff starts and ends with loads of visual noise
less method:
1. ma
2. Reload browser
3. 'a
1. ma
2. Reload browser
3. mb
4. 'aRe: Stop using tail -f (mostly)
#118Earlier quoted context omitted.
This has nothing to do with what is being discussed in this subthread.
Sure it does. If your log lines are distinct (e.g. they have timestamps or unique IDs) then you can use less's search highlighting to provide a visual marker for a specific line, similar to what you can do by manually inserting a bunch of blank lines on the console. This trick doesn't work if your log file has a bunch of identical lines and you want to keep an eye on their rate, though.
A bunch of blank lines is a lot more grokable than timestamps lost in a bunch of text.
Re: Stop using tail -f (mostly)
#119Earlier quoted context omitted.
An example of some of the vi commands that less(1) has adopted. You can also: * page forward and backward, including using numeric prefixes to indicate a lineno or indicate "repeat 'n' times". * launch vi if less(1) is working on a file (versus (eg) stdout of some process) * search fwd/backward * start examining a new file w/o leaving less(1) * ...
Another less-vs-vi question: As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are: - I might write to the log file. That seems like a real worry…
I think Edix's answer is the most pragmatic.
Also, someone at work actually crashed a service by opening the log files in vim.
I think the problem was that the memory and processor were already getting stomped on (thus the need to look at the logs) and vim tried to do a lot of fancy stuffs to get more info on the file as a whole.
Re: Stop using tail -f (mostly)
#120For something simple and very lightweight for Windows development, check out BareTail: http://www.baremetalsoft.com/baretail/