Live data from Hacker News

look(1): lines beginning with given string

linux.die.net

41–47 of 47 posts

Re: look(1): lines beginning with given string

#42
post #23

If you don't have this basic linux utility, you can use grep '^string' file.txt but there's a greater chance of having look installed than grep.

According to Wikipedia, grep was first used in "Version 4 AT&T UNIX" According to my 1984 Eunice manual, Look was part of the 7th edition of Unix.

Re: look(1): lines beginning with given string

#43

Earlier quoted context omitted.

Binary search is only faster if access to the elements is constant-time, which is not true for a file with varying-length records (lines). Line 1000 (counting from 0) could be at offset 2000 if each line is a single character + newline, or it could be at offset 2000000 if each line is 999 characters and a newline, or somewhere in between.

You don't need to know the number of the line you're looking at. You jump to the middle of the file , and you compare the nearest line to the search pattern. Then you recurse on the top or bottom half.

That still requires random seeking; it's not so bad on SSDs, but abysmal on rotating disks, and most of the filesystem and OS caching code is optimised for linear forward reads, so things like prefetching are not going to help at all.

Re: look(1): lines beginning with given string

#45
post #22
post #2

The universe of Unix filters never ceases to turn up things I haven't seen before. Other examples for those inclined: tac, sponge, pv, join, paste.

In the same vein of 'improvements over things everyone assumes you have to do with the more famous tool', we could say grep '^string' : look :: sort -R : shuf

look does a binary search and only works on sorted files, while grep '^string' works on any text file but would be slower.

Re: look(1): lines beginning with given string

#46

Earlier quoted context omitted.

You don't need to know the number of the line you're looking at. You jump to the middle of the file , and you compare the nearest line to the search pattern. Then you recurse on the top or bottom half.

That still requires random seeking; it's not so bad on SSDs, but abysmal on rotating disks, and most of the filesystem and OS caching code is optimised for linear forward reads, so things like prefetching are not going to help at all.

At some point the cost of scanning a long file will overcome the cost of doing log(n) seeks. On short files, it really doesn't matter what you do, searching will be fast regardless.

Re: look(1): lines beginning with given string

#47
post #22

Earlier quoted context omitted.

In the same vein of 'improvements over things everyone assumes you have to do with the more famous tool', we could say grep '^string' : look :: sort -R : shuf

Thanks to @climagic on Twitter, I recently learned that sort -R and shuf are not in fact the same thing. (sort -R doesn't actually shuffle...)

My point there was that besides being a true shuffle, 'shuf' also has an algorithmic advantage when you only need a few lines (similar to how lookup has an advantage when you only need a few).
Post reply on HN