Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The most surprising Unix programs

minnie.tuhs.org

Post reply on HN
Print view View original

81–90 of 182 posts

Re: The most surprising Unix programs

#81

Post by kps »

saati wrote: post #78
atq2119 wrote: post #75

Earlier quoted context omitted.

I'd suggest you go and actually read the article -- it's not about Pascal the language, but about a specific implementation of it.

I actually did, it did not answer my question, that's why I asked here.

    > pascal
    > 
    > The syntax diagnostics from the compiler made by Sue Graham's group at
    > Berkeley were the mmost helpful I have ever seen--and they were generated
    > automatically. At a syntax error the compiler would suggest a token that
    > could be inserted that would allow parsing to proceed further. No attempt
    > was made to explain what was wrong. The compiler taught me Pascal in
    > an evening, with no manual at hand.

Re: The most surprising Unix programs

#82

Post by kmstout »

sl

```

                          (  ) (@@) ( )  (@)  ()    @@    O     @     O     @      O
                     (@@@)
                 (    )
              (@@@@)

            (   )
         ====        ________                ___________
     _D _|  |_______/        \__I_I_____===__|_________|
      |(_)---  |   H\________/ |   |        =|___ ___|      _________________
      /     |  |   H  |  |     |   |         ||_| |_||     _|                \_____A
     |      |  |   H  |__--------------------| [___] |   =|                        |
     | ________|___H__/__|_____/[][]~\_______|       |   -|                        |
     |/ |   |-----------I_____I [][] []  D   |=======|____|________________________|_
   __/ =| o |=-O=====O=====O=====O \ ____Y___________|__|__________________________|_
    |/-=|___|=    ||    ||    ||    |_____/~\___/          |_D__D__D_|  |_D__D__D_|
     \_/      \__/  \__/  \__/  \__/      \_/               \_/   \_/    \_/   \_/
```

Re: The most surprising Unix programs

#83

Post by morelisp »

DagAgren wrote: post #57
mjw1007 wrote: post #24

« Typo was as surprising inside as it was outside. Its similarity measure was based on trigram frequencies, which it counted in a 26x26x26 array. The small memory, which had barely room enough for 1-byte counters, spurred a scheme for squeezing large numbers into small counters. To avoid overflow, counters were updated probabilistically to maintain an estimate of the logarithm of the count. » This sounds like somethi…

Probably not related. Sounds like it's just doing something like replacing `counter++` with `if(rand() % counter == 0) counter++`, so that the counter will increase slower and slower the larger it gets.

Absolutely related! This is essentially the same observation that makes Flajolet-Martin and HyperLogLog work - that when comparing counts, the exact low bits of large numbers "matter less" than the low bits of small numbers, so you can store the logarithm of the count. They differ in how they calculate the "incremental log" without storing the real values, based on what they are counting (high-dimensional events vs. high-cardinality sets).

Re: The most surprising Unix programs

#84

Post by nn3 »

swixmix wrote: post #80
nn3 wrote: post #59

Earlier quoted context omitted.

I doubt the modern GNU or BSD versions of it that you are likely using do. Noone uses the original anymore.

Is scale factor the same as error bounds in http://man.openbsd.org/dc ?

I believe it's just the number of digits when the printing cuts off

Re: The most surprising Unix programs

#85

Post by jawilson »

I've written a few useful scripts that everyone should have.

histogram - simply counts each occurrence of a line and then outputs from highest to lowest. I've implemented this program in several different languages for learning purposes. There are practical tricks that one can apply, such as hashing any line longer than the hash itself.

unique - like uniq but doesn't need to have sorted input! again, one can simply hash very long lines to save memory.

datetimes - looks for numbers that might be dates (seconds or milliseconds in certain reasonable ranges) and adds the human readable version of the date as comments to the end of the line they appear in. This is probably my most used script (I work with protocol buffers that often store dates as int64s).

human - reformats numbers into either powers of 2 or powers of 10. inspired obviously by the -h and -H flags from df.

I'm sure I have a few more but if I can't remember them from the top of my head, then they clearly aren't quite as generally useful.

Anyone else have some useful scripts like these?

Re: The most surprising Unix programs

#86

Post by morelisp »

aasasd wrote: post #45
mjw1007 wrote: post #24

« Typo was as surprising inside as it was outside. Its similarity measure was based on trigram frequencies, which it counted in a 26x26x26 array. The small memory, which had barely room enough for 1-byte counters, spurred a scheme for squeezing large numbers into small counters. To avoid overflow, counters were updated probabilistically to maintain an estimate of the logarithm of the count. » This sounds like somethi…

Seems to me like a variant of a counting Bloom filter.

Nope - counting bloom filters store an exact count of approximate events. This stores approximate counts of exact events.

If you count 5 "abc" and 5 "xyz" in a counting bloom filter, it will always say you had 10 events, but might say they were 10 of the same event.

If you count the same in Morris's structure, it will never confuse the two different sets, but might say one occurred 4 times and the other 8.

Of course, that means you can combine the two, for the benefits and downsides of both - storing very high (and inaccurate) counts of very sparse (and maybe misattributed) event sets.

Re: The most surprising Unix programs

#87

Post by projektfu »

thaumasiotes wrote: post #15
saagarjha wrote: post #7

And people say theoretical computer science isn’t useful in “the real world”… I am curious about this one, though, has anyone used it? > The syntax diagnostics from the compiler made by Sue Graham's group at Berkeley were the mmost helpful I have ever seen--and they were generated automatically. At a syntax error the compiler would suggest a token that could be inserted that would allow parsing to proceed further. No…

> On the surface it sounds a lot like it would produce error messages like “expected ‘;’” that most beginner programmers come to hate Do people really come to hate these? I'd expect the opposite -- that people would start off hating messages like "expected ';'", but fairly quickly become accustomed to what they almost always mean. As long as you can look at the message and have a good idea of what's wrong, it's not a…

They were frustrating in pascal because of its original ;-as-separator philosophy. Lightspeed/Think pascal would give you those errors and guide you to a compilable program, but it was still too easy to make that mistake in the first place.

On the other hand, a missing semicolon in Microsoft C would often give a litany of unrelated errors.

Re: The most surprising Unix programs

#88

Post by nonesuchluck »

jawilson wrote: post #85

I've written a few useful scripts that everyone should have. histogram - simply counts each occurrence of a line and then outputs from highest to lowest. I've implemented this program in several different languages for learning purposes. There are practical tricks that one can apply, such as hashing any line longer than the hash itself. unique - like uniq but doesn't need to have sorted input! again, one can simply h…

I work with csv files a lot. I have a short awk script which truncates/pads each column to a fixed width which I can specify at runtime. It also repeats the top column (headers) every 20 rows in a different ANSI color. I pipe the output to less -SR for interactive use so I can scan delimited data in a scrollable grid, with all columns aligned and labeled.

I understand there's vim plugins for this, but, ehh.

Re: The most surprising Unix programs

#89

Post by hinkley »

tangue wrote: post #29

I didn't knew about typo. One surprising unix program I discovered this year is cal (or ncal). Having a calendar in your terminal is sometimes useful and I wish I knew earlier I could type things like ncal -w 2020

A similarly flavored one I’ve always appreciated is the man page for ascii, which shows the octal, decimal, and hex values for each character in the ASCII space.

Most unixes have one, although the format differs.

Re: The most surprising Unix programs

#90

Post by nonesuchluck »

indigochill wrote: post #31
cmroanirgo wrote: post #18

Earlier quoted context omitted.

Also interesting: > Originators of nearly half the list--pascal, struct, parts, eqn--were women, well beyond women's demographic share of computer science.

In the 40s, computing was seen as primarily women's work (similar to the stereotype of switchboard operators). Into the 60s, women still comprised up to half of the computing workforce. In 84, they peaked at 37%. So demographically speaking, the ratio was not as bad as it is today. (Source: https://en.wikipedia.org/wiki/Women_in_computing )

I feel like the line between computer user/operator and computer programmer used to be fuzzier. I've always wanted to better understand how and where that distinction has shifted over time. To this day, IBM calls mainframe operators, "systems programmers."

I suspect that shifting narrative may tell a chapter of the story of how women were gradually pushed out of our industry.

Post reply on HN