The most surprising Unix programs • Hacker News Board

Re: The most surprising Unix programs

#51

Post by Someone » Sun, Mar 15, 2020, 1:03 PM UTC

saagarjha wrote: post #7

And people say theoretical computer science isn’t useful in “the real world”… I am curious about this one, though, has anyone used it? > The syntax diagnostics from the compiler made by Sue Graham's group at Berkeley were the mmost helpful I have ever seen--and they were generated automatically. At a syntax error the compiler would suggest a token that could be inserted that would allow parsing to proceed further. No…

I would guess the most useful part of that is that would allow parsing to proceed further.

I used an Algol compiler that had messages such as:

  Semicolon missing after end (inserted)

  Undeclared identifier ‘foo’ (assumed integer)

Both of these hugely improved the compiler output, as far fewer utterly useless error messages would be produced (yes, I know I didn’t declare ‘foo’. You told me so the previous 12 times I used it)

Parsing valid programs is easy, so are bailing out or going into the woods when encountering invalid syntax. Producing meaningful error messages for line 100 after having seen errors on lines 13, 42 and 78 can be fairly hard.

Re: The most surprising Unix programs

#52

Post by dottedmag » Sun, Mar 15, 2020, 1:06 PM UTC

saagarjha wrote: post #20

thaumasiotes wrote: post #15

Earlier quoted context omitted.

> On the surface it sounds a lot like it would produce error messages like “expected ‘;’” that most beginner programmers come to hate Do people really come to hate these? I'd expect the opposite -- that people would start off hating messages like "expected ';'", but fairly quickly become accustomed to what they almost always mean. As long as you can look at the message and have a good idea of what's wrong, it's not a…

The issue is that the solution to errors like these is often not adding a semicolon, but something else like “the compiler has no idea what is going on in this line” and the actual problem can range from something like an unbalanced delimiter, a misspelled keyword, or even a “syntactically valid-looking” (that a stupid parser, like a code highlighter or formatter, would approve of) but ultimately subtly illegal const…

Pascal grammar is sufficiently different from C-alikes, so it might explain why for Pascal source code these messages are more meaningful.

Re: The most surprising Unix programs

#53

Post by londons_explore » Sun, Mar 15, 2020, 1:08 PM UTC

> The math library for Bob Morris's variable-precision desk calculator used backward error analysis to determine the precision necessary at each step to attain the user-specified precision of the result.

I wonder if compilers could do this today? If you can bound values for floating point operations, you might be able to replace them with fixed point equivalents and get a big speedup. You might also be able to replace them with ints or smaller floats if you can detect the result is rounded to an int.

CPU's also have the possibility to do this since they know (some of) the actual values at runtime, and could take shortcuts with floating point calculation in places where not needed for the result.

Re: The most surprising Unix programs

#54

Post by mci » Sun, Mar 15, 2020, 1:09 PM UTC

mjw1007 wrote: post #24

« Typo was as surprising inside as it was outside. Its similarity measure was based on trigram frequencies, which it counted in a 26x26x26 array. The small memory, which had barely room enough for 1-byte counters, spurred a scheme for squeezing large numbers into small counters. To avoid overflow, counters were updated probabilistically to maintain an estimate of the logarithm of the count. » This sounds like somethi…

You are confusing the approximate counting of distinct elements (done by ingenious algorithms like hyperloglog or Flajolet–Martin) with the approximate counting of each element from a manageable set (done by incrementing the counters less and less often as they grow).

Re: The most surprising Unix programs

#55

Post by sn41 » Sun, Mar 15, 2020, 1:16 PM UTC

amelius wrote: post #48

pjc50 wrote: post #43

Earlier quoted context omitted.

I think the "manual review" phase makes this OK, in a way that simply autobanning Mr Null from your system isn't.

I'm not sure if that would work either. Why not send them an email to verify? Or use captcha tech developed by big companies that actually has some science behind it.

These are high school students in India. Many of them are from rural backgrounds and do not have personal email accounts. From our experience, the forms are many times filled up by employees at cyber cafes who fill their own email addresses and mobile numbers instead of the students.

The only reliable means of communications back to students is by a government approved website, or newspapers, or official media. (The process also has to stand up in court in case some student says that (s)he did not get the communication, and newspaper ads are a documentable evidence of communications on a specified date.)

(BTW, the forms do have captchas, the spurious forms are manually filled in by mischievous/malicious/curious applicants.)

Re: The most surprising Unix programs

#56

Post by butterthebuddha » Sun, Mar 15, 2020, 1:27 PM UTC

aasasd wrote: post #49

tangue wrote: post #29

I didn't knew about typo. One surprising unix program I discovered this year is cal (or ncal). Having a calendar in your terminal is sometimes useful and I wish I knew earlier I could type things like ncal -w 2020

Personally I prefer using the Mac app Alfred for things like that—basically a graphical one-shot terminal with autocompletion for a bunch of frequently-used stuff, in the vein of Spotlight. I whipped me up a script in Lua just so the calendar is faster than a readymade one in Python. However, Alfred needs to be bent somewhat to output content like a calendar in its suggestions.

I would like to invite you to share your setup.

Re: The most surprising Unix programs

#57

Post by DagAgren » Sun, Mar 15, 2020, 1:28 PM UTC

mjw1007 wrote: post #24

« Typo was as surprising inside as it was outside. Its similarity measure was based on trigram frequencies, which it counted in a 26x26x26 array. The small memory, which had barely room enough for 1-byte counters, spurred a scheme for squeezing large numbers into small counters. To avoid overflow, counters were updated probabilistically to maintain an estimate of the logarithm of the count. » This sounds like somethi…

Probably not related.

Sounds like it's just doing something like replacing `counter++` with `if(rand() % counter == 0) counter++`, so that the counter will increase slower and slower the larger it gets.

Re: The most surprising Unix programs

#58

Post by pavlov » Sun, Mar 15, 2020, 1:34 PM UTC

londons_explore wrote: post #53

> The math library for Bob Morris's variable-precision desk calculator used backward error analysis to determine the precision necessary at each step to attain the user-specified precision of the result. I wonder if compilers could do this today? If you can bound values for floating point operations, you might be able to replace them with fixed point equivalents and get a big speedup. You might also be able to replac…

Replacing floats with fixed point isn’t usually a meaningful optimization on modern CPUs. The FPU runs in parallel to the integer units, so you can easily end up idling the FPU while the integer units are too busy doing both the math and the necessary state management (counters, pointer arithmetic etc.)

This could make sense for SIMD however, but then the problem is getting the array data in the right format before the computation — if you’re converting from float to int and back within the loop, it destroys any performance gain.

Re: The most surprising Unix programs

#59

Post by nn3 » Sun, Mar 15, 2020, 1:54 PM UTC

ur-whale wrote: post #9

The fact that dc does (or at least tries to) guarantee error bounds on the result is news to me. And if that does indeed work, that's pretty cool.

I doubt the modern GNU or BSD versions of it that you are likely using do. Noone uses the original anymore.

Re: The most surprising Unix programs

#60

Post by qubex » Sun, Mar 15, 2020, 2:08 PM UTC

davidwihl wrote: post #32

qubex wrote: post #4

Earlier quoted context omitted.

I had to look up ‘Ratfor’ because I’d never heard of it — apparently it’s a FORTRAN preprocessor that added control structures.

One of the books that most influenced my coding was Software Tools by Brian W. Kernighan, P.J. Plauger [0]. Even though I never used Ratfor, the clear descriptions were immensely useful. [0] https://www.goodreads.com/book/show/515603.Software_Tools

I have that book. I have read snippets of it. Evidently I should read all the way through it.