Live data from Hacker News

Ask HN: Is Knuth's TAOCP worth the time and effort?

news.ycombinator.com

171–180 of 192 posts

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#171

Earlier quoted context omitted.

Wanna sell it?

Weirdly, probably not. I know if I sold it, then I'd suddenly get the urge to read it. Also, it's really heavy (I'd say at least over 1kg), so by the time I sold it at less than the current price because it's no longer new (also mine is 1-4A, the current edition is 1-4B) and paid postage to package and ship it somewhere, it's probably not really worth the effort.

Older editions might actually fetch good prices. That's because newer editions aren't just new material plus minor edits and corrections. They also drop old material.

For example if you are interested in large integer arithmetic the 2nd edition of Volume 2 has some exercises that develop some methods that, if I recall correctly, I found more suited to my needs than the methods in the text when I was working on a large integer library.

The 3rd edition dropped those. I've long forgotten most of what I learned when working on my large integer library, but if I had to learn that subject again I'd probably actually start with the 2nd edition, and then check the 3rd edition to add to that.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#172

    If all you want to do is write CRUD apps, the book is not for you. That being said...

    To quote directly fromt a quote in the book, "Now I see with eye serene, the very pulse of the machine." I finally jumped the gun and started reading it during the summer after my acquiring my bachelors. I finished the first volume during that summer. Here are some notes:

    The Mathematical premliminaries reads like a novel if you have some mathematical maturity, you don't dwell too much on the sections that are over your head, and you treat the exercises as optional. This is because the book can be read from a HS level all the way through the Post-Grad level. Choose your depth.

    The MMIX approach was eye-opening because it opened my eyes to see computers as the 'computing' machines that they are. I indirectly gained a fundamental understanding how everything starts with the CPU. I started seeing all the high-level languages (C++, Python, Rust, Java, etc...) as the abstractions that they are. It is liberating if you have ever felt the uneasiness when you are programming in a high level language when you are nagged by the question "how is the computer doing this?"

    Algorithms and Data Structures( a.k.a Information Structures in vol 1): It is like going to grammar school for algorithms! Have you ever asked yourself "Why are we learning all of these abstract things about algorithms and data structures?" I got my answer from the book. What I mean by this is that after studying Knuth's approach to algorithms, you gain a firm, fundamental, and concrete understanding of their necessity, their role. Afterwards ANYTHING I would encounter on Hackerank or Leetcode became digestible. You gain a "first principles" understanding of algorithms and information structures.

    I believe that the book stands the test of time largely because of these points.

    If you have programmed to the point where you've been exposed to some sorting algorithms and some data structures and you want to continue the road to becoming an expert programmer on a firm footing, it is worth the time and effort. The caveat that I would add is do not wait until you finish all the books before you enter the job market or write software out of your own interest.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#173
post #160

Earlier quoted context omitted.

When Knuth wrote MIX, it was indeed a mix of existing machine languages ( https://retrocomputing.stackexchange.com/a/18176 ) and very similar to them — but an improvement over them for pedagogy, e.g. even something as simple as his abstracting away the detail of whether the machine is a binary or a decimal computer. Of course decimal computers and self-modifying code, and a lot of other things besides, went out of fa…

I think you’re missing the forest for the trees here. Knuth shouldn’t have been using a machine language for his examples, full-stop. Nothing was gained by giving examples in needlessly complex and obtuse assembly language.

It's not clear whether your argument is that:

(1) TAOCP shouldn't have used an assembly language at all,

or

(2) the specific assembly language used in TAOCP (MIX in the 1960s, MMIX since the 1990s) has a bad design.

In the comment I replied to, it seemed the argument was (2) rather than (1), so that's what I replied to.

As for (1), it has been discussed many times before, but apart from the reasons he himself gives (https://cs.stanford.edu/~knuth/mmix.html#:~:text=Why%20have%... and the preface to TAOCP and to the MMIX supplement — “Indeed, Tony Hoare once told me that I should never even think of condensing these books by removing the machine-language parts, because of their educational value”), in short:

• [quantitative] Most of the book is in English pseudocode, dropping down to assembly language only on occasions when the concreteness is relevant. (See also https://news.ycombinator.com/item?id=38444482 above.)

• [historical] The job he was hired by Addison-Wesley to do (you've written a few compilers in machine language — e.g. https://ed-thelen.org/comp-hist/B5000-AlgolRWaychoff.html#7 — write a book for us showing others how to do the same), meant a book for working programmers writing real programs (most of whom would be writing in languages for various different machines): no one in the 1960s was writing production compilers in FORTRAN or ALGOL (or COBOL or BASIC!); I may be wrong but I imagine very few compilers were written in “high-level” languages (PL/I notwithstanding) until C took off in the latter half of the 1980s, and Turbo Pascal was written in assembly even in the 1990s. A language that is similar to what serious readers would use, but abstracts away some of their finicky details and is a bit nicer for teaching in, is exactly what MIX and MMIX are. (Yes MIX was getting seriously out of date by the 1990s; that's valid criticism and why he spent so many years replacing it with MMIX.)

• [cultural] Among people seriously interested in computing/programming/algorithms, on the one hand there are the theorists, most of mainstream academia, who are happy thinking abstractly and asymptotically, not bothering too much with constant factors or machine-level considerations. Knuth, despite being part of academia and having to a great extent spawned this field, is at heart very much on the other side (here not counting the even large number of programmers who simply do it as a job): of hackers with “mechanical sympathy” who simply care about accomplishing a bit more with computers, getting their programs to run faster, and so on. In fact, this subculture is somewhat underground (occasionally surfacing like HAKMEM) and Knuth may be one of its few “respected” representatives in academia. For example: his presentation of circular lists of course includes the XOR trick to save one field (https://en.wikipedia.org/wiki/XOR_linked_list) from the very first edition (Exercise 2.2.4–18). “Packing” tries to put the children of one nodes among the “holes” of others? Of course (Exercise 6.3–4, and how his student Liang and him did hyphenation in TeX, and what he used for the word count program) (unlike CLRS which barely even mentions tries https://news.ycombinator.com/item?id=21924265). He has written (still being published) multiple volumes devoted to “backtracking” programs that many theorists are not interested in because the programs are only applicable in special instances, but Knuth is super excited to be able to count (say) the exact number of knight tours; there are hundreds of pages (each) on binary decision diagrams and on “dancing links”, a low-level way of implementing the “undo” operation in backtracking programs. A lot of what he writes is backed by his actually having written programs and seeing how they run; his volume on SAT solvers gives detailed measurements in “mems”. He has spoken multiple times about how skill in programming is being able to zoom in/out across multiple level of abstractions, knowing how some high-level goal of your program is being achieved by a specific register changing its contents. From this perspective, there is often value in being concrete about what the machine is actually doing.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#174
post #170

Earlier quoted context omitted.

Of course but i am curious to know somebody's personal preference (i have my own collection and preference) particularly when they juxtapose it with TAOCP. It would be nice if you could find the book which seems to have made a significant impression on you and add it here so it is useful to others.

Found the book I had at college, it was: https://dl.acm.org/doi/book/10.5555/156619 https://www.amazon.co.uk/Introduction-Parallel-Computing-Ana...

Nice! I didn't know of this particular book and hence now have to hunt it up.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#175
post #173

Earlier quoted context omitted.

I think you’re missing the forest for the trees here. Knuth shouldn’t have been using a machine language for his examples, full-stop. Nothing was gained by giving examples in needlessly complex and obtuse assembly language.

It's not clear whether your argument is that: (1) TAOCP shouldn't have used an assembly language at all, or (2) the specific assembly language used in TAOCP (MIX in the 1960s, MMIX since the 1990s) has a bad design. In the comment I replied to, it seemed the argument was (2) rather than (1), so that's what I replied to. As for (1), it has been discussed many times before, but apart from the reasons he himself gives (…

> I may be wrong but I imagine very few compilers were written in “high-level” languages (PL/I notwithstanding) until C took off in the latter half of the 1980s,

There was an interesting time in the late 70's when many vendors had their own high-level systems software development programming languages: CDC's Cybil, Sperry-UNIVAC's PLUS, &c., and these were often used for compiler development as a replacement for assembly language. At Cray, the CFT compiler was in assembly language, and CFT77 was in Pascal.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#176
post #168

Earlier quoted context omitted.

Of course but i am curious to know somebody's personal preference (i have my own collection and preference) particularly when they juxtapose it with TAOCP. It would be nice if you could find the book which seems to have made a significant impression on you and add it here so it is useful to others.

Do you have any recommendations of your own that you can share?

Sure. IMO it is important to have an overall picture of the fields of Concurrent/Parallel/Distributed architecture/programming before delving into the details of a language/library implementation. To that end i have found the following books useful;

1) Foundations of Multithreaded, Parallel, and Distributed Programming by Gregory R. Andrews - This is one of my favourites even though old. You get to learn/compare the different paradigms in one single book.

2) Parallel Programming: Concepts and Practice by Bertil Schmidt et al. - This is a more recent book with good explanations/coverage including CUDA.

3) The Art of Multiprocessor Programming by Maurice Herlihy et al. - Well known classic though with a main focus on shared-memory architectures.

Along with the above you also need more detailed language/library specific implementation books;

a) C++ Concurrency in Action by Anthony Williams.

b) Programming with POSIX Threads by David Butenhof.

c) UNIX Systems Programming: Communication, Concurrency and Threads by Kay Robbins and Steve Robbins.

Finally; for studying concurrency at the OS level where software meets hardware, i have found nothing better than;

i) Unix Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers by Curt Schimmel.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#177
post #57

Earlier quoted context omitted.

This is a really nice comment, but I find the comparison at the end so strange. You shouldn't compare reading these books to using social media, a better comparison would be: given a limited amount of time, should you read these books, or some other one?

> This is a really nice comment, but I find the comparison at the end so strange. You shouldn't compare reading these books to using social media, a better comparison would be: given a limited amount of time, should you read these books, or some other one? I don't get what you find strange about comparing reading TAOCP vs reading social media. Your time is limited, and each activity that you choose bears an opportuni…

Reading professional material means you want to improve yourself/your craft/your knowledge, while reading social media is shown to have mostly negative effects, so obviously nobody is going to advocate the latter.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#178

Earlier quoted context omitted.

I've wanted to buy these since the late 90s when I used to enjoy thinking about algorithms in the abstract, but never bought them because at the start of my career I felt quite poor and it didn't seem like a good use of my money. In 2015, I decided I was no longer poor and should buy the box-set as a treat. I set aside time to read a chapter in the first week after it arrived, but realised that it would take years to…

Don't feel too bad -- I strongly believe that 99.99% of the copies of TAOCP are never read beyond the first chapter. I'd love to own them and be the kind of person to read them, but I know I would also struggle to give them the proper attention (not with the hundred other books to read on my shelf).

TAOCP is best referred to, not read in it's entirety.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#179

Does a christian needs to read the bible to a be christian not really. Same with TAOCP and programmers, but the more you read about CS the more complete of a programmer you become. For example i never thought i would use the basic compiler knowledge i gained during my bachelor. But at one of my first consulting gigs it did helped knowing about abstract syntax tree, where you can add and remove nodes. This knowledge h…

Scientists dont need bibles.

He might not need it to do scientific work, but he might want to read the bible, koran etc for his/her spiritual development.

Re: Ask HN: Is Knuth's TAOCP worth the time and effort?

#180
post #30

A little bit of history about the book series may help understand what is in it. In 1956, Knuth graduated high school and entered college, where he encountered a computer for the first time (the IBM 650, to which the series of books is dedicated). He took to programming like a fish to water, and by the time he finished college in 1960, he was a legendary programmer, single-handedly writing several compilers on par wi…

So it's like the principia mathematica equivalent for computer programming

It continues to surprise me that Principia Mathematica (PM) still gets mentioned so regularly in discussions related to computer science topics. As far as I can tell, PM was one of the least influential works in any branch of mathematics or logic or philosophy or computer science. It is must be one of the least-read books (3 volumes, 1994 pages) ever written. PM's sole claim to fame is that it was mentioned in Kurt Goedel's famous paper "Ueber formal unentscheidbare Saetze der Principia Mathematica und verwandter Systeme I". The emphasis strongly is on "verwandter Systeme" - related systems, of which PM is just one example.
Post reply on HN