Live data from Hacker News

The Most Important Algorithms

risc.jku.at

31–40 of 50 posts

Re: The Most Important Algorithms

#31
post #28

This reads like a textbook, i.e. it's algorithms that are highly useful to CS professors and students but not necessarily to practitioners. If I had to pick a top-10 list based on my professional experience, it would be: 1. Tree traversal. This comes up all the time , from directory walks to DOM traversal to code analysis to folds over abstract data types. It's also something that you need to know and can't always re…

Nicely observed, but I have two just for the records (again, I know): i.) this is a list of/for CS professors; RISC expands to Research Institute for Symbolic Computation which is a CS/Math department of Prof. Buchberger who discovered the famous Groebner bases. ii.) AFAIR (too many years gone since), the A* algorithm is pretty much a generalization for graph traversal. I remember my professor showing us that how by…

ii.) I'd noticed a while ago that all graph traversal algorithms follow the same pattern. There is a white set of undiscovered nodes, a grey set of discovered-but-unvisited nodes, and a black set of visited nodes. To execute the graph traversal, you push a start node into the grey set, and then recursively pop a node off the grey set, add any children of it that are currently in the white set to the grey set, and repeat until everything is in the black set. The structure of the grey set determines the particular traversal algorithm:

1. If it is a stack, then you have a depth-first search, which visits all children first and then moves on to siblings.

2. If it is a queue, then you have a breadth-first search, which visits all siblings first and then starts recursing into children.

3. If it is a priority queue keyed by edge weights, then you have Dijkstra's algorithm, which will find the shortest path to any node in the graph.

4. If it is a priority queue keyed by in-degree of nodes, then you have a topological sort.

5. If it is a priority queue keyed by the lowest edge weight such that the endpoint of the edge is in the black set, then you have Prim's algorithm for minimum spanning trees.

You can have more exotic data structures too, eg.:

6. If it is a linked-list encoding of a stack done through pointer reversal, you have a mark & sweep garbage collector.

7. If it is a test on whether the pointer points to from-space or to-space, you have a Cheney-style breadth-first copying garbage collector.

I'd never seen a formalism that described this - it was just something I figured out when I was implementing my 3rd or so topological sort. I assumed that something like that must exist though - the notion of white/grey/black sets for graph traversal is well established in at least the garbage collection literature, and from there it's a short leap to figure out what structure of the grey set corresponds to which graph algorithm.

I just took a quick look at the A* entry in Wikipedia - I've heard of it but never had a reason to implement it myself - but it seems like it's a generalization of Dijkstra's algorithm where edge weights can be arbitrary functions. In this case, it's the same as case #3, but the keys to the priority queue are a function of the node instead of a straight list of edge weights.

Re: The Most Important Algorithms

#32
post #28

Earlier quoted context omitted.

Nicely observed, but I have two just for the records (again, I know): i.) this is a list of/for CS professors; RISC expands to Research Institute for Symbolic Computation which is a CS/Math department of Prof. Buchberger who discovered the famous Groebner bases. ii.) AFAIR (too many years gone since), the A* algorithm is pretty much a generalization for graph traversal. I remember my professor showing us that how by…

ii.) I'd noticed a while ago that all graph traversal algorithms follow the same pattern. There is a white set of undiscovered nodes, a grey set of discovered-but-unvisited nodes, and a black set of visited nodes. To execute the graph traversal, you push a start node into the grey set, and then recursively pop a node off the grey set, add any children of it that are currently in the white set to the grey set, and rep…

Very nice presentation!

In A* if you use a heuristic to select an edge (using a priority queue, i.e., your #3) it gives a best-first search. But it is is possible to just use a stack/queue an forget about the heuristic, then you get a nice generic way of tree (graph) traversal.

Regarding your use of white/black/grey lists: in a tree (which is acylic), you would only need a list of nodes to visit, however, in a general graph you need to keep a list of nodes that you have already seen/visited so as not to get into a cycle when the graph is cyclic. In tracing garbage collection algorithms this is often used because the live program data generated by the mutator is cyclic, or at least potentially can be. Thanks for mentioning this, though I know my way around gc stuff, too, I would have never come up with this link here! Probably it is a good advice to anybody interested to spend some time reading into garbage collection algorithms, they contain many interesting algorithms on graphs, such as the Deutsch-Schorr-Waite algorithm for tree traversing without requiring an additional stack. The definitive book on garbage collection is 1996's Jones and Lins: http://www.amazon.com/Garbage-Collection-Algorithms-Automati...

Re: The Most Important Algorithms

#33
post #30
post #5

Earlier quoted context omitted.

I am going through Cormen et al. with the objective of getting my feet wet but the whole process is painfully slow. I thought it would be nice to have at least some familiarity on the widely used algorithms across the globe. However, you are right on ditching programming languages with huge boot times. I am happily trying out my algorithms in Python.

CLR is an extremely well written algorithms textbook, but I use it more as a reference than for self-study. My first algorithms book was Sedgewick's Algorithms (where all algorithms were presented in Pascal), which is a very good algorithms text that is much lighter on several aspects. Recently, however, I came across the following gem: Algorihtms + Data Structures = Programs, by Niklaus Wirth. (an Oberon version of…

Thanks. BTW, here is the correct link for the book that you referred to on your comment - http://www-old.oberon.ethz.ch/WirthPubl/AD.pdf.

Re: The Most Important Algorithms

#36
post #33
post #30

Earlier quoted context omitted.

CLR is an extremely well written algorithms textbook, but I use it more as a reference than for self-study. My first algorithms book was Sedgewick's Algorithms (where all algorithms were presented in Pascal), which is a very good algorithms text that is much lighter on several aspects. Recently, however, I came across the following gem: Algorihtms + Data Structures = Programs, by Niklaus Wirth. (an Oberon version of…

Thanks. BTW, here is the correct link for the book that you referred to on your comment - http://www-old.oberon.ethz.ch/WirthPubl/AD.pdf .

Thanks for the correction!

Re: The Most Important Algorithms

#37
post #19
post #15

Erm... How did this list include the merge sort and heap sort, but not the quicksort?

I don't agree with that list, but I think quicksort is given way more weight than it deserves. I guess it's a desired result of whomever named it "quicksort". It's not particularly quick, has horrible worst case guarantees (which, if you care to improve, would make it slower still and very complicated), and is easy to get wrong on many accounts (repeating elements; already sorted input; unlimited recursion depth). he…

The main advantage to quicksort is that it's still the fastest average-case-in-practice of the common algorithms. If you want an O(nlogn) worst-case, introsort combines quicksort, which is fast on average but O(n^2) worst-case, with a fall-back to an O(nlogn) heapsort, with a guarantee that it'll never take more than O(nlogn) before it falls back, making it still worst-case no worse than O(nlogn). It'll of course be a constant-factor slower in those fall-back cases than if you had just used heapsort or something directly, but it'll asymptotically be no worse, and on average will get you the nice quicksort win.

Link: http://en.wikipedia.org/wiki/Introsort

Re: The Most Important Algorithms

#38
post #22

This reads like a textbook, i.e. it's algorithms that are highly useful to CS professors and students but not necessarily to practitioners. If I had to pick a top-10 list based on my professional experience, it would be: 1. Tree traversal. This comes up all the time , from directory walks to DOM traversal to code analysis to folds over abstract data types. It's also something that you need to know and can't always re…

Physics simulations. I've found it surprising how useful my intro physics course knowledge has been. Things like vectors, position vs. velocity vs. acceleration, damped harmonic oscillators, FFTs, etc. It's most useful when building UIs and games - oftentimes, you can make a UI feel significantly more natural by adding easing functions that behave like acceleration when you ease in and frictional damping when you eas…

> Any good physics book that I could look into? Preferably books that integrate physics with programming?

any game physics book would do that for you e.g. http://www.amazon.com/Game-Physics-Interactive-3d-Technology... also there is structure-and-interpretation-of-classical-mechanics (yes, by the same cohort)

Re: The Most Important Algorithms

#39
post #28

Earlier quoted context omitted.

Nicely observed, but I have two just for the records (again, I know): i.) this is a list of/for CS professors; RISC expands to Research Institute for Symbolic Computation which is a CS/Math department of Prof. Buchberger who discovered the famous Groebner bases. ii.) AFAIR (too many years gone since), the A* algorithm is pretty much a generalization for graph traversal. I remember my professor showing us that how by…

ii.) I'd noticed a while ago that all graph traversal algorithms follow the same pattern. There is a white set of undiscovered nodes, a grey set of discovered-but-unvisited nodes, and a black set of visited nodes. To execute the graph traversal, you push a start node into the grey set, and then recursively pop a node off the grey set, add any children of it that are currently in the white set to the grey set, and rep…

Hi nostrademons, do you mind if I put this observation onto Wikipedia if it is not already there? I would add the cyclic/acyclic observation of sb's reply also. http://en.wikipedia.org/wiki/Graph_traversal is just a stub, it could nearly go there.

Re: The Most Important Algorithms

#40
post #12
post #5

Earlier quoted context omitted.

I am going through Cormen et al. with the objective of getting my feet wet but the whole process is painfully slow. I thought it would be nice to have at least some familiarity on the widely used algorithms across the globe. However, you are right on ditching programming languages with huge boot times. I am happily trying out my algorithms in Python.

Strongly recommend Skiena over Cormen, for what it's worth.

Link for both please, and why? :)
Post reply on HN