Live data from Hacker News

New algorithm can dramatically streamline solutions to the ‘max flow’ problem

web.mit.edu

41–46 of 46 posts

Re: New algorithm can dramatically streamline solutions to the ‘max flow’ problem

#41
post #38

Earlier quoted context omitted.

i'm kinda hoping to have some time to hack out a haskell implementation of the SDD solver next month (not sure if i'll have the time then, but I hope to)

that's cool! I've not seen any use of a SDD, what do you intent to use this for, or could you tell how it could be used?

excellent question! Basically you could use the SDD solver http://math.mit.edu/~kelner/Publications/Docs/1301.6628v1.pd... in any context where you'd use an SVD (pseudo inverse / least squares) solver, and your matrix is symmetric and diagonally dominant.

The key bit however, is that the SDD solver as above, has really nice asymptotics for sparse matrices.

theres a few fancy algorithms that need to be engineered along the way, and theres also the question about how the constant factors work out in practice!

Re: New algorithm can dramatically streamline solutions to the ‘max flow’ problem

#42

I'm always wary of these kinds of research papers because they are often not using comparative benchmarks sanely. However, that said, there are all kinds of interesting max flow problems, so hopefully it pans out in practice. Heck, even some profile based compiler optimizations can be formulated as max-flow problems.

These papers improve time complexity, not actual running time. I'm not sure it ever makes sense to do comparative benchmarking, since you can't test on arbitrarily large inputs (which is what's important when looking at algorithm complexity). For smaller inputs (for example, N in the 1000-1 million range), constant factors impact running time significantly. To compensate, you'd have to run a benchmark of size 10^10,0…

". I'm not sure it ever makes sense to do comparative benchmarking, since you can't test on arbitrarily large inputs (which is what's important when looking at algorithm complexity)"

Of course it does, if you actually want the algorithm to be used in the real world.

Re: New algorithm can dramatically streamline solutions to the ‘max flow’ problem

#43
post #39

Earlier quoted context omitted.

These papers improve time complexity, not actual running time. I'm not sure it ever makes sense to do comparative benchmarking, since you can't test on arbitrarily large inputs (which is what's important when looking at algorithm complexity). For smaller inputs (for example, N in the 1000-1 million range), constant factors impact running time significantly. To compensate, you'd have to run a benchmark of size 10^10,0…

As you are talking about reasonably sized input, your constant examples should also be reasonable. 10000 times difference in the constant is not reasonable. You wouldn't even get that from going from all data in the level 1 cache to random memory access with 10 times more instructions. And "these papers" often do care about running time and will say whether it is possible to implement the algorithm efficiently, if yo…

> And even if it were slower in practice, research like this is very useful, because it helps gives other researchers inspiration, ever since someone thought of looking at these problems like electrical flow in 2010, there have been ever faster approximations for it, while others will find ways to implement the algorithm efficiently, just in case a naive implementation of the offered algorithm isn't efficient enough.

> Dismissing such research without having found any actual extreme inefficiencies in the proposed algorithm is not an attitude suited to anyone who wants to see progress.

I never dismissed this research or said it's not useful; on the contrary, it's very interesting. I was just arguing against the usefulness of benchmarking results for such algorithms, when the main improvement is the lower complexity.

EDIT:

> And finally, the actual speed increase in the past decades hasn't been from O(N^2) to O(N^1.9) but from O(VE^2) to O(VE), which obviously does have a real impact, and even with a non-realistic 10000 times bigger constant it would still be far faster in any graph with more 10000 edges. And graphs with billions of edges aren't unusual anymore.

One concrete example I had in mind is Strassen's algorithm, which reduces matrix multiplication from O(N^3) to O(N^2.8) or O(N^log 7). It doesn't seem to be worth it to use Strassen over the simple algorithm for relatively small matrices.

There are other examples of algorithms and data structures that offer an O(N log log N) time, where the previous algorithm needed O(N log N) to solve the same problem. You need a very large data set to justify using the former, especially if the latter are much simpler to implement.

> As you are talking about reasonably sized input, your constant examples should also be reasonable. 10000 times difference in the constant is not reasonable. You wouldn't even get that from going from all data in the level 1 cache to random memory access with 10 times more instructions.

10000 was just an example, I guess it was unrealistic. In practice, I think an 100x slowdown from an algorithm that has bad cache behavior makes some sense. [1] says that an L3 cache hit has a latency of 1-300 cycles, as opposed to a L1 hit that is only 4 cycles. That's almost an 100x difference, so theoretically a cache-friendly algorithm can be that much faster (and that's just the difference between L3 and L1, main memory takes even longer to access).

1 - http://stackoverflow.com/questions/4087280/approximate-cost-...

Re: New algorithm can dramatically streamline solutions to the ‘max flow’ problem

#44

Earlier quoted context omitted.

These papers improve time complexity, not actual running time. I'm not sure it ever makes sense to do comparative benchmarking, since you can't test on arbitrarily large inputs (which is what's important when looking at algorithm complexity). For smaller inputs (for example, N in the 1000-1 million range), constant factors impact running time significantly. To compensate, you'd have to run a benchmark of size 10^10,0…

". I'm not sure it ever makes sense to do comparative benchmarking, since you can't test on arbitrarily large inputs (which is what's important when looking at algorithm complexity)" Of course it does, if you actually want the algorithm to be used in the real world .

It will get used in the real world in cases where the size of the input data justifies it. The main selling point of a better algorithm (and the main reason users would use it) is usually improved time complexity, not actual running time. Users pick it when the data is large enough to warrant it, after possibly running their own benchmarks.

Re: New algorithm can dramatically streamline solutions to the ‘max flow’ problem

#45

I just skimmed through the paper thinking maybe I can grab one thing or two. What I've found there is completely black magic.

Your method of learnings is flawed. The specific lesson to be learned here is that they changed their view on the solution for a max-flow problem from a serial probing algorithm to a parallel electrical probing solution and lowered the time-complexity. The general lesson to be learned here is that they changed their view on the solution for a massive parallel multiple choice problem from a serial probing algorithm to a parallel probing solution and lowered the time-complexity.

The actual implementation is uninteresting and will always vary according to domain and optimization, but the intuition lesson is always a valid tool that we now have. And in hindsight this was a really obvious solution, carouse one should try all the routes and identify the bottlenecks.

Food for thought, single cpu core vs GPGPU.

Re: New algorithm can dramatically streamline solutions to the ‘max flow’ problem

#46

Earlier quoted context omitted.

". I'm not sure it ever makes sense to do comparative benchmarking, since you can't test on arbitrarily large inputs (which is what's important when looking at algorithm complexity)" Of course it does, if you actually want the algorithm to be used in the real world .

It will get used in the real world in cases where the size of the input data justifies it. The main selling point of a better algorithm (and the main reason users would use it) is usually improved time complexity, not actual running time. Users pick it when the data is large enough to warrant it, after possibly running their own benchmarks.

If you can't even come show cases where the size of the input data gives you a better running time, nobody will ever use it.

"The main selling point of a better algorithm (and the main reason users would use it) is usually improved time complexity, not actual running time. "

I'm not sure where you get this. I have literally never seen anyone pick an algorithm based whose only benefit is "improved time complexity" when "actual running time" is worse for the majority of real world cases.

In particular, there are plenty of algorithms with "improved time complexity" but higher constant factors, and they don't get used.

As a concrete example: Most computations of dominators in compilers use theoretically worse (n log n instead of inverse ackerman) but practically better algorithms.

Also, you can't possible know when data is large enough to warrant it if nobody has taken the time to benchmark it.

Post reply on HN