Live data from Hacker News

Markets are efficient if and only if P = NP (2010)

arxiv.org

161–170 of 201 posts

Re: Markets are efficient if and only if P = NP (2010)

#161

Earlier quoted context omitted.

No. The reason why computation in P is considered tractable is because the exponents in polynomial coefficients in complexity analyses tend to be small. Linear-, quadratic-, and cubic-time algorithms dominate P, and we pride ourselves on finding ways to reduce those exponents. Some problems in P, like matrix multiplication and context-free parsing, have been analyzed to death this way. We give importance to this ques…

Well I'm trying to understand why this question is big. The fact that the exponents tend to be small in the algorithms that our puny brains were able to produce does not convince me one bit that all problems in P have algorithms with small exponents. Why are you so sure that P vs NP is important? Does it really relate to so many parts of computer science? I do agree that complexity analysis is a very (perhaps the mos…

There is a reasonable-to-read paper [0] that explains everything; I will only share a few responses to your concerns.

"Our puny brains" have no problem constructing O(n^100) or similarly-ridiculous problems. Here is an informal survey [1] of several good examples. In my previous message, I indicated that matrix multiplication and CFG parsing are good examples; they are both cubic-time but we have come up with even cheaper algorithms for special cases, like sparse matrices or LR(k) grammars.

Whether P equals NP is not only a fundamental question, but it's the first of infinitely-many questions about the "polynomial hierarchy" or PH, a tower of ever-more-complex classes. And part of why the question is so important is that deciding whether P=NP is actually a problem in a higher rung of another complexity class further up in PH! P!=NP is like the first of an infinite family of very very difficult meta-proofs about complexity.

"RSA wouldn't break" because PRIMES is already known to be in P, via a famous paper titled "PRIMES is in P", and RSA keys are already sized to have impractical exponents.

What P=NP implies is much more powerful. First, let's consider the actual cost of a poly-time reduction. P is self-low, which means that P with a P oracle is equivalent to P. So if P=NP, then the cost of a poly-time reduction can be rolled in, and the entire problem is still in P.

Now, which problems would be transformable via P=NP? Well, we typically assume that the solution would include a poly-time reduction for 3SAT instances into some P-complete problem. Okay, great. However, we know that 3SAT can't be transformed opaquely; if this transformation exists, it must be able to examine and specialize the structure within each specific 3SAT instance. So we'd get one powerful theorem about logic (specifically, about SAT). But that wouldn't be the end.

We'd also get transformations for the following NP-complete problems, and each transformation would embody a free theorem about the nature of the structure of all instances of the problem ([2] for a bigger list):

* Graph theory: CLIQUE, HAMILTONIAN, K-COLORING (faster compilers!), COVER

* Number theory: TSP (faster packet routing!), KNAPSACK, SUBSET-SUM, PARTITION

* Formal systems: BOUNDED-POST-CORRESPONDENCE (holy shit, bounded Turing-complete analysis in P!?)

That's some serious free theorems! There's no reason to expect that whatever would give us so much stuff for free would be easy to prove. In fact, it's quite the opposite: The fact that any NP-complete problem, if it yielded, would give us deep insight into all of the others, should give us extreme pause before hoping that P=NP.

[0] https://www.scottaaronson.com/papers/pnp.pdf

[1] https://cs.stackexchange.com/questions/87073/what-are-the-ex...

[2] https://en.wikipedia.org/wiki/List_of_NP-complete_problems

Re: Markets are efficient if and only if P = NP (2010)

#162
post #7

Earlier quoted context omitted.

weak-form efficiency in markets means something special.

Not really. It means that the market incorporates all past data. This paper is entertaining but it depends on the assumption that computing power grows exponentially forever and will eventually be able to solve all P problems at negligible cost. That assumption is clearly not justified. And the belief that any market perfectly incorporates data, not a single point off in any stock, is basically a strawman anyway.

I don't know! I'm wondering! Because weak-form efficient means something different than strong efficiency, which is what you say when the market perfectly incorporates all data. Just out of curiosity, if the data doesn't go into the market, where does it go? Strong econ minds believe that prices do incorporate all the data necessary for valuation

Re: Markets are efficient if and only if P = NP (2010)

#163
Reading this comment thread is hilarious to a degree. I think that it really highlights how much noise gets thrown into a discussion when your naming becomes too relatable. Essentially bike shedding of theory discussion. If the Efficient Market Hypothesis has been called the “Fundamental Market Property Hypothesis M = H” or similar I doubt most would have made the comments they made. But because “efficient market” sounds to relatable most feel that they can comment without even knowing what the specifics of the EMH are.

Re: Markets are efficient if and only if P = NP (2010)

#164

Earlier quoted context omitted.

> It implies simply that markets are not efficient. No it implies that efficient market states are an NP complete problem. And that we are likely approximating the optimization of efficiency (of the allocation of resources) using markets. Different approximation algorithms have different properties. Might markets be the best in every possible way, sure, but it's very unlikely given what we know about approximation al…

> If those equivalent resources are thousands (or millions) of government panels then we should be able to mathematically prove equivalency. There is a method of solving an NP complete problem with thousands (or millions) of government panels?

There is a method of solving an NP complete problem with millions of companies participating in stock markets?

Your question is not related to my point. And you'd have to at least answer it for markets first before I would bother to try. My point is that it is an interesting area of research, we should answer both questions and their interrelationships.

Re: Markets are efficient if and only if P = NP (2010)

#165

Earlier quoted context omitted.

> The idea of the principle agent problem is that the best person who is able to understand their own wants and desires is the person themselves. I don't see how markets are supposed to solve that when we allow countries with slaves to participate on the markets. Or landlords to exercise economic rent over the land that other people maintain and live on. I guess my point here is I'm not sure markets as we have them t…

I don't see how markets are supposed to solve that when we allow countries with slaves to participate on the markets. The idea that the small number of countries with however many number of slaves distorts the markets is ludicrous.

It's not that it distorts the market, but it's one reason why the assumption that markets some how solve the principle agent problem in the first place is wrong.

My personal argument is that I don't believe markets solve either the principle agent problem or the economic calculation problem, and so to require them of other systems is hypocritical. But even if markets somehow do, then Turing completeness would strongly imply that there are other economic systems that also have those properties. Otherwise magic (which this paper is evidence against).

Re: Markets are efficient if and only if P = NP (2010)

#166
post #57

Have another account here but don't want to take the karma hit for being a crank. I published this 2-page proof that the EMH is false: https://arxiv.org/abs/1011.0423 (didn't set the date so in the document it's wrong, this was published 2010.) It doesn't depend on P = NP, it's simply a rigorous proof that EMH is false. Let's switch gears a second. Here's a famous elementary proof[1] that there are infinite primes. S…

Your proof for the EMH hinges on a very specific technicality -- whether the secret prime, P, is public knowledge, or to what degree it is public knowledge. I think if we were to approach the EMH from a constructive or computability approach, we'd have to make that definition much crisper. That is, the feasibility of computing the factorization based on limited information determines whether the information is actual…

If you can show an inefficient market experimentally whenever you want, that's good enough to disprove EMH. The EMH doesn't say, "All information is normally factored into the price, except for technicalities."

I like your drug trial example but a "hypothetical ideal expert" isn't assured to exist. Do you have an existence proof?

On the other hand, unique prime factorization is assured, we know that given infinite time anyone can factor any large number.

Re: Markets are efficient if and only if P = NP (2010)

#167
post #86

Earlier quoted context omitted.

Maybe I'm being super pedantic and possibly confrontational (I apologise in advance) but it jumped out on me here. I think you've misunderstood the 'famous elementary proof' that there are infinite primes. You do not create a new prime as you have suggested (quoted below). "Multiply them together and add one. No prime divides the new number (because "every" prime leaves a remainder 1), so you've just produced a new p…

Technically, the parent post literally said "No prime divides the new number ... so you've just produced a new prime". It does not make the direct claim that the new prime is equal to the new number. It is possible that the OP did not make the mistake that you are attributing to them, although best case is that the exposition was simply unclear.

Thanks, very charitable reading but doesn't work because next sentence I said that the new prime is larger than largest. As other poster points out if 2,7 are all your primes you produce 15, whose primes are not larger than 7 so the "new prime" wasn't referring to one of them.

But that's not how I argued. Instead I showed that 15 is the new prime because it's relatively prime to all the primes. (By the assumption we had started from all the primes.) And this is what falsifies the assumption.

I think my argument is fine and I don't need to address whether 15 is really a prime.

Re: Markets are efficient if and only if P = NP (2010)

#168
post #6

P = NP when you nail it. From the paper: "if markets are weak-form efficient, meaning current prices fully reflect all information available in past prices, then P = NP, meaning every computational problem whose solution can be verified in polynomial time can also be solved in polynomial time. I also prove the converse by showing how we can "program" the market to solve NP-complete problems. However, there has been n…

That actually sounds like a refreshingly fun paper to read.

Only if you can nerd out about EMH "Efficient Market Hypothesis" for 33 pages

Re: Markets are efficient if and only if P = NP (2010)

#169

Earlier quoted context omitted.

Well I'm trying to understand why this question is big. The fact that the exponents tend to be small in the algorithms that our puny brains were able to produce does not convince me one bit that all problems in P have algorithms with small exponents. Why are you so sure that P vs NP is important? Does it really relate to so many parts of computer science? I do agree that complexity analysis is a very (perhaps the mos…

There is a reasonable-to-read paper [0] that explains everything; I will only share a few responses to your concerns. "Our puny brains" have no problem constructing O(n^100) or similarly-ridiculous problems. Here is an informal survey [1] of several good examples. In my previous message, I indicated that matrix multiplication and CFG parsing are good examples; they are both cubic-time but we have come up with even ch…

Thanks for the explanation.

But, I do have some comments:

1)

As far as a I can tell the fact that PRIMES is in P doesn't mean breaking RSA is. The problem in rsa is integer factorization for which we do not know whether it is in P. (But also do not know for it to be NP-complete.)

But here you are actually supporting my point: event in that case we could choose sufficiently sized keys.

2)

And yes, the fact that P is self-low does seem to be the best justification we have to give such importance to P vs NP. That's also somewhat related to what I was saying about Turing machines and random access machines having polynomial reductions between them. But that's just because we decided to ignore polynomial reductions.

Continuing my example with even number being the same: We can see that multiplying an even number by any number yields an even number. Does that in any way justify ignoring the multiplication alltogether?

P with a P oracle really is equivalent to P, but O(n) with a O(n) oracle is not O(n), it's O(n^2).

3)

We have many NP-complete problems that have practical algorithms that perform fast on real-world inputs. I don't see how getting a free theorem about K-COLORING would practically mean faster compilers if the polynomial exponent was galactic.

Re: Markets are efficient if and only if P = NP (2010)

#170

Earlier quoted context omitted.

In my opinion P = NP does not mean that. There still might be a big polynomial difference between verifying a solution and finding one.

Of course, the concrete costs can be different; I was talking asymptotically :)

Well that's the problem, man.

And asymptotically is not the right word either, because n^3 is asymptotically different from n^2. You probably meant poly-time reducible, whichs is quite different from "the same".

Post reply on HN