Live data from Hacker News

My unusual hobby

stephanboyer.com

51–60 of 157 posts

Re: My unusual hobby

#51
post #20
post #10

And this... " To give you an idea of what an actual proof looks like in Coq, below is a proof of one of the easier lemmas above. The proof is virtually impossible to read without stepping through it interactively, so don’t worry if it doesn’t make any sense to you here. " is why Coq is not one of my favorite tools. A proof consists of two things: the "proof state"---a statement of what you know at a given point--and…

I you prefer readable proofs, try Isabelle or Lean.

Or TLA+.

Re: My unusual hobby

#52
post #50

I had to take a semester on formal proofs using Coq, the same tool the article talks about. Putting aside their steep learning curve, formal proof methods do not guarantee that the code you've written is bug free. They only guarantee that the code follows the requirements you defined given the conditions you also set on your inputs. You can think of it as a mathematical proof of your postconditions will hold given th…

”formal proof methods do not guarantee that the code you've written is bug free. They only guarantee that the code follows the requirements you defined given the conditions you also set on your inputs.” I think you are mixing “bug” with “out of spec”. If you ask me to write a perfect chess program, you can’t say it’s a bug if it can’t play go. Bad or incomplete requirements are a problem, but that’s a different probl…

If the purpose of the program is to entertain go players, then I'd say the bug is in the spec, which describes the wrong game.

Re: My unusual hobby

#53
post #41

So I'm wondering what is the bridge between "proof-assistant" and "automated (or partially automated) theorem prover". As someone with "journalistic" (but fairly in-depth) knowledge about these topics, my guess is that something like a proof-assistant will have to be paired up with some kind of logic programming system (like Prolog, or a kanren-derived system or something). I guess then the problem of combinatorial e…

I've done a bit of work on a "next generation" Metamath, Ghilbert[1], but have come to the conclusion that it's too ambitious to do as a part-time project. People might find it interesting, though, because it attempts to take interactivity to a higher level than Metamath, and is (imho) a cleaner language. [1] http://ghilbert.org/

Hey glad to find someone involved in metamath related project.

I guess if I could simplify my discussion to one question, it would be: Does metamath or ghilbert use a logic inference engine to automate parts of a proof?

The way I understand metamath (the website, not the language) is that it's like an encyclopedia of mathematical knowledge. The encyclopedia is in the form of theorems, lemmas, corollaries, and their proof. But unlike an encyclopedia or disconnected articles, all the propositions build on top of one another. As a result, it builds (or tries to build) the whole mathematical edifice starting from core axioms (well, I guess it didn't build itself, but hundreds of mathematicians contributed to the database, like people contributing to wikipedia in the form of article-writing. As to how automated this process is, I don't know).

Now the question is, can we use this edifice as a database, to be input to a logic inference engine. What this would accomplish is that:

- If I make a query about a mathematical proposition (e.g., Are there infinite prime numbers), the logic inference engine would use the database to conclude that there are if it is able to access all the relevant propositions.

- If it's not able to reach a conclusion, e.g., if my query was "Are there infinite twin primes?" it would start a process of exploring new propositions from old propositions. Then either the search would conclude very quickly (if the proposition is only one or two steps away from the knowledge in the database) or it would give up after a few million, or billion attempts (e.g., this is more likely the case with the twin prime query). There will have to be a manually-specified cutoff point otherwise it could go into an infinite loop (I guess it may have something to do with decidability, incompleteness theorem, halting problem, etc, etc. Though even many decidable problems are computationally intractable due to very very large search space).

- Suffice to say, when I say "query" I don't mean a question in english language but a well-crafted mathematical statement in language of metamath itself.

P.S.: As for your mention of this work not doable as part-time, I fully agree with you. It would have enough cognitive load to be a very complex project even if done full time.

Re: My unusual hobby

#54
Terrific!

I took the liberty of translating your module to TLA+, my formal tool of choice (recently, I've been learning Lean, which is similar to Coq, but I find it much harder than TLA+). I tried to stick to your naming and style (which deviate somewhat from TLA+'s idiomatic styling), and, as you can see, the result is extremely similar, but I guess that when I try writing the proofs, some changes to the definitions would need to be made.

One major difference is that proofs in TLA+ are completely declarative. You just list the target and the required axioms, lemmas and definitions that require expanding. Usually, the proof requires breaking the theorem apart to multiple intermediary steps, but in the case of the proof you listed, TLA+ is able to find the proof completely automatically:

    LEMMA supremumUniqueness ≜
      ∀ P ∈ SUBSET T : ∀ x1, x2 ∈ P : supremum(P, x1) ∧ supremum(P, x2) ⇒ x1 = x2
    PROOF BY antisym DEF supremum (* we tell the prover to use the antisym axiom and expand the definition of supremum *)
The natDiff lemma doesn't even need to be stated, as it's automatically deduced from the built-in axioms/theorems:

    LEMMA natDiff ≜ ∀ n1, n2 ∈ Nat : ∃ n3 ∈ Nat : n1 = n2 + n3 ∨ n2 = n1 + n3
    PROOF OBVIOUS (* this is automatically verified using just built-in axioms/theorems *)
Another difference is that TLA+ is untyped (which makes the notation more similar to ordinary math), but, as you can see, this doesn't make much of a difference. The only things that are different from ordinary math notation is that function application uses square brackets (parentheses are used for operator substitution; operators are different from functions, but that's a subtelty; you can think of operators as polymorphic functions or as macros), set comprehension uses a colon instead of a vertical bar, a colon is also used in lieu of parentheses after quantifiers, and aligned lists of connectives (conjunctions and disjunctions) are read as if there were parentheses surrounding each aligned clause. Also `SUBSET T` means the powerset of T.

Here's the module (without proofs, except for the one above):

    ------------------------------- MODULE Kleene -------------------------------
    EXTENDS Naturals
    
    (* 
      Assumption: Let (T, leq) be a partially ordered set, or poset. A poset is
      a set with a binary relation which is reflexive, transitive, and
      antisymmetric. 
    *)
      
    CONSTANT T
    CONSTANT _ ≼ _
    
    AXIOM refl    ≜ ∀ x ∈ T : x ≼ x
    AXIOM trans   ≜ ∀ x, y, z ∈ T : x ≼ y ∧ y ≼ z ⇒ x ≼ z
    AXIOM antisym ≜ ∀ x, y ∈ T : x ≼ y ∧ y ≼ x ⇒ x = y
    
    (*
      A supremum of a subset of T is a least element of T which is greater than
      or equal to every element in the subset. This is also called a join or least
      upper bound.
    *)
    
    supremum(P, x1) ≜ ∧ x1 ∈ P
                      ∧ ∀ x2 ∈ P : x2 ≼ x1
                      ∧ ∀ x3 ∈ P : ∀ x2 ∈ P : x2 ≼ x3 ⇒ x1 ≼ x3 
    
    (*
      A directed subset of T is a non-empty subset of T such that any two elements
      in the subset have an upper bound in the subset.
    *)
    
    directed(P) ≜ ∧ P ≠ {}
                  ∧ ∀ x1, x2 ∈ P : ∃ x3 ∈ P : x1 ≼ x3 ∧ x2 ≼ x3
    
    (*
      Assumption: Let the partial order be directed-complete. That means every
      directed subset has a supremum.
    *)
    
    AXIOM directedComplete ≜ ∀ P ∈ SUBSET T : directed(P) ⇒ ∃ x : supremum(P, x)
      
    (*
      Assumption: Let T have a least element called bottom. This makes our partial
      order a pointed directed-complete partial order.
    *)
    
    CONSTANT bottom
    
    AXIOM bottomLeast ≜ bottom ∈ T ∧ ∀ x ∈ T : bottom ≼ x
    
    (*
      A monotone function is one which preserves order. We only need to consider
      functions for which the domain and codomain are identical and have the same
      order relation, but this need not be the case for monotone functions in
      general.
    *)
    
    monotone(f) ≜ ∀ x1, x2 ∈ DOMAIN f : x1 ≼ x2 ⇒ f[x1] ≼ f[x2]
    
    (*
      A function is Scott-continuous if it preserves suprema of directed subsets.
      We only need to consider functions for which the domain and codomain are
      identical and have the same order relation, but this need not be the case for
      continuous functions in general.
    *)
    
    Range(f) ≜ { f[x] : x ∈ DOMAIN f }
    
    continuous(f) ≜
      ∀ P ∈ SUBSET T: ∀ x1 ∈ P :
        directed(P) ∧ supremum(P, x1) ⇒ supremum(Range(f), f[x1])
        
    (* This function performs iterated application of a function to bottom. *)
    
    RECURSIVE approx(_, _)
    approx(f, n) ≜ IF n = 0 THEN bottom ELSE f[approx(f, n-1)]
      
    (* We will need this simple lemma about pairs of natural numbers. *)
    
    LEMMA natDiff ≜ ∀ n1, n2 ∈ Nat : ∃ n3 ∈ Nat : n1 = n2 + n3 ∨ n2 = n1 + n3
    
    (* The supremum of a subset of T, if it exists, is unique. *)
    
    LEMMA supremumUniqueness ≜
      ∀ P ∈ SUBSET T : ∀ x1, x2 ∈ P : supremum(P, x1) ∧ supremum(P, x2) ⇒ x1 = x2
    PROOF BY antisym DEF supremum
        
    (* Scott-continuity implies monotonicity. *)
    
    LEMMA continuousImpliesMonotone ≜ ∀ f : continuous(f) ⇒ monotone(f)
    
    (*
      Iterated applications of a monotone function f to bottom form an ω-chain,
      which means they are a totally ordered subset of T. This ω-chain is called
      the ascending Kleene chain of f.
    *)
    
    LEMMA omegaChain ≜
      ∀ f : ∀ n, m ∈ Nat :
        monotone(f) ⇒
            approx(f, n) ≼ approx(f, m) ∨ approx(f, m) ≼ approx(f, n)
    
    
    (* The ascending Kleene chain of f is directed. *)
    
    LEMMA kleeneChainDirected ≜
      ∀ f : monotone(f) ⇒ directed({ approx(f, n) : n ∈ Nat })
      
    (*
      The Kleene fixed-point theorem states that the least fixed-point of a Scott-
      continuous function over a pointed directed-complete partial order exists and
      is the supremum of the ascending Kleene chain.
    *)
    
    THEOREM kleene ≜
      ∀ f : continuous(f) ⇒ 
        ∃ x1 ∈ T : 
            ∧ supremum({ approx(f, n) : n ∈ Nat }, x1)
            ∧ f[x1] = x1
            ∧ ∀ x2 : f[x2] = x2 ⇒ x1 ≼ x2
    
    =============================================================================

Re: My unusual hobby

#55
post #54

Terrific! I took the liberty of translating your module to TLA+, my formal tool of choice (recently, I've been learning Lean, which is similar to Coq, but I find it much harder than TLA+). I tried to stick to your naming and style (which deviate somewhat from TLA+'s idiomatic styling), and, as you can see, the result is extremely similar, but I guess that when I try writing the proofs, some changes to the definitions…

If what you say is true, then this is one more reason for me to learn TLA+.

As I asked in a different question, is TLA+ able to do the proofs declaratively and automatically because it has an internal 'logic inference' engine?

Re: My unusual hobby

#56
post #5

> What's really amazing to me is that Stephen Kleene probably proved this without the help of a computer, but I was able to verify it with the highest possible scrutiny. It's as if he wrote an impressive program without ever having run it, and it turned out not to have any bugs! This offhand comment (which we can all forgive) makes it seem like mathematics happens in a vacuum. I can understand the temptation to think…

> Mathematicians don't just write down a proof and cross their fingers.

On the flip side, this is exactly what a lot of lawyers do when writing contracts. Not necessarily maliciously or negligently, but it can still make for fun questions of interpretation when things don't go as expected.

Re: My unusual hobby

#58
post #54

Terrific! I took the liberty of translating your module to TLA+, my formal tool of choice (recently, I've been learning Lean, which is similar to Coq, but I find it much harder than TLA+). I tried to stick to your naming and style (which deviate somewhat from TLA+'s idiomatic styling), and, as you can see, the result is extremely similar, but I guess that when I try writing the proofs, some changes to the definitions…

Wow, this is amazing. Can you share a link to the full module with proofs? I'd love to compare it to the Coq version.

Re: My unusual hobby

#59

I had to take a semester on formal proofs using Coq, the same tool the article talks about. Putting aside their steep learning curve, formal proof methods do not guarantee that the code you've written is bug free. They only guarantee that the code follows the requirements you defined given the conditions you also set on your inputs. You can think of it as a mathematical proof of your postconditions will hold given th…

> Putting aside their steep learning curve, formal proof methods do not guarantee that the code you've written is bug free. People seem to always bring this up but what's better? Verified code is about as close as you're ever going to get to bug free. If you're doing a large proof, getting the specification wrong and not eventually noticing while working on the proof isn't common. You'll likely have something that is…

> I haven't heard of any examples of complex verified programs where later someone found a huge flaw in the specification.

Look at

> https://www.krackattacks.com/

"The 4-way handshake was mathematically proven as secure. How is your attack possible?

The brief answer is that the formal proof does not assure a key is installed only once. Instead, it merely assures the negotiated key remains secret, and that handshake messages cannot be forged.

The longer answer is mentioned in the introduction of our research paper: our attacks do not violate the security properties proven in formal analysis of the 4-way handshake. In particular, these proofs state that the negotiated encryption key remains private, and that the identity of both the client and Access Point (AP) is confirmed. Our attacks do not leak the encryption key. Additionally, although normal data frames can be forged if TKIP or GCMP is used, an attacker cannot forge handshake messages and hence cannot impersonate the client or AP during handshakes. Therefore, the properties that were proven in formal analysis of the 4-way handshake remain true. However, the problem is that the proofs do not model key installation. Put differently, the formal models did not define when a negotiated key should be installed. In practice, this means the same key can be installed multiple times, thereby resetting nonces and replay counters used by the encryption protocol (e.g. by WPA-TKIP or AES-CCMP)."

I personally would consider this as a clear example of a huge flaw in a specification where a correctness proof was done on.

Re: My unusual hobby

#60

I had to take a semester on formal proofs using Coq, the same tool the article talks about. Putting aside their steep learning curve, formal proof methods do not guarantee that the code you've written is bug free. They only guarantee that the code follows the requirements you defined given the conditions you also set on your inputs. You can think of it as a mathematical proof of your postconditions will hold given th…

> Putting aside their steep learning curve, formal proof methods do not guarantee that the code you've written is bug free. People seem to always bring this up but what's better? Verified code is about as close as you're ever going to get to bug free. If you're doing a large proof, getting the specification wrong and not eventually noticing while working on the proof isn't common. You'll likely have something that is…

> Verified code is about as close as you're ever going to get to bug free.

Code verified by legible proofs is safer than merely verified. The quality of the verification language matters, it's not only the programming language.

Post reply on HN