Live data from Hacker News

Coq: The World's Best Macro Assembler? (2013) [pdf]

nickbenton.name

51–60 of 75 posts

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#51
post #20

Earlier quoted context omitted.

I'm curious about how you'd do that, too. I haven't tried doing anything like that, but I'd think you'd start by trying to formalize the usual optimality proof for the Kalman filter, transfer it to actual program logic on the assumption that the program manipulates real numbers, try to get the proof to work on floating-point numbers, and finally extract the program from the proof to run it. https://youtu.be/_LjN3UclY…

It is really not that difficult. Here is a paper that formalizes a version of feed forward networks to prove properties about them. https://arxiv.org/pdf/2304.10558

I only skimmed this paper, but it doesn't mention floating point (it's only modeling the FNN as a function on reals), and I don't think you can extract a working FNN implementation from an SMT-LIB or Z3 problem statement, so I think you have to take on faith the correspondence between the implementation you're running and the thing you proved correct.

But that's the actual problem we're trying to solve; nobody really doubts Kalman's proof of his filter's optimality.

So this paper is not a relevant answer to the question, "how would you prove an implementation of a Kalman filter is correct?"

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#53
post #48
post #44

Earlier quoted context omitted.

I have found Isabelle very useful, and Dafny even more so. Amazon AWS uses Dafny to prove the correctness of some complex components. Then, they extract verified Java code. There are other target languages. Being based on Hoare logic, Dafny is really simple. The barrier of entry is low.

Have you tried using them as macro assemblers in the way described in the paper?

No, I haven't. I have used them with shallow embedding techniques that are relatively similar, but not in this way.

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#54
post #16

Earlier quoted context omitted.

You use floating point numbers instead of real numbers in your theorems and function definitions. This sounds flippant, but I'm being entirely earnest. It's a significantly larger pain because floating point numbers have some messy behavior, but the essential steps remain the same. I've proved theorems about floating point numbers, not reals. Although, again, it's a huge pain, and when I can get away with it I'd pref…

> when I can get away with it I'd prefer to prove things with real numbers and assume magically they transfer to floating point. True for some approaches, but numerical analysis does account for machine epsilon and truncation errors. I am aware that Inria works with Coq as your link shows. However, the link itself does not answer my question. As a concrete example, how would you prove an implementation of a Kalman fi…

The link was to show Coq's floating point library.

> As a concrete example, how would you prove an implementation of a Kalman filter is correct?

This would be fun enough to implement if I had more time. But I need to spend time with my family. I'm usually loathe to just post LLM output, but in this case, since you're looking just for the gist, and I don't have the time to write it out in detail, here's some LLM output: https://pastebin.com/0ZU51vN0

Based on a quick skim, I can vouch for the overall approach. There's some minor errors/things I would do differently, and I'm extremely doubtful it typechecks (I didn't use a coding agent, just a chat window), but it captures the overall idea and gives some of the concrete syntax. I would also probably have a third implementation of the 1-d Kalman filter and would prefer some more theorems around the correctness of the Kalman filter itself (in addition to numerical stability). And of course a lot of the theorems are just left as proof stubs at the moment (but those could be filled in given enough time).

But it's enough to demonstrate the overall outline of what it would look like.

The overall approach I would have with floating point algorithms is exemplified by the following pseudo-code for a simple single argument function. We will first need some floating point machinery that establishes the relationship between floating point numbers and real numbers.

  # Calculates the corresponding real number from the sign, mantissa, and exponent
  floatToR : Float -> R
  
  # Logical predicate that holds if and only if x can be represented exactly in floating point
  IsFloat(x: R)
Then define a set of three functions, `f0` which is on the idealized reals, `f1` which is on floating point numbers as mechanically defined via binary mantissa, exponent, and sign, and `f2` which is on the finite subset of the reals which have an exact floating point representation.

  f0 : R -> R
  f1 : Float -> Float
  f2 : (x : R) -> IsFloat(x) -> R
The idea here is that `f0` is our "ideal" algorithm. `f1` is the algorithm implemented with exact adherence to low-level floating point operations. And `f2` is the ultimate algorithm we'll extract out to runnable code (because it might be extremely tedious, messy, and horrendous for runtime performance, to directly tie `f1`'s explicit representation of mantissa, exponent, and sign, which is likely some record structure with various pointers, to the machine 32-bit float).

Then I prove the exact correspondence between `f1` and `f2`.

  f1_equal_to_f2 : forall (x: R), (isFloatProof: IsFloat(x)) -> f1(floatToR(x)) = floatToR(f2(x, isFloatProof))
This gives the following corollary FWIW.

  f2_always_returns_floats : forall (x: R), IsFloat(x) -> IsFloat(f2(x))
This lets me throw away `f1` and work directly with `f2`, which lets me more easily relate `f2` and `f0` since they both work on reals and I don't need to constantly convert between `Float` and `R`.

So then I prove numerical stability on `f2` relative to `f0`. I start with a Wilkinson style backwards error analysis.

  f2_is_backwards_stable : forall (x: R), exists (y: R), IsFloat(x) -> f2(x) = f0(y) and IsCloseTo(y, x)
Then, if applicable, I would prove a forward error analysis (which requires bounding the input and then showing that overflow doesn't happen and optionally underflow doesn't happen)

  f2_is_forwards_stable : forall (x: R), IsFloat(x) -> IsReasonablyBounded(x) -> IsCloseTo(f2(x), f0(x))
And then given both of those I might then go on to prove some conditioning properties.

Then finally I extract out `f2` to runnable code (and make sure that compilation process after that has all the right floating point flags, e.g. taking into consideration FMA or just outright disabling FMA if we didn't include it in our verification, making sure various fast math modes are turned off, etc.).

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#55
post #53
post #48

Earlier quoted context omitted.

Have you tried using them as macro assemblers in the way described in the paper?

No, I haven't. I have used them with shallow embedding techniques that are relatively similar, but not in this way.

That sounds interesting!

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#56

Earlier quoted context omitted.

bit (in English) is not pronounced the same as bite (in French). The French word is closer in pronunciation to “beet” or “beat” in English. Also, “coq” and “cock” are not really pronounced the same either. The English word with the closest pronunciation to “coq” is “coke”.

> bit (in English) is not pronounced the same as bite (in French). The French word is closer in pronunciation to “beet” or “beat” in English. Wrong, it's pronounced exactly like the English "bit".

That’s not true, at least in France. Perhaps it’s true in some other dialect, e.g. Quebec French; I don’t know.

From Wiktionary, the pronunciation of English bit is /bɪt/, and French bite is /bit/. The sounds represented in IPA by ɪ and i are not the same, which is precisely why “bit” and “beet” sound different to Americans.

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#57
post #12
post #4

Poster here. I re-read this paper about once a year. I continue to think that it may be one of the most important papers I've read. As someone who works on high-reliability safety-critical real-time systems (automotive, avionics), being able to work in an environment where I could prove semantic properties of assembly code is pretty close to my dream -- the cost of demonstrating code correct is already so much higher…

During the days I was studying/working with Coq, one visiting professor gave a presentation on defense software design. An example presented was control logic for F-16, which the professor presumably worked on. A student asked how do you prove "correctness", i.e. operability, of a jet fighter and its control logic? I don't think the professor had a satisfying answer. My question is the same, albeit more technically r…

Curry-Howard is not needed for theorem proving, it's just that type theorists like it.

See https://www.cl.cam.ac.uk/~jrh13/papers/thesis.html for your question, John Harrison got a job with Intel based on this after their floating point disaster.

But in short: theorem proving is not about equalities, it is about inequalities. And theorems about numerical algorithms are a great example of this.

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#58
post #31
post #18

Earlier quoted context omitted.

I don't think they're particularly less bad. All five architectures just treat memory as a single untyped array of integers, and registers as integer global variables. Their only control structure is goto (and conditional goto.) If you forget to pass an argument to a subroutine, or pass a pointer to an integer instead of the integer, or forget to save a callee-saved register you're using, or do a signed comparison on…

I have a postit note idea that says simply "typesafe macro assembler". I've not fleshed this out yet, but I think a relatively simple system would help deal with all the issues you mention in the first paragraph while allowing escape hatches.

Check out typed assembly languages like TALx86.

https://en.wikipedia.org/wiki/Typed_assembly_language

Re: Coq: The World's Best Macro Assembler? (2013) [pdf]

#60
post #31

Earlier quoted context omitted.

I have a postit note idea that says simply "typesafe macro assembler". I've not fleshed this out yet, but I think a relatively simple system would help deal with all the issues you mention in the first paragraph while allowing escape hatches.

Check out typed assembly languages like TALx86. https://en.wikipedia.org/wiki/Typed_assembly_language

Thanks for the link!
Post reply on HN