We verify the correctness of a spreadsheet engine
1–8 of 8 posts
Re: We verify the correctness of a spreadsheet engine
#2I had written this comment[0] about our pattern here a few months ago and we decided to turn it into a full blog post.
It's crazy how many weird bugs you can detect with such a trivial method.
For applications like a spreadsheet that have a million overlapping features, it's impossible to manually write unit tests for every combination of features, so randomized tests do a lot of heavy lifting.
Re: We verify the correctness of a spreadsheet engine
#3I had written this comment[0] about our pattern here a few months ago and we decided to turn it into a full blog post. It's crazy how many weird bugs you can detect with such a trivial method. For applications like a spreadsheet that have a million overlapping features, it's impossible to manually write unit tests for every combination of features, so randomized tests do a lot of heavy lifting. [0] https://news.ycomb…
Unless your spread sheet can calculate unit tests, then it can combine tests in combinatorial fashion, and build Weinerstrauss test monsters.
( I am kidding on this last point, Spreadsheets will never operate in fractional dimensions. )
If, of course, you can calculate rule 110/124, then your spread sheet can be considered Turing complete.
"With LAMBDA, Excel has become Turing-complete. You can now, in principle, write any computation in the Excel formula language.Jan 25, 2021"."
The only caveat to Excel being Turing-complete is that it will never ever display the results properly.
Re: We verify the correctness of a spreadsheet engine
#4I had written this comment[0] about our pattern here a few months ago and we decided to turn it into a full blog post. It's crazy how many weird bugs you can detect with such a trivial method. For applications like a spreadsheet that have a million overlapping features, it's impossible to manually write unit tests for every combination of features, so randomized tests do a lot of heavy lifting. [0] https://news.ycomb…
Unless your spread sheet can calculate unit tests, then it can combine tests in combinatorial fashion, and build Weinerstrauss test monsters. ( I am kidding on this last point, Spreadsheets will never operate in fractional dimensions. ) If, of course, you can calculate rule 110/124, then your spread sheet can be considered Turing complete. "With LAMBDA, Excel has become Turing-complete. You can now, in principle, wri…
I tested Excel, and even early versions can calculate Rule 124.
Now I have loaded it down with rotating a right triangle in 1/n radians.
( The right triangle has sides e+1/Pi and Pi+1/e. )...
Maybe your spreadsheet will do better?
Re: We verify the correctness of a spreadsheet engine
#5FTA: Sometimes, for the most general tests, the only invariant you can assert is that the program doesn't crash. One of the most valuable tests we ever wrote at Row Zero is the one that just takes the top-level spreadsheet interface, and takes completely random actions on it
That strategy was also part of testing the original Mac. See https://www.folklore.org/Monkey_Lives.html.
As that article hunts at (“usually it didn't run for more than 20 minutes, even if it didn't crash, because the Monkey would invariably select the quit command”), there’s a black art in determining what “completely random” means.
You probably want the ability to change that depending on your testing needs. For example, if you’ve been working on the code for array formulas, the randomizer better create and edit them fairly frequently.
Re: We verify the correctness of a spreadsheet engine
#6> "Eliminate all local excel files.."
Isn't the ability to interact with local environment, half the reason why people use Excel in the first place?
Okey maybe if they connect the cloud instance to a shared drive, that could indeed replace it
Re: We verify the correctness of a spreadsheet engine
#7> "Eliminate all local excel files.." Isn't the ability to interact with local environment, half the reason why people use Excel in the first place? Okey maybe if they connect the cloud instance to a shared drive, that could indeed replace it
A lot of companies view employees having CSVs and Excel files locally on their laptops to be a huge security risk — it just takes one employee getting their machine compromised to leak ~all the company's sensitive data.
But you're right, there are people who really want local-only. We'll probably make a desktop app in the future.
Re: We verify the correctness of a spreadsheet engine
#8FTA: Sometimes, for the most general tests, the only invariant you can assert is that the program doesn't crash. One of the most valuable tests we ever wrote at Row Zero is the one that just takes the top-level spreadsheet interface, and takes completely random actions on it That strategy was also part of testing the original Mac. See https://www.folklore.org/Monkey_Lives.html . As that article hunts at (“usually it…
We actually have two versions of this test. One does 100% totally random actions. The other tries to simulate an actual user (i.e. maintains the state of their cursor, which sheet they are looking at, etc), but still quite random.
So the former finds crazy obscure edges, but the combinatorics make it less likely to hit any particular combination. The latter is the opposite. They pair really well.