Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if writing tests was a joyful experience? (2023)

blog.janestreet.com

Post reply on HN
Print view View original

21–30 of 47 posts

Re: What if writing tests was a joyful experience? (2023)

#21

Post by shruubi »

I realize that the example is contrived, but what is the point of writing a test of a fibonacci function if your test harness is designed to just take whatever it tells you and updates the assert to verify that what it told you is indeed what it just told you.

This assumes the code you wrote is already correct and giving the correct answer, so why bother writing tests? If, however you accept that you may have got it wrong, figure out the expected outcome through some reliable means (in this case, dig out your old TI-89), get the result and write your test to assert against a known correct value.

I wouldn't trust any tests that are written this way.

Re: What if writing tests was a joyful experience? (2023)

#22

Post by lihaoyi »

I was inspired by the jane street post and implemented exactly this in my Scala unit testing library uTest (http://www.lihaoyi.com/post/GoldenLiteralTestinginuTest090.h...). Can confirm that auto updating golden test assertions does make working with a test suite much more joyful than struggling with each assertion by hand

Re: What if writing tests was a joyful experience? (2023)

#23

Post by pickleRick243 »

shruubi wrote: post #21

I realize that the example is contrived, but what is the point of writing a test of a fibonacci function if your test harness is designed to just take whatever it tells you and updates the assert to verify that what it told you is indeed what it just told you. This assumes the code you wrote is already correct and giving the correct answer, so why bother writing tests? If, however you accept that you may have got it…

Oftentimes, the main purpose of writing the tests is to prevent future regressions. This is pretty common for instance in ML (pytorch) code. You put in some tests with various random inputs and assert it against whatever your network says the output is. That way you don't accidentally change how the network works in future refactors.

Re: What if writing tests was a joyful experience? (2023)

#24

Post by benrutter »

There's some cool ideas about unit testing here, and I know I'm kind of missing the point, but am I the only one who finds unit tests and documentation sort of, soothing?

Of course I love solving the initial problem / building the feature etc, but I always find unit tests a calming easy going exercise. They are sometimes interesting to think about writing, but normally fairly simple. Either way, once you're testing, you're normally on the home straight with whatever it is you're developing.

Re: What if writing tests was a joyful experience? (2023)

#25

Post by globular-toast »

xenophonf wrote: post #17
Storment33 wrote: post #8

Earlier quoted context omitted.

It is called snapshot testing, very valid technique. Maybe not best suited to a mathematical function like they have here, but I have found it useful for stuff like compilers asserting on the AST, where it would be a pain to write out and assert on the output and may also change shape.

TIL. That looks like a nice way to add tests to legacy code without having to re-create what TDD would have had the developers started that way.

It is indeed a good way to add regression testing to code with no tests. But it's no substitute for TDD. It can't tell you why something is the way it is, nor can it distinguish between intentional and incidental (although maybe some would argue you shouldn't, given Hyrum's law and all). But it will at least guide you as you try to figure that out and stop you breaking stuff constantly.

Re: What if writing tests was a joyful experience? (2023)

#26

Post by Smaug123 »

o_nate wrote: post #2

This is a cool idea. I wish something like this existed for C#.

The thing that most surprises me is that IDEs don't have a standard protocol for this, so you basically need a custom test runner if you want one-click "this snapshot failed; update it" self-modifying tests.

I wrote WoofWare.Expect for F#, which has an "update my snapshots on disk" mode, but you can't go straight from test failure to snapshot update without a fresh test run, even though I'm literally outputting a patience diff that an IDE could apply if it knew how.

Worse, e.g. Rider is really bad at knowing when files have changed underneath it, so you have to manually tell it to reload the files after running the update or else you clobber them in the editor.

Re: What if writing tests was a joyful experience? (2023)

#27

Post by twic »

lelandfe wrote: post #20
deathanatos wrote: post #4

> You start writing assert fibonacci(15) == ... and already you’re forced to think. What does fibonacci(15) equal? If you already know, terrific—but what are you meant to do if you don’t? Um …duh? Get out a calculator. Consult a reference, etc. Otherwise compute the result, and ensure you've done that correctly , ideally as independent of the code under test as possible. A lot of even mathematical stuff has "test vec…

> Um …duh? Get out a calculator. Consult a reference, etc. Otherwise compute the result Article: > This is a perfectly lovely test. But think: everything in those describe blocks had to be written by hand. The programmer first had to decide what properties they cared about... then also had to say explicitly what state they expected each field to be in. Then they had to type it all out. The article is about not gettin…

> The programmer first had to decide what properties they cared about... then also had to say explicitly what state they expected each field to be in.

Yes, this is the point of testing. You have to think about what you're about to write! Before you write it! The technique in the article completely discards this. It's a terrible way to write tests.

Re: What if writing tests was a joyful experience? (2023)

#28

Post by shafyy »

benrutter wrote: post #24

There's some cool ideas about unit testing here, and I know I'm kind of missing the point, but am I the only one who finds unit tests and documentation sort of, soothing? Of course I love solving the initial problem / building the feature etc, but I always find unit tests a calming easy going exercise. They are sometimes interesting to think about writing, but normally fairly simple. Either way, once you're testing,…

Yes, same feeling here =)

Re: What if writing tests was a joyful experience? (2023)

#29

Post by prophesi »

shruubi wrote: post #21

I realize that the example is contrived, but what is the point of writing a test of a fibonacci function if your test harness is designed to just take whatever it tells you and updates the assert to verify that what it told you is indeed what it just told you. This assumes the code you wrote is already correct and giving the correct answer, so why bother writing tests? If, however you accept that you may have got it…

I believe it's called test-driven development but often I write tests hoping that what I tell myself the application code will do does what I want it to do. It's also self-describing of the changes made, and what people new to the codebase should reference if they actually want to learn what's going on.

Re: What if writing tests was a joyful experience? (2023)

#30

Post by imtringued »

deathanatos wrote: post #4

> You start writing assert fibonacci(15) == ... and already you’re forced to think. What does fibonacci(15) equal? If you already know, terrific—but what are you meant to do if you don’t? Um …duh? Get out a calculator. Consult a reference, etc. Otherwise compute the result, and ensure you've done that correctly , ideally as independent of the code under test as possible. A lot of even mathematical stuff has "test vec…

Yeah this is one of the weirdest takes ever.

"If you already know, terrific—but what are you meant to do if you don’t?"

You're supposed to look at the first gif that visualizes a waveform diagram. How are HDL designs tested? Testbenches (akin to unit tests) and model checking. With model checking you define the property you want to test and the model checker will try to find a counter example.

Said property is so obvious for fibonacci, that it is staring right at your face and you're consciously trying to avoid looking it in the eyes. Fibonacci is defined as fib(n) = fib(n-1) + fib(n-2), so that's what you need to test. This means you can simply test fib(1) = 1, fib(2) = 1, fib(3) = 2, for a fixed set of n to cover the edge cases, then choose a fixed set of random n and make sure that fib(n) = fib(n-1) + fib(n-2) is true. Obviously the only way to be 100% sure is to use a model checker and write code that is bounded in its runtime.

Post reply on HN