Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if writing tests was a joyful experience? (2023)

blog.janestreet.com

Post reply on HN
Print view View original

31–40 of 47 posts

Re: What if writing tests was a joyful experience? (2023)

#31

Post by debugnik »

shruubi wrote: post #21

I realize that the example is contrived, but what is the point of writing a test of a fibonacci function if your test harness is designed to just take whatever it tells you and updates the assert to verify that what it told you is indeed what it just told you. This assumes the code you wrote is already correct and giving the correct answer, so why bother writing tests? If, however you accept that you may have got it…

These are great to test code that has no formal specification or oracle, so you're writing a reference implementation.

First, the test fails because there's no expected output, and you get to check the existing behaviour pretty-printed. Then, if it's correct, you approve it by promoting the diff into the source code, and it becomes a regression test.

Re: What if writing tests was a joyful experience? (2023)

#32

Post by warpspin »

shruubi wrote: post #21

I realize that the example is contrived, but what is the point of writing a test of a fibonacci function if your test harness is designed to just take whatever it tells you and updates the assert to verify that what it told you is indeed what it just told you. This assumes the code you wrote is already correct and giving the correct answer, so why bother writing tests? If, however you accept that you may have got it…

> This assumes the code you wrote is already correct and giving the correct answer, so why bother writing tests?

It catches regressions. Which is the one thing where such semi-automated testing is most useful in my eyes.

No clue though why they gave it that weird "expect" name. Basically, it's semi-automated regression testing.

Re: What if writing tests was a joyful experience? (2023)

#33

Post by RhysU »

Smaug123 wrote: post #26
o_nate wrote: post #2

This is a cool idea. I wish something like this existed for C#.

The thing that most surprises me is that IDEs don't have a standard protocol for this, so you basically need a custom test runner if you want one-click "this snapshot failed; update it" self-modifying tests. I wrote WoofWare.Expect for F#, which has an "update my snapshots on disk" mode, but you can't go straight from test failure to snapshot update without a fresh test run, even though I'm literally outputting a pat…

> ...if you want one-click "this snapshot failed; update it" self-modifying tests.

I am envisioning the PR arguments now when the first instinct of the junior developer is to clobber the prior gold standard outputs. Especially lovely when testing floating point functionality using tests with tolerances.

Some things should be hatefully slow so one's brain has sufficient chance to subconsciously mull over "what if I am wrong?"

Re: What if writing tests was a joyful experience? (2023)

#34

Post by krater23 »

mlmonkey wrote: post #12

Recently, I have given up on writing unit tests, instead prompting an LLM to write them for me. I just sit back and keep prompting it until it gets it right. Sometimes it goes a little haywire in our Monorepo, but I don't have to accept its changes. It feels ... strangely empowering.

i think thats the best use you can get from LLM's in programming. Doing the boring simple test code that doesn't have to meet any quality requirements. Normally on a code review I ignore unit tests, written from humans oder LLM's out of this reason.

Re: What if writing tests was a joyful experience? (2023)

#35

Post by Cthulhu_ »

mlmonkey wrote: post #12

Recently, I have given up on writing unit tests, instead prompting an LLM to write them for me. I just sit back and keep prompting it until it gets it right. Sometimes it goes a little haywire in our Monorepo, but I don't have to accept its changes. It feels ... strangely empowering.

Same, and to avoid it going haywire I wrote an agents.md file with some prompts, like how to run a test for a single file and what to do before saying "I am done".

Re: What if writing tests was a joyful experience? (2023)

#36

Post by YetAnotherNick »

shruubi wrote: post #21

I realize that the example is contrived, but what is the point of writing a test of a fibonacci function if your test harness is designed to just take whatever it tells you and updates the assert to verify that what it told you is indeed what it just told you. This assumes the code you wrote is already correct and giving the correct answer, so why bother writing tests? If, however you accept that you may have got it…

And that's why the example is good. As you first get fib(3) then see the result, verify it and then freeze and then do for fib(20). That's how software development works that you can spot errors easier than a test could.

Re: What if writing tests was a joyful experience? (2023)

#38

Post by i_don_t_know »

mdx[1] is another variation on this, also in the Ocaml ecosystem. It’s Ocaml’s version of documentation tests as in Elixir and Rust.

But it’s not limited to that. You can write tests in markdown files independently from your documentation. Use “dune test” to run the tests and review failures with “git diff”. Accept the changes if they are correct (changed behavior) with “dune promote”. Very nice workflow.

[1] https://github.com/realworldocaml/mdx

Re: What if writing tests was a joyful experience? (2023)

#39

Post by tasuki »

mlmonkey wrote: post #12

Recently, I have given up on writing unit tests, instead prompting an LLM to write them for me. I just sit back and keep prompting it until it gets it right. Sometimes it goes a little haywire in our Monorepo, but I don't have to accept its changes. It feels ... strangely empowering.

I do it the other way around: I write the specs and unleash an agent to turn my test suite from red to green.

Each of us does half the work, the other half being done by a LLM. The difference is that I specify the desired behavior, while you leave the specification up to the LLM. A little strange if you ask me!

Post reply on HN