Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Solving Algorithmic Problems in Python with Pytest (2019)

adamj.eu

Post reply on HN
Print view View original

31–40 of 45 posts

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#31

Post by daveFNbuck »

tcollier wrote: post #24
henrikeh wrote: post #14

Earlier quoted context omitted.

You could approach this a number of ways. First, if you can ensure the seed of the generator, then that could be used on a case by case basis. But this tricky, since it introduces a subtle dependency on the implementation of biasedcoin. Another option is to split the function/process into a stochastic and deterministic part. Test the deterministic part thoroughly. biasedcoin is too trivial for this, but it works nice…

The statistical approach definitely gives a developer more confidence in the correctness of a solution and guards against regressions. Though any test that checks the outcome of a random process against statistical measures is expected to fail occasionally. For example, if the test flips the coin 1,000 times, there is better than a 99% chance that the outcome would result in 450 to 550 heads. So if you write the test…

You can safely use this type of test in your pipeline if you set a fixed random seed.

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#32

Post by tmoertel »

When testing solutions to algorithmic problems, it's often useful to use randomized property checking to verify that the solution's expected properties hold for approximately all inputs. The QuickCheck family of testing tools is probably the best known application of this approach. It's also pretty easy to roll your own. Some hand-rolled examples in Python:

https://github.com/tmoertel/practice/blob/master/dailycoding...

https://github.com/tmoertel/practice/blob/master/dailycoding...

https://github.com/tmoertel/practice/blob/master/dailycoding...

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#34

Post by fnord123 »

tmoertel wrote: post #32

When testing solutions to algorithmic problems, it's often useful to use randomized property checking to verify that the solution's expected properties hold for approximately all inputs. The QuickCheck family of testing tools is probably the best known application of this approach. It's also pretty easy to roll your own. Some hand-rolled examples in Python: https://github.com/tmoertel/practice/blob/master/dailycoding…

Hypohesis is a tool to do this https://github.com/HypothesisWorks/hypothesis/tree/master/hy...

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#35

Post by andreareina »

henrikeh wrote: post #33

> unittest is an old horse and cart, while pytest is the batmobile. I really wish more was said than just this. Why is it that pytest is often preferred and unittest is considered a bad choice? What is the impact on the quality of the tests?

Pytest:

- lets you use regular assert statements, rather than assertEqual, assertTrue, etc.

- does not impose the use of classes

- the way it does fixtures just feels nicer to use than unittest

Basically, it's more ergonomic and IME more flexible.

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#36

Post by chriswarbo »

mlthoughts2018 wrote: post #13
abdullahkhalids wrote: post #9

How would you write tests for the following problem with random output? Write a function biasedcoin(n,p) that takes the number of coin flips, n, and the probability of heads, p. It flips a biased coin n times, and returns the ratio of number of heads/number of tails. p is guaranteed to have two significant numbers eg. p = 0.60 or p = 0.74. You should use the random.randrange function to generate random numbers. Examp…

Simple: use mocker.patch to patch out random.randrange and have it return a fixed sequence of results that would imply a known result. Your test should not depend on the internal workings of randrange or anything relying on unfixed random state. Your test is only checking if, given correct results from randrange (or any other external world source of random draws) that the rest of your function correctly produces the…

> Simple: use mocker.patch to patch out random.randrange

Why not just pass in 'randrange' as a function argument?

    def findBias(randrange):
      heads = tails = 0
      for i in range(n):
        if randrange(100)
It's amazing to me how many complicated and convoluted hacks people can come up with, to avoid calling a function with an argument.

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#37

Post by ploika »

Yajirobe wrote: post #19
ploika wrote: post #10

Earlier quoted context omitted.

It's doable once you're out of pure experimentation and into the development phase at which test driven development can help. Test that this ETL function expects a DataFrame with a given schema and returns one with a different (but also known) schema, even with all these edge cases in the filters and group-bys. Test that the "train_classifier" method/function rejects negative penalisation parameters, returns an objec…

And how do you test that your model has sufficient accuracy? How do you make sure that your model does not deteriorate over time?

Those are statistics/ML questions, not software development ones, so software development processes like TDD are only tangentially relevant really.

But in any case, it's actually fairly easy to test that your trained model has sufficient accuracy: choose a metric, choose a threshold for said metric, and check that the observed metric on a testing set (data that the model was not trained on) is above the desired threshold. Repeat for several different metrics for a better understanding of how well the model performs. This can be put in a set of unit tests.

Check residuals, inspect the logical implications of the regression coefficients (or whatever), plot a few curves etc to be more sure again. This can't really be put in unit tests, nor should it be. But again, this is more statistics than software development.

Same goes for model deterioration - every so often you check that the metric(s) still beat the minimum threshold on more recent data.

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#38

Post by mlthoughts2018 »

inertiatic wrote: post #28
mlthoughts2018 wrote: post #13

Earlier quoted context omitted.

Simple: use mocker.patch to patch out random.randrange and have it return a fixed sequence of results that would imply a known result. Your test should not depend on the internal workings of randrange or anything relying on unfixed random state. Your test is only checking if, given correct results from randrange (or any other external world source of random draws) that the rest of your function correctly produces the…

Mocks should be used when the call you depend on is hard to get to behave in the ways you need for your test. "Randrange" isn't hard to get to behave in the way you want. In fact, the way you want it to behave here is all it does! I'd argue if what you want is to do engineering, mocking this isn't what you'd want to do. You want to ensure your code works. And your test can tell you that it works, even if "randrange"…

No, mocks should be used for any kind of external dependency, whether it is the external OS system calls, built in library functions that interface with sockets, databases, etc. It’s not about whether the resource you are patching is easy / hard to deal with, for example the pytest built in fixture tmpdir abstracts this patching for temporary directories and filesystem ops even though that stuff is “easy” and behaves. Anytime you rely on something outside the runtime environment (even if through a stdlib) you should patch it.

You should have a completely different set of tests (integration tests / end to end tests) that exercises important unmocked validation points. And your test runner should allow you to seamlessly switch between the two sets or combine them.

For example, use @pytest.mark.integration to distinguish all tests needed unmocked dependencies, and have some “integration-tests.ini” config for that.

Then “pytest” runs all the tests with mocks (runs fast, tests logical correctness with tight feedback) and “pytest -c integration-tests.ini” runs all tests or runs the subset requiring real third party resource access. It can run slower, sometimes fail for flaky reasons like network blip, etc.

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#39

Post by mlthoughts2018 »

chriswarbo wrote: post #36
mlthoughts2018 wrote: post #13

Earlier quoted context omitted.

Simple: use mocker.patch to patch out random.randrange and have it return a fixed sequence of results that would imply a known result. Your test should not depend on the internal workings of randrange or anything relying on unfixed random state. Your test is only checking if, given correct results from randrange (or any other external world source of random draws) that the rest of your function correctly produces the…

> Simple: use mocker.patch to patch out random.randrange Why not just pass in 'randrange' as a function argument? def findBias(randrange): heads = tails = 0 for i in range(n): if randrange(100) It's amazing to me how many complicated and convoluted hacks people can come up with, to avoid calling a function with an argument.

Putting randrange as an argument creates deeply worse complexity issues and externalizes a needless burden on your user who must carry around their rand function and must understand what type of return value format the passed-in function has to give in order to be usable in the internals of findBias. It’s shocking to me you think this is somehow simpler or that the testability is somehow better.

This is how nasty convoluted API get created. Now other functions up the call stack will also have to have a randrange param if they have to pass it forward.

At best I could say in some unique use cases there could be API trade offs that make this ok, but definitely not at all for passing around pseudorandom sampler functions. Your change is strictly more confusing, has strictly worse coupling and has a strictly worse API and even after all that, the test is not simpler and isn’t even less code than a one-liner mocker.patch in pytest.

Re: Solving Algorithmic Problems in Python with Pytest (2019)

#40

Post by sn9 »

fnord123 wrote: post #34
tmoertel wrote: post #32

When testing solutions to algorithmic problems, it's often useful to use randomized property checking to verify that the solution's expected properties hold for approximately all inputs. The QuickCheck family of testing tools is probably the best known application of this approach. It's also pretty easy to roll your own. Some hand-rolled examples in Python: https://github.com/tmoertel/practice/blob/master/dailycoding…

Hypohesis is a tool to do this https://github.com/HypothesisWorks/hypothesis/tree/master/hy...

For Python, it's basically the tool as far as I can tell.
Post reply on HN