Running C unit tests with Pytest • Hacker News Board

Re: Running C unit tests with Pytest

#21

Post by drchickensalad » Sat, Feb 12, 2022, 7:06 PM UTC

simonw wrote: post #11

This is brilliant. pytest is by far the most productive testing framework I've ever tried, for any language. It's so good that it switched me from treating tests as a necessary chore to actively enjoying writing them. Using ctypes to exercise a C module like this is brilliant - especially the mechanisms used here to work around segmentation faults. Reminds me of SQLite, which is written in C but uses TCL for most of…

How do you add a side effect and still call the original in pytest? This is easy in rspec but everywhere I google people say there's no good way.

For example, you're calling code which internally does Too().bar() and you want to advance your time mock after it's called.

Re: Running C unit tests with Pytest

#22

Post by masklinn » Sat, Feb 12, 2022, 7:15 PM UTC

drchickensalad wrote: post #21

simonw wrote: post #11

This is brilliant. pytest is by far the most productive testing framework I've ever tried, for any language. It's so good that it switched me from treating tests as a necessary chore to actively enjoying writing them. Using ctypes to exercise a C module like this is brilliant - especially the mechanisms used here to work around segmentation faults. Reminds me of SQLite, which is written in C but uses TCL for most of…

How do you add a side effect and still call the original in pytest? This is easy in rspec but everywhere I google people say there's no good way. For example, you're calling code which internally does Too().bar() and you want to advance your time mock after it's called.

That… has nothing to do with pytest?

You’d probably use something like `wraps` or `side_effect` in `unittest.mock` to delegate to the original with some extra behaviours.

And / or use freezegun’s tick features, when you’re specifically dealing with time.

Re: Running C unit tests with Pytest

#23

Post by simonw » Sat, Feb 12, 2022, 7:24 PM UTC

drchickensalad wrote: post #21

simonw wrote: post #11

This is brilliant. pytest is by far the most productive testing framework I've ever tried, for any language. It's so good that it switched me from treating tests as a necessary chore to actively enjoying writing them. Using ctypes to exercise a C module like this is brilliant - especially the mechanisms used here to work around segmentation faults. Reminds me of SQLite, which is written in C but uses TCL for most of…

How do you add a side effect and still call the original in pytest? This is easy in rspec but everywhere I google people say there's no good way. For example, you're calling code which internally does Too().bar() and you want to advance your time mock after it's called.

There are lots of good mocking fixtures available for pytest. I've used this one for clock stuff in the past: https://github.com/adamchainz/time-machine#pytest-plugin

Re: Running C unit tests with Pytest

#24

Post by ereyes01 » Sat, Feb 12, 2022, 7:31 PM UTC

Really cool. Several jobs ago, I used cffi to create bindings to a library that controlled a camera's pan/tilt/zoom motors. Those were used to implement a test suite that validates the cameras in the manufacturing facility before shipping. The embedded developers also found being able to use the cffi bindings in the REPL really useful when prototyping changes. Python is a really useful tool for these kinds of interfaces.

Re: Running C unit tests with Pytest

#25

Post by haberman » Sat, Feb 12, 2022, 7:38 PM UTC

A notable downside that is not mentioned: the Python interpreter is far from Valgrind-clean. Valgrind is generally a powerful tool for debugging memory errors, but if you wrap your C code in Python, Valgrind will be so noisy as to be ineffective. Python startup is also very heavyweight; combined with the ~60x slowdown from Valgrind this is something you are going to notice.

The Python interpreter does not even use malloc()/free() directly by default; it layers its own memory allocator on top called PyMalloc (https://docs.python.org/3/c-api/memory.html#pymalloc). You can disable this by setting an environment variable (PYTHONMALLOC=malloc), but even then you will see many Valgrind warnings inside the Python interpreter itself.

Re: Running C unit tests with Pytest

#26

Post by p403n1x87 » Sat, Feb 12, 2022, 7:46 PM UTC

haberman wrote: post #25

A notable downside that is not mentioned: the Python interpreter is far from Valgrind-clean. Valgrind is generally a powerful tool for debugging memory errors, but if you wrap your C code in Python, Valgrind will be so noisy as to be ineffective. Python startup is also very heavyweight; combined with the ~60x slowdown from Valgrind this is something you are going to notice. The Python interpreter does not even use ma…

That's a good argument for keeping these kind of tests separate. Indeed Austin has dedicated Valgrind integration tests just for this reason.

Re: Running C unit tests with Pytest

#27

Post by pkhuong » Sat, Feb 12, 2022, 7:51 PM UTC

jaromir_ wrote: post #17

pkhuong wrote: post #12

We've had a lot of success combining that approach (cffi instead of ctypes) with property-based testing ( https://github.com/HypothesisWorks/hypothesis ) for the query engine at backtrace: https://engineering.backtrace.io/2020-03-11-how-hard-is-it-t... .

Have you exposed the C library to Python via ctypes as OP or have you taken a different approach?

Just cffi, and keeping the headers free of inline implementation noise (a side benefit, in my opinion).

Re: Running C unit tests with Pytest

#28

Post by mhh__ » Sat, Feb 12, 2022, 8:10 PM UTC

When I need to test C I just use D unittest blocks.

You can import the C header file directly now so it's literally trivial.

Re: Running C unit tests with Pytest

#29

Post by wiredfool » Sat, Feb 12, 2022, 11:27 PM UTC

haberman wrote: post #25

A notable downside that is not mentioned: the Python interpreter is far from Valgrind-clean. Valgrind is generally a powerful tool for debugging memory errors, but if you wrap your C code in Python, Valgrind will be so noisy as to be ineffective. Python startup is also very heavyweight; combined with the ~60x slowdown from Valgrind this is something you are going to notice. The Python interpreter does not even use ma…

I’ve had success running valgrind on python code with a very small suppression list to cover all of the python interpreter issues (at least for safety, not leaks) and let me focus on the code.

Re: Running C unit tests with Pytest

#30

Post by malkia » Sat, Feb 12, 2022, 11:58 PM UTC

I'm just amazed the lengths people go to complicate debugging (two runtimes), building (two things) and deploying (not single executable). Okay, maybe it's worth it for something (e.g. real "C" code used in Python, but just for the sake of testing... hmmm... no)