Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running C unit tests with Pytest

p403n1x87.github.io

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Running C unit tests with Pytest

#31

Post by 10000truths »

slowking2 wrote: post #6
masklinn wrote: post #3

@pytest.fixture def libfact(): yield CDLL("./fact.so") FWIW that's pretty confusing as it gives the impression the library is reloaded for every test, but iirc dlopen() will just return a handle to the existing one. I don't think ctypes has a good way to unload dlls so it should probably be `fixture(scope='session')`. That gets more relevant when on-the-fly compilation is added to the mix, spawning a compiler for eve…

You can use tempfiles and (on OSX) use the private API of ctypes to dlclose to close the handle. Different call on Windows I think. Something like import _ctypes import shutil import tempfile @pytest.fixture def libfact(): tmp = tempfile.NamedTemporaryFile(delete=True) shutil.copy2("./fact.so", tmp.name) lib = CDLL(tmp.name) yield lib _ctypes.dlclose(lib.handle) EDIT: fixed error mentioned in reply.

If you're using Linux, I would recommend using TemporaryFile instead of NamedTemporaryFile. It takes advantage of the O_TMPFILE flag, which guarantees that the kernel will clean up the file for you when the process exits:

  import shutil
  import tempfile

  @pytest.fixture
  def libfact():
      tmp = tempfile.TemporaryFile()
      tmp_name = f"/dev/fd/{tmp.name}"
      shutil.copy2("./fact.so", tmp_name)
      lib = CDLL(tmp_name)
      yield lib
It's better than relying on your application to clean up the file for you. With application-level cleanup hooks, you're still vulnerable to a resource leak if the process gets a SIGKILL or crashes or otherwise ends before your hooks run.

Re: Running C unit tests with Pytest

#32

Post by cozzyd »

wiredfool wrote: post #29
haberman wrote: post #25

A notable downside that is not mentioned: the Python interpreter is far from Valgrind-clean. Valgrind is generally a powerful tool for debugging memory errors, but if you wrap your C code in Python, Valgrind will be so noisy as to be ineffective. Python startup is also very heavyweight; combined with the ~60x slowdown from Valgrind this is something you are going to notice. The Python interpreter does not even use ma…

I’ve had success running valgrind on python code with a very small suppression list to cover all of the python interpreter issues (at least for safety, not leaks) and let me focus on the code.

Is such a suppression list publicly available somewhere?

Re: Running C unit tests with Pytest

#33

Post by [deleted] »

matheusmoreira wrote: post #16
rustybolt wrote: post #4

Shameless self-plug since I recently made a small header-only C testing "framework": https://github.com/rubenvannieuwpoort/c_unit_tests Feels a bit more in line with the spirit of C (small language, few dependencies).

This is really nice. I wonder if there is a way to make this work without constructors.

[deleted]

Re: Running C unit tests with Pytest

#34

Post by bb88 »

drchickensalad wrote: post #21
simonw wrote: post #11

This is brilliant. pytest is by far the most productive testing framework I've ever tried, for any language. It's so good that it switched me from treating tests as a necessary chore to actively enjoying writing them. Using ctypes to exercise a C module like this is brilliant - especially the mechanisms used here to work around segmentation faults. Reminds me of SQLite, which is written in C but uses TCL for most of…

How do you add a side effect and still call the original in pytest? This is easy in rspec but everywhere I google people say there's no good way. For example, you're calling code which internally does Too().bar() and you want to advance your time mock after it's called.

Replace the original function with a wrapper that calls the function then does the side effect.

This could effectively be something like a decorator.

As far as mocking time objects, look into pytest.freezegun. You should be able to control the date and time as you move forward.

I was able to mock out the datetime.now() at some point in the past. It wasn't with freezegun...

Re: Running C unit tests with Pytest

#35

Post by bb88 »

I did something similar with java and jython many years ago.

It was rather nice to write java unit tests with python, because they didn't need to be compiled, they just ran in the interpreter.

Re: Running C unit tests with Pytest

#36

Post by cozzyd »

matheusmoreira wrote: post #16
rustybolt wrote: post #4

Shameless self-plug since I recently made a small header-only C testing "framework": https://github.com/rubenvannieuwpoort/c_unit_tests Feels a bit more in line with the spirit of C (small language, few dependencies).

This is really nice. I wonder if there is a way to make this work without constructors.

Can possibly be done with a custom section, though not sure that is better and might require fancy compiler flags.

Re: Running C unit tests with Pytest

#37

Post by cozzyd »

cozzyd wrote: post #36
matheusmoreira wrote: post #16

Earlier quoted context omitted.

This is really nice. I wonder if there is a way to make this work without constructors.

Can possibly be done with a custom section, though not sure that is better and might require fancy compiler flags.

Indeed, I just made a POC with compiler sections: https://github.com/cozzyd/examc

This implementation only works with gcc though probably (it uses the automatic __start_SECTION and __stop_SECTION that gcc generates but clang doesn't seem to... there are likely hacks to make this work anyway though).

In principle, this approach allows interspersing your tests throughout a shared library instead of all in one file (though in that case you wouldn't want the testing function to be called main and you would need a separate driver program for test).

Re: Running C unit tests with Pytest

#38

Post by Too »

10000truths wrote: post #31
slowking2 wrote: post #6

Earlier quoted context omitted.

You can use tempfiles and (on OSX) use the private API of ctypes to dlclose to close the handle. Different call on Windows I think. Something like import _ctypes import shutil import tempfile @pytest.fixture def libfact(): tmp = tempfile.NamedTemporaryFile(delete=True) shutil.copy2("./fact.so", tmp.name) lib = CDLL(tmp.name) yield lib _ctypes.dlclose(lib.handle) EDIT: fixed error mentioned in reply.

If you're using Linux, I would recommend using TemporaryFile instead of NamedTemporaryFile. It takes advantage of the O_TMPFILE flag, which guarantees that the kernel will clean up the file for you when the process exits: import shutil import tempfile @pytest.fixture def libfact(): tmp = tempfile.TemporaryFile() tmp_name = f"/dev/fd/{tmp.name}" shutil.copy2("./fact.so", tmp_name) lib = CDLL(tmp_name) yield lib It's b…

Does this even work? The documentation states that one should not rely on TemporaryFile having a name. Otherwise why would NamedTemporaryFile exist?

For the rare occurrence of sigkill before cleanup, I think your /tmp has ample of space to keep a few more kb until the next reboot. Servers running tests probably restart more often than desktops nowadays, when wrapped in containers.

Re: Running C unit tests with Pytest

#39

Post by Too »

malkia wrote: post #30

I'm just amazed the lengths people go to complicate debugging (two runtimes), building (two things) and deploying (not single executable). Okay, maybe it's worth it for something (e.g. real "C" code used in Python, but just for the sake of testing... hmmm... no)

Depending on the size of your program, investing in test is something that shouldn’t be underrated. For big projects never call it “just” testing.

This example can be useful to generate rich test-data. Imagine you have some C program parsing json. Generating a sample json input in C would be very long and error prone, whereas in python it’s just a few loops for the generation and finally json.dumps.

This also decouples your test from your program, so you don’t do the same mistake in both of them, negating the test.

In same manner, parameterizing tests is surely possibly with GTest but it’s awkward and complicated, a lot easier in py.

Pytest also comes with lots of fixtures and possibility to make your own, such as spinning up a mock-db, although then we are more into integration testing rather than unit.

Not saying everyone should go rewrite their tests like this, but for some cases there is value that can be utilized.

Post reply on HN