Oh my! I though they were doing TDD :)
How is the Linux kernel tested?
31–40 of 59 posts
Re: How is the Linux kernel tested?
#32Earlier quoted context omitted.
I think the main reason is that the Linux kernel (and similarly the *BSD kernels) are written in a programming language (C) that doesn't make it easy to do that. Code is often directly built on top of other kernel subsystems without any dependency injection whatsoever. This means that it's still possible to do unit testing of parts of the kernel, but it takes a crazy amount of effort, such as overriding symbols, over…
You don’t need a dependency injection framework to write unit tests, you just need cleanly separable units with well defined interfaces.
And, in the absence of a dependency injection framework, it's likely that the units are not cleanly separable - because, without a DI framework, all classes are (presumably?) instantiating their dependencies directly.
Unless I've missed something? I've only ever worked in Java so maybe things are different in C-world,
Re: How is the Linux kernel tested?
#33Earlier quoted context omitted.
You don’t need a dependency injection framework to write unit tests, you just need cleanly separable units with well defined interfaces.
And, in the absence of a dependency injection framework, it's likely that the units are not cleanly separable - because, without a DI framework, all classes are (presumably?) instantiating their dependencies directly. Unless I've missed something? I've only ever worked in Java so maybe things are different in C-world,
> without a DI framework, all classes are (presumably?) instantiating their dependencies directly. ... I've only ever worked in Java so maybe things are different in C-world,
Well, for one thing, there are no classes in C. :) It is possible but unfun to emulate them with function pointers. Iiuc, little of the Linux kernel is written in that style.
Also, FYI, for many years we did DI without frameworks, using the factory pattern and other techniques. It wasn't always fun but it can certainly be done without Spring or whatever the new thing on the block is.
Re: How is the Linux kernel tested?
#34Earlier quoted context omitted.
so you have a very well tested boot sequence - is this not a slightly better kernel equivalent of 'it builds, ship it' ?
> a slightly better kernel equivalent of 'it builds, ship it' ? Surely, "it runs , ship it"? That seems quite a bit better.
The grand majority of code in the Linux kernel will never be hit by booting it in a QEMU with init=/bin/shutdown. The pure line coverage of these builds is probably like 5 or 10 %.
Re: How is the Linux kernel tested?
#35Earlier quoted context omitted.
And, in the absence of a dependency injection framework, it's likely that the units are not cleanly separable - because, without a DI framework, all classes are (presumably?) instantiating their dependencies directly. Unless I've missed something? I've only ever worked in Java so maybe things are different in C-world,
> without a DI framework, all classes are (presumably?) instantiating their dependencies directly. ... I've only ever worked in Java so maybe things are different in C-world, Well, for one thing, there are no classes in C. :) It is possible but unfun to emulate them with function pointers. Iiuc, little of the Linux kernel is written in that style. Also, FYI, for many years we did DI without frameworks, using the fact…
> Well, for one thing, there are no classes in C. :) It is possible but unfun to emulate them with function pointers. Iiuc, little of the Linux kernel is written in that style.
object-structs with function-pointers-for-methods are super-common in the Linux kernel and basically used everywhere for everything where modules can plug something into the kernel (e.g. virtually all drivers have at least one of these).
Re: How is the Linux kernel tested?
#36Why is there such little emphasis on “traditional” testing, that is, regular unit tests? At least some portion of the code base is surely suitable for normal unit tests. For example data structures, scheduling algorithms, file systems, ...
I think the main reason is that the Linux kernel (and similarly the *BSD kernels) are written in a programming language (C) that doesn't make it easy to do that. Code is often directly built on top of other kernel subsystems without any dependency injection whatsoever. This means that it's still possible to do unit testing of parts of the kernel, but it takes a crazy amount of effort, such as overriding symbols, over…
> it takes a crazy amount of effort
> it's also possible to have dependency injection in C by using structs with function
> pointers, but I think we can all agree that it's a lot less pleasant to use than C++
> abstract base classes
If you want to sort an array:
int arr[] = {5, 10, 15, 17, 20};
size_t size = sizeof(arr) / sizeof(*arr);
QSORT(arr, size, arr[a] ssize_t index;
BSEARCH_INDEX(index, size, arr[index] - 5);
// Now 'index' has the result.Re: How is the Linux kernel tested?
#37Why is there such little emphasis on “traditional” testing, that is, regular unit tests? At least some portion of the code base is surely suitable for normal unit tests. For example data structures, scheduling algorithms, file systems, ...
How do I test a driver for which I do not possess the hardware?
You test the logic in unit tests with any hardware interaction mocked. Even if you can't test a lot of the driver, there is surely some logic (data structure manipulation, buffer construction, etc) that you have factored out into testable functions.
Re: How is the Linux kernel tested?
#38Earlier quoted context omitted.
You could test data structures (which tend to be implemented using macros), but any nontrivial subsystem of a monolithic kernel lives in a web of dependencies with other subsystems. This is especially true of Linux. To solve this, you would need to either use a hierarchical decomposition of subsystems or do some crazy mocking to run subsystems outside of the full kernel. There is a recent project (KUnit) to add a uni…
NetBSD has their rump kernel tech, which specifically exists to run chunks of kernel code independently. It can do a lot of neat things (I liked the "run netbsd drivers on other OSs" trick, personally), but one of the big uses they've mentioned is that it helps testing and development.
A lot of the linux kernel is implemented as modules, which does allow said modules to be run under different OSes. You could probably use the module interface to implement unit testing.
Re: How is the Linux kernel tested?
#39Long way of saying "it's not".
It is, by everyone who uses it every day.
Re: How is the Linux kernel tested?
#40> Another major challenge is to automate device drivers and hardware platform tests because they require the hardware to be tested.
Anyone have any thoughts on how to automate testing of coupled hardware-software systems? This is really hard; we've attacked it in the past by writing hardware simulators in accordance with the ICD. This falls flat once you find that the hardware doesn't precisely match the ICD, and usually it's much cheaper to change the software than the hardware. And at that point, the simulator hasn't actually helped you at all.
I recently built a system which involved several tightly coupled hardware components and we fought many bugs on a tight schedule. It would have been nice to find a good way to think about this beyond the basic hardware-in-the-loop manual testing.