[dead]
Modernizing C arrays for greater memory safety: a case study in the Linux kernel
61–70 of 126 posts
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#62Earlier quoted context omitted.
Yes, you can do this in C++, but keep in mind that implicitly generated copy/move constructors don't understand this and will not copy the full object. This can produce surprising memory corruption that can be difficult to debug. So you should be sure to either explicitly mark the struct not copy/movable, or implement smarter copy/move operators.
Tbf I don't think C copies the whole flexible array member either[0]. So the care you would normally take while using a flexible array member in C should also be taken in C++ :) [0]: https://stackoverflow.com/questions/35423293/flexible-array-...
Right. But C has fewer scenarios where implicit copies are possible, in part because it does not have references.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#63Earlier quoted context omitted.
That's nice, but attributes let us easily retrofit existing C code such as the Linux kernel in a way that supports multiple compilers and compiler versions. Just extensions, not compiler switches. And they don't muck with the ABI which is a requirement for stable kernel driver interfaces. Also what you're proposing...would be an extension!
Reading the article, it doesn't look easy at all. With the [..] proposal, it is easy enough to convert it back and forth between pointers and [..] to conform to required interfaces. One could even make the [..] implicitly convertible to a pointer.
Most of the article is devoted to complications related to handling arrays that are inline at the tail of existing structs that usually are layout and size sensitive. Hence the related two restrictions that the count variable keep the same name and location in the struct, and there is no explicit array head pointer (just the implicit location at which the array starts at the end of the struct).
The reasoning most likely being that a bunch of annotations to structs, and perhaps some changes to calls to kmalloc() would be less destructive and much simpler than breaking the kernel ABI and altering the base size of any struct that used that idiom while also having to change every for loop or whatnot in the kernel that uses an explicit counter member name.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#64> C is not just a fancy assembler any more I wish this trope would die. It really never was one.
In what sense was C never a fancy assembler? I am not an expert on C nor assembly and would be curious if you could expand on this. The statement makes sense to me because my impression is that most of what happens in C code gets translated fairly straightforwardly to machine code, with the compiler taking care of bridging differences in the instruction sets of targeted architectures. I guess the reason this is simpl…
The basic problem with this assumption is that people who follow it tend to get it in their heads that since C is merely "translat[ing] fairly straightforwardly to machine code", they assume they can rely on the semantics of the machine code being the semantics of their C program. That isn't true, and hasn't been for a long time [1]: compilers are only required to uphold the looser semantics of C, and they will happily apply optimizations that deviate from the semantics of a purported naive translation to machine code. The usual example brought out to explore this difference is signed integer overflow, which has nothing to do with inlining or loop unrolling.
[1] I don't know enough about the early history of C to be able to assert that it was never true, but it certainly hasn't been true since at least 1989.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#65Earlier quoted context omitted.
That's nice, but attributes let us easily retrofit existing C code such as the Linux kernel in a way that supports multiple compilers and compiler versions. Just extensions, not compiler switches. And they don't muck with the ABI which is a requirement for stable kernel driver interfaces. Also what you're proposing...would be an extension!
Reading the article, it doesn't look easy at all. With the [..] proposal, it is easy enough to convert it back and forth between pointers and [..] to conform to required interfaces. One could even make the [..] implicitly convertible to a pointer.
Take a look at how we use __has_attribute in the kernel.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
That pattern allows us to gracefully support new compiler features optionally and gracefully when people upgrade to toolchains that support them.
I suspect with `..` we could do something similar with __has_feature preprocessor guards.
Whether it's these extensions or `..`, we still would need to update struct definitions. That's at least the same amount of work. If we need to update member names due to `..`, then that's even more work!
That said, the two production compilers that can build the Linux kernel already support member attributes, and compiler inserted bounds checks against immediates. It's trivial to key off an attribute to change the inserted bounds check to a runtime load of a the corresponding member (and maybe a min between that and a fixed length, for non-flexible arrays that may not have been fully initialized for instance) and check against that. With `..` I would need to add new tokens, parsing (both for the declaration, slicing, and implicit conversions), and then the codegen.
There are times where we don't care about ABI within the kernel, so I do think it makes sense to have two different extensions here (implicit fat pointers and non-ABI modifying ways of denoting existing struct members are meant to be runtime bounds). Deploying the correct variant will take careful thought I suspect.
I haven't put enough thought into converting to/from fat pointers, so I'm glad you brought that up. It's something I'll have to think about more.
My initial gut reaction to implicit conversions is that C has enough wretched implicit conversions and promotions that are error prone, but perhaps if it is such an ergonomics win...though having a fat pointer decay to a regular pointer _implicitly_ feels like what I never want to happen.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#66Earlier quoted context omitted.
malloc() doesn't allocate arrays. It allocates blocks of memory. Hence sizeof doesn't work the same for malloc() objects as it does on arrays.
sizeof doesn't work the same for malloc() because the type of the returned value is a pointer, and the behavior of sizeof is dependent solely on the static type. For comparison, calloc() is specifically defined as "allocates space for an array of ... objects" in the Standard, but since return type is still void*, the caveat with sizeof still applies.
calloc() doesn't allocate an array either. It's purpose is to allocate blocks of memory larger than SIZE_MAX. Mostly irrelevant now but was an issue on 16-bit systems.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#67Earlier quoted context omitted.
> Does modern C++ have a better solution to implement the same thing? I'm no guru, but I know from experience you can do it in C++: {0}[calvin ~] cat test.cpp #include #include struct foo { int len; int v[]; }; int main(void) { auto p = std::unique_ptr (reinterpret_cast ( malloc(sizeof(struct foo) + sizeof(int) * 2))); p->v[1] = 99; std::cerr v[1] EDIT: Remove unnecessary extern block, as pointed out by wahern in the…
Flexible array members are commonly supported as an extension in C++ compilers, but the C++ standard itself does not permit them. And FWIW `extern "C"` doesn't drop a C++ compiler into a "C mode", it merely effects linkage (i.e. name mangling); all the code within the scope must still be valid, compiler-supported C++. Your example code compiles the same without the extern "C" declaration. And it compiles the same if…
Unrelated to the article but since you seem to know this stuff, do you know what happens when a function in an “extern C” block exposes C++ types (parameters or return types)?
I assume it’s broken because it’d expect the actual C++ types to be passed in despite the lack of ABI stability, and only the linking / name mangling is affected (possibly calling conventions as well?) but I’m not actually sure.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#68Earlier quoted context omitted.
Reading the article, it doesn't look easy at all. With the [..] proposal, it is easy enough to convert it back and forth between pointers and [..] to conform to required interfaces. One could even make the [..] implicitly convertible to a pointer.
Take a look at how we use __has_attribute in the kernel. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... That pattern allows us to gracefully support new compiler features optionally and gracefully when people upgrade to toolchains that support them. I suspect with `..` we could do something similar with __has_feature preprocessor guards. Whether it's these extensions or `..`, we still would need to u…
> having a fat pointer decay to a regular pointer _implicitly_ feels like what I never want to happen
D doesn't do that implicit decay, but I was thinking of the kernel requirements of no API change. A simple way to convert a phat pointer to a pointer is:
&a[0]
Since the pointer phat pointer conversions are trivial operations, one can easily go back and forth between them depending on what part of the code one wants the overflow checks on vs the legacy interfaces.
With a couple of C macros one could turn on/off declaring arrays as pointers or phat pointers, and turn on/off the slicing code. In this way it will still compile with old compilers.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#69For code that is critical to performance, C99's "flexible array at the end of a struct" is an useful tool. It basically allows you to attach a header at the beginning of some dynamically-allocated binary data of infinite length (yes, it can be implemented as a pointer at the end of the struct, but the extra latency of another pointer chasing can reduce performance). Before C99, the "size-1 hack" or "size-0 GCC extens…
> yes, it can be implemented as a pointer at the end of the struct, but the extra latency of another pointer chasing can reduce performance For the latency to be significant digits, there must be a lot of repeated calls which only read/write a very small number of elements in the array. Otherwise the accumulation of read/write operations performed when iterating over the data would dwarf the single pointer dereferenc…
I think they were slightly off the mark. The bigger deal is avoiding an extra allocation for an entity that has a variable sized part.
Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel
#70Earlier quoted context omitted.
sizeof doesn't work the same for malloc() because the type of the returned value is a pointer, and the behavior of sizeof is dependent solely on the static type. For comparison, calloc() is specifically defined as "allocates space for an array of ... objects" in the Standard, but since return type is still void*, the caveat with sizeof still applies.
calloc() doesn't allocate an array either. It's purpose is to allocate blocks of memory larger than SIZE_MAX. Mostly irrelevant now but was an issue on 16-bit systems.
I literally quoted the Standard (C17 7.22.3.2) in my earlier comment, and it very specifically says that calloc allocates an array. This language isn't from a new standard, either - it goes all the way back to ISO C90.