Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modernizing C arrays for greater memory safety: a case study in the Linux kernel

people.kernel.org

Post reply on HN
Print view View original

51–60 of 126 posts

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#51

Post by WalterBright »

> int flex[] __attribute__((__element_count__(items)));

While what the article describes is clever, it is needlessly complex, and filled with various compiler switches and extensions.

In contrast, here's a stupid simple approach:

https://www.digitalmars.com/articles/C-biggest-mistake.html

where bounds-checkable arrays are declared as:

    int a[..];
`a` consists of two fields, a `length` and a `pointer`. Indexing it means the compiler can (optionally) insert a bounds check it.

    int s[..] = "string";
    s[10] = 'x'; // fatal runtime error
We can turn a pointer into a bounds checked array by "slicing" it:

    int *p = (int*) malloc(10);
    int a[..] = p[0 .. 10];
A bounds checked array can be turned into a pointer:

    int *p = &a[3];  // point to 3rd element of a[..]
That's all there is to it. No pages and pages of compiler switches and extensions.

Does it work? We've been doing that with D for over 20 years. Hell yeah, it works. It works fantastically well. It does not disturb any existing C code.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#52

Post by jcalvinowens »

wahern wrote: post #43
jcalvinowens wrote: post #30

Earlier quoted context omitted.

> Does modern C++ have a better solution to implement the same thing? I'm no guru, but I know from experience you can do it in C++: {0}[calvin ~] cat test.cpp #include #include struct foo { int len; int v[]; }; int main(void) { auto p = std::unique_ptr (reinterpret_cast ( malloc(sizeof(struct foo) + sizeof(int) * 2))); p->v[1] = 99; std::cerr v[1] EDIT: Remove unnecessary extern block, as pointed out by wahern in the…

Flexible array members are commonly supported as an extension in C++ compilers, but the C++ standard itself does not permit them. And FWIW `extern "C"` doesn't drop a C++ compiler into a "C mode", it merely effects linkage (i.e. name mangling); all the code within the scope must still be valid, compiler-supported C++. Your example code compiles the same without the extern "C" declaration. And it compiles the same if…

D'oh, thanks for pointing that out.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#53

Post by andreareina »

alerighi wrote: post #34
fsckboy wrote: post #15

Earlier quoted context omitted.

you should be specific about which C spec you are referring to, you talk about "the C spec" and then you mention the "pre-C99 implementation". Maybe you mean they've always been illegal in every version, but it would be more clear.

In reality, who cares of what the C spec says? Unless you have to port code on different compilers (which is very unlikely, unless you are building a library meant to be shared with different projects) you only care about the fact that the code works correctly with the compiler you choose to use. I don't get all the programmers that scandalize if you use GNU extensions, they are fine, and mostly useful, so if you are…

It’s all well and good until someone else wants to compile it on clang[1]. There’s a lot of value in C being a lingua franca of sorts, and that depends on a cross-implementation understanding of semantics, i.e. standards, (though that doesn’t have to be from whatever official body that produced C99 etc).

[1] IIRC on OSX[2] gcc actually invokes clang

[2] stopped upgrading before it became macOS

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#54

Post by wahern »

ntrz wrote: post #14
zabzonk wrote: post #3

> Is it actually a 4 element array, or is it sized by the bytes member? i give up, what does sizeof say? and why would it be sized by bytes?

The previous paragraph says > ...due to yet more historical situations (e.g. struct sockaddr, which has a fixed-size trailing array that is not supposed to actually be treated as fixed-size), GCC and Clang actually treat all trailing arrays as flexible arrays. But I don't know, that doesn't seem to match the result I am getting with clang 13.1.6. It does seem to respect the array size declared in the struct, not trea…

It treats them as flexible arrays in the sense that it doesn't assume indexing beyond the declared size is undefined behavior, which would have implications for code elision and other optimizations.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#55

Post by astrange »

andreareina wrote: post #53
alerighi wrote: post #34

Earlier quoted context omitted.

In reality, who cares of what the C spec says? Unless you have to port code on different compilers (which is very unlikely, unless you are building a library meant to be shared with different projects) you only care about the fact that the code works correctly with the compiler you choose to use. I don't get all the programmers that scandalize if you use GNU extensions, they are fine, and mostly useful, so if you are…

It’s all well and good until someone else wants to compile it on clang[1]. There’s a lot of value in C being a lingua franca of sorts, and that depends on a cross-implementation understanding of semantics, i.e. standards, (though that doesn’t have to be from whatever official body that produced C99 etc). [1] IIRC on OSX[2] gcc actually invokes clang [2] stopped upgrading before it became macOS

Clang implements nearly all GCC extensions. (No nested functions.)

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#56

Post by astrange »

nequo wrote: post #31
chungy wrote: post #21

> C is not just a fancy assembler any more I wish this trope would die. It really never was one.

In what sense was C never a fancy assembler? I am not an expert on C nor assembly and would be curious if you could expand on this. The statement makes sense to me because my impression is that most of what happens in C code gets translated fairly straightforwardly to machine code, with the compiler taking care of bridging differences in the instruction sets of targeted architectures. I guess the reason this is simpl…

C is a language with a specification which defines it in terms of a virtual machine, not translation to machine code. The memory model is also totally different and it has lots of undefined behavior.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#57

Post by ndesaulniers »

WalterBright wrote: post #51

> int flex[] __attribute__((__element_count__(items))); While what the article describes is clever, it is needlessly complex, and filled with various compiler switches and extensions. In contrast, here's a stupid simple approach: https://www.digitalmars.com/articles/C-biggest-mistake.html where bounds-checkable arrays are declared as: int a[..]; `a` consists of two fields, a `length` and a `pointer`. Indexing it mean…

That's nice, but attributes let us easily retrofit existing C code such as the Linux kernel in a way that supports multiple compilers and compiler versions. Just extensions, not compiler switches. And they don't muck with the ABI which is a requirement for stable kernel driver interfaces.

Also what you're proposing...would be an extension!

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#58

Post by WalterBright »

ndesaulniers wrote: post #57
WalterBright wrote: post #51

> int flex[] __attribute__((__element_count__(items))); While what the article describes is clever, it is needlessly complex, and filled with various compiler switches and extensions. In contrast, here's a stupid simple approach: https://www.digitalmars.com/articles/C-biggest-mistake.html where bounds-checkable arrays are declared as: int a[..]; `a` consists of two fields, a `length` and a `pointer`. Indexing it mean…

That's nice, but attributes let us easily retrofit existing C code such as the Linux kernel in a way that supports multiple compilers and compiler versions. Just extensions, not compiler switches. And they don't muck with the ABI which is a requirement for stable kernel driver interfaces. Also what you're proposing...would be an extension!

Reading the article, it doesn't look easy at all.

With the [..] proposal, it is easy enough to convert it back and forth between pointers and [..] to conform to required interfaces. One could even make the [..] implicitly convertible to a pointer.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#59

Post by _gabe_ »

loeg wrote: post #36
jcalvinowens wrote: post #30

Earlier quoted context omitted.

> Does modern C++ have a better solution to implement the same thing? I'm no guru, but I know from experience you can do it in C++: {0}[calvin ~] cat test.cpp #include #include struct foo { int len; int v[]; }; int main(void) { auto p = std::unique_ptr (reinterpret_cast ( malloc(sizeof(struct foo) + sizeof(int) * 2))); p->v[1] = 99; std::cerr v[1] EDIT: Remove unnecessary extern block, as pointed out by wahern in the…

Yes, you can do this in C++, but keep in mind that implicitly generated copy/move constructors don't understand this and will not copy the full object. This can produce surprising memory corruption that can be difficult to debug. So you should be sure to either explicitly mark the struct not copy/movable, or implement smarter copy/move operators.

Tbf I don't think C copies the whole flexible array member either[0]. So the care you would normally take while using a flexible array member in C should also be taken in C++ :)

[0]: https://stackoverflow.com/questions/35423293/flexible-array-...

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#60

Post by jancsika »

segfaultbuserr wrote: post #23

For code that is critical to performance, C99's "flexible array at the end of a struct" is an useful tool. It basically allows you to attach a header at the beginning of some dynamically-allocated binary data of infinite length (yes, it can be implemented as a pointer at the end of the struct, but the extra latency of another pointer chasing can reduce performance). Before C99, the "size-1 hack" or "size-0 GCC extens…

> yes, it can be implemented as a pointer at the end of the struct, but the extra latency of another pointer chasing can reduce performance

For the latency to be significant digits, there must be a lot of repeated calls which only read/write a very small number of elements in the array. Otherwise the accumulation of read/write operations performed when iterating over the data would dwarf the single pointer dereference.

So I'm curious now-- do OS kernels spend most of their time doing lots of calls that dive into such dynamically-allocated data just to extract a single datum or two?

Post reply on HN