Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modernizing C arrays for greater memory safety: a case study in the Linux kernel

people.kernel.org

Post reply on HN
Print view View original

71–80 of 126 posts

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#71

Post by downvotetruth »

ndesaulniers wrote: post #57
WalterBright wrote: post #51

> int flex[] __attribute__((__element_count__(items))); While what the article describes is clever, it is needlessly complex, and filled with various compiler switches and extensions. In contrast, here's a stupid simple approach: https://www.digitalmars.com/articles/C-biggest-mistake.html where bounds-checkable arrays are declared as: int a[..]; `a` consists of two fields, a `length` and a `pointer`. Indexing it mean…

That's nice, but attributes let us easily retrofit existing C code such as the Linux kernel in a way that supports multiple compilers and compiler versions. Just extensions, not compiler switches. And they don't muck with the ABI which is a requirement for stable kernel driver interfaces. Also what you're proposing...would be an extension!

Starting with "That's nice" is extremely flippant and is an immediate turn off to any subsequent statement. A stronger opening to discredit the [..] proposal would be to take the closing quote "We've been doing that with D for over 20 years" and point out that storing the length of an array of a certain type is a specialization to arrays of dependent typing that has been around since Howard and de Bruijn extended lambda calculus to match predicate logic by creating types for dependent functions and pairs. Each axis of the lambda cube is another source of nondeterminism that has to be reasoned with and languages that explicitly exclude said functionality for simplicity can point to as justification for such restrictions.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#72

Post by ndesaulniers »

WalterBright wrote: post #68
ndesaulniers wrote: post #65

Earlier quoted context omitted.

Take a look at how we use __has_attribute in the kernel. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... That pattern allows us to gracefully support new compiler features optionally and gracefully when people upgrade to toolchains that support them. I suspect with `..` we could do something similar with __has_feature preprocessor guards. Whether it's these extensions or `..`, we still would need to u…

> having a fat pointer decay to a regular pointer _implicitly_ feels like what I never want to happen D doesn't do that implicit decay, but I was thinking of the kernel requirements of no API change. A simple way to convert a phat pointer to a pointer is: &a[0] which is used in D to interface with C code. Note that the syntax still works if `a` is a pointer! Since the pointer phat pointer conversions are trivial oper…

Is https://dlang.org/spec/arrays.html the best place to learn more?

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#73

Post by cozzyd »

neerajsi wrote: post #69
jancsika wrote: post #60

Earlier quoted context omitted.

> yes, it can be implemented as a pointer at the end of the struct, but the extra latency of another pointer chasing can reduce performance For the latency to be significant digits, there must be a lot of repeated calls which only read/write a very small number of elements in the array. Otherwise the accumulation of read/write operations performed when iterating over the data would dwarf the single pointer dereferenc…

I think they were slightly off the mark. The bigger deal is avoiding an extra allocation for an entity that has a variable sized part.

While avoiding the allocation (and potential memory fragmentation) is probably nice, I bet the main benefit is memory locality.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#74

Post by WalterBright »

ndesaulniers wrote: post #72
WalterBright wrote: post #68

Earlier quoted context omitted.

> having a fat pointer decay to a regular pointer _implicitly_ feels like what I never want to happen D doesn't do that implicit decay, but I was thinking of the kernel requirements of no API change. A simple way to convert a phat pointer to a pointer is: &a[0] which is used in D to interface with C code. Note that the syntax still works if `a` is a pointer! Since the pointer phat pointer conversions are trivial oper…

Is https://dlang.org/spec/arrays.html the best place to learn more?

yes

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#75

Post by ndesaulniers »

downvotetruth wrote: post #71
ndesaulniers wrote: post #57

Earlier quoted context omitted.

That's nice, but attributes let us easily retrofit existing C code such as the Linux kernel in a way that supports multiple compilers and compiler versions. Just extensions, not compiler switches. And they don't muck with the ABI which is a requirement for stable kernel driver interfaces. Also what you're proposing...would be an extension!

Starting with "That's nice" is extremely flippant and is an immediate turn off to any subsequent statement. A stronger opening to discredit the [..] proposal would be to take the closing quote "We've been doing that with D for over 20 years" and point out that storing the length of an array of a certain type is a specialization to arrays of dependent typing that has been around since Howard and de Bruijn extended lam…

That's nice, but this is a Wendy's.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#76

Post by WalterBright »

downvotetruth wrote: post #71
ndesaulniers wrote: post #57

Earlier quoted context omitted.

That's nice, but attributes let us easily retrofit existing C code such as the Linux kernel in a way that supports multiple compilers and compiler versions. Just extensions, not compiler switches. And they don't muck with the ABI which is a requirement for stable kernel driver interfaces. Also what you're proposing...would be an extension!

Starting with "That's nice" is extremely flippant and is an immediate turn off to any subsequent statement. A stronger opening to discredit the [..] proposal would be to take the closing quote "We've been doing that with D for over 20 years" and point out that storing the length of an array of a certain type is a specialization to arrays of dependent typing that has been around since Howard and de Bruijn extended lam…

I didn't say D invented it, so no need to discredit it. I said D has been using it for 20 years and proves it works famously with C style pointers and arrays.

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#77

Post by cryptonector »

IMO the right approach is to start with counted array struct wrapper types like `struct array_of_xyz { unsigned count; xyz a[1]; };` and use them to hold and pass by reference. When the array sizes are fixed, then use `struct array5_of_xyz { xyz a[5]; };` and pass by reference or by value as needed. Add to this a decoration to indicate that the `count` field is a count of the number of elements in the array and now the compiler can do bounds checking.

Then fix codebases recursively until it's all ok. At ABI boundaries that don't use such types create values of such types corresponding to the given arguments (e.g., you could count the elements of `argv[]` then create a wrapper for the `argv`).

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#78

Post by jenadine »

jcalvinowens wrote: post #30
segfaultbuserr wrote: post #23

For code that is critical to performance, C99's "flexible array at the end of a struct" is an useful tool. It basically allows you to attach a header at the beginning of some dynamically-allocated binary data of infinite length (yes, it can be implemented as a pointer at the end of the struct, but the extra latency of another pointer chasing can reduce performance). Before C99, the "size-1 hack" or "size-0 GCC extens…

> Does modern C++ have a better solution to implement the same thing? I'm no guru, but I know from experience you can do it in C++: {0}[calvin ~] cat test.cpp #include #include struct foo { int len; int v[]; }; int main(void) { auto p = std::unique_ptr (reinterpret_cast ( malloc(sizeof(struct foo) + sizeof(int) * 2))); p->v[1] = 99; std::cerr v[1] EDIT: Remove unnecessary extern block, as pointed out by wahern in the…

It compiles and work doesn't mean it is not undefined behaviour. You should at least try with the ub sanitizer.

But in this case, as other said, it may be accepted as an extension. Still not part of C++

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#79

Post by [deleted] »

segfaultbuserr wrote: post #23

For code that is critical to performance, C99's "flexible array at the end of a struct" is an useful tool. It basically allows you to attach a header at the beginning of some dynamically-allocated binary data of infinite length (yes, it can be implemented as a pointer at the end of the struct, but the extra latency of another pointer chasing can reduce performance). Before C99, the "size-1 hack" or "size-0 GCC extens…

[deleted]

Re: Modernizing C arrays for greater memory safety: a case study in the Linux kernel

#80

Post by ykonstant »

ddtaylor wrote: post #37
monocasa wrote: post #5

Earlier quoted context omitted.

I think the parent is talking about the c pattern of having the last member of a struct be a zero length array, which is actually a dynamically sized array that the struct is only the header to (ostensibly with another field of the struct specifying the length of the array). It's fallen a bit out of favor, but it is a handy way to commingle the header and array with one allocation/pointer. And interestingly COBOL han…

> And interestingly COBOL handled this in a cleaner way That is the least reassuring sentence I have read on HN in a while!

ROFL I can use that.

- Boss!

- ..yes?!

- I was reading a COBOL code base and got a brilliant idea--

- Say no more, you need a vacation. I am sorry, I have been too pushy. Take 3 weeks; just promise me one thing: no COBOL.

Post reply on HN