Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Passing nothing is surprisingly difficult

davidben.net

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Passing nothing is surprisingly difficult

#71

Post by matheusmoreira »

JonChesterfield wrote: post #55
pizlonator wrote: post #54

It’s so silly to talk about C not allowing null on memcpy. That’s a thing the spec says, I guess? The solution is clear: just ignore the C spec. It’s total garbage. Of course you can memcpy between any ptr values if the count is zero and those ptr values don’t have to point to anything.

Better be rolling your own compiler in that case. Or your own memcpy with a different name. UB to pass memcpy to null means after that call, the pointer is assumed to be non-null. So if(ptr) can constant fold. Maybe faster. I'm in agreement with you on this but your compiler probably isn't.

> Better be rolling your own compiler in that case.

No need for that.

> the pointer is assumed to be non-null

Just give us an option to tell the compiler to stop assuming nonsense like that. I'm gonna make it standard on my makefiles just like -fno-strict-aliasing and -fwrapv.

There's no use trying to work around C standard problems. Compilers should just be told to define the undefined and to disable everything that can't be defined. Then we can write code on solid foundations instead of quicksand.

> Or your own memcpy with a different name.

I wish. I couldn't escape that function even on my freestanding nolibc project. The compilers will happily emit calls to memcpy and memset all by themselves whenever they feel like it and god help you if you don't provide them because for some reason this nonsense can't be disabled.

Re: Passing nothing is surprisingly difficult

#72

Post by swiftcoder »

cozzyd wrote: post #61
swiftcoder wrote: post #46

It's obviously too late to change this in Rust's case, but I wonder whether being able to differentiate between None and the empty slice is actually a necessary property in general? There are a bunch of languages where empty arrays are "falsy", and in those it's not recommendable to use the two to differentiate valid states. Feels like the same could apply here

I thought the justification for Rust having an unstable ABI is so such things like binary representations of types could change?

The binary representation could change, yes, but collapsing two distinct types into one would cause actual logic errors in existing code (i.e. this is an API change, not an ABI change)

Re: Passing nothing is surprisingly difficult

#73

Post by swiftcoder »

tialaramex wrote: post #53
swiftcoder wrote: post #46

It's obviously too late to change this in Rust's case, but I wonder whether being able to differentiate between None and the empty slice is actually a necessary property in general? There are a bunch of languages where empty arrays are "falsy", and in those it's not recommendable to use the two to differentiate valid states. Feels like the same could apply here

The main complaint in the post is basically that Rust's actual bona fide slice type doesn't work the way cobbled together library types for this purpose in C or C++ do. The C++ type discussed is much newer than Rust (std::span was standardized in C++ 20). Yes in many cases what C++ APIs mean here isn't a slice of zero Ts at all but instead None, and Rust has an appropriate type for that Option which works as expected…

I'm guess I'm inclined to go the other way. I tend to object to wrapping arrays in Option, because while semantically similar, the empty slice supports the full set of array APIs, whereas Option requires unwrapping

Re: Passing nothing is surprisingly difficult

#74

Post by torstenvl »

kazinator wrote: post #64
torstenvl wrote: post #58

Earlier quoted context omitted.

I literally quoted someone from WG14. HN really needs the ability to block trolls.

You quoted someone who stated they are not happy with C23, and deflected personal blame for that issue. If I'm only trolling, then why, having learned about this, am I having to go into code and make defensive fixes?

They aren't defensive foxes. They're just fixes. Any C99+ code that relied on your assumed realloc(p, 0) behavior was always incorrect.

Re: Passing nothing is surprisingly difficult

#75

Post by kazinator »

torstenvl wrote: post #74
kazinator wrote: post #64

Earlier quoted context omitted.

You quoted someone who stated they are not happy with C23, and deflected personal blame for that issue. If I'm only trolling, then why, having learned about this, am I having to go into code and make defensive fixes?

They aren't defensive foxes. They're just fixes. Any C99+ code that relied on your assumed realloc(p, 0) behavior was always incorrect.

Yes, a fix inside a realloc wrapper is a defensive fix, if nothing is actually calling with the zero size in the current code base, and/or we are not yet compiling with C dialect selection that corresponds to the vandalized standard. It's an example of defensive programming, coding intended to anticipate and thwart a future problem.

Code that calls realloc(p, 0) and does not assume any one of the behaviors that are described is not incorrect according to C99.

That's par for the course in C. In C, there are situations left and right in which you're stepping on implementation defined behavior and mustn't assume a particular one.

A lot of portable coding is defensive. It can end up more portable than required. That is to say, if we consider all the platforms but the program actually ends up used over its lifetime, we can deduce that a less portable approach in some part of the code would have worked just fine.

Re: Passing nothing is surprisingly difficult

#76

Post by tialaramex »

swiftcoder wrote: post #73
tialaramex wrote: post #53

Earlier quoted context omitted.

The main complaint in the post is basically that Rust's actual bona fide slice type doesn't work the way cobbled together library types for this purpose in C or C++ do. The C++ type discussed is much newer than Rust (std::span was standardized in C++ 20). Yes in many cases what C++ APIs mean here isn't a slice of zero Ts at all but instead None, and Rust has an appropriate type for that Option which works as expected…

I'm guess I'm inclined to go the other way. I tend to object to wrapping arrays in Option, because while semantically similar, the empty slice supports the full set of array APIs, whereas Option requires unwrapping

But that's not (a reference to) an array, that would be [T; N] it's a reference to a slice hence the syntax [T]

Arrays know their size, so the "I'll interpret it as zero Ts" makes even less sense for an array where we know up front the size as it is part of the type.

Re: Passing nothing is surprisingly difficult

#77

Post by JonChesterfield »

matheusmoreira wrote: post #71
JonChesterfield wrote: post #55

Earlier quoted context omitted.

Better be rolling your own compiler in that case. Or your own memcpy with a different name. UB to pass memcpy to null means after that call, the pointer is assumed to be non-null. So if(ptr) can constant fold. Maybe faster. I'm in agreement with you on this but your compiler probably isn't.

> Better be rolling your own compiler in that case. No need for that. > the pointer is assumed to be non-null Just give us an option to tell the compiler to stop assuming nonsense like that. I'm gonna make it standard on my makefiles just like -fno-strict-aliasing and -fwrapv. There's no use trying to work around C standard problems. Compilers should just be told to define the undefined and to disable everything that…

LLVM's handling of libc is roughly "assume libc always exists and is linked as machine code". This is deeply unhelpful when that is not true, such as when you're implementing libc. -ffreestanding and -fno-builtins (might be spelled differently) should kill the pattern match into memcpy/memset logic, if it doesn't we have another bug.

I don't trust the clang -fno-strict-aliasing -fno-pointer-whatever strategy. There's too many ways for that to go wrong. Code needs to be correct/safe by default and opt into optimisations to have a chance of working, otherwise it is really easy to fail to check that flag.

There are a few fairly simple C compilers out there. LCC, the one that derives from the obfuscated project, one in gnu mes associated with guix. There's a grammar from the compcert people.

I haven't convinced myself writing a working C compiler is a weekend project but it's surely less than a year, seriously considering it on paranoia grounds. Idea being use it as a reference - when I suspect clang to be breaking things, run against the dumb one that doesn't really do optimisations as a comparison.

Re: Passing nothing is surprisingly difficult

#78

Post by matheusmoreira »

JonChesterfield wrote: post #77
matheusmoreira wrote: post #71

Earlier quoted context omitted.

> Better be rolling your own compiler in that case. No need for that. > the pointer is assumed to be non-null Just give us an option to tell the compiler to stop assuming nonsense like that. I'm gonna make it standard on my makefiles just like -fno-strict-aliasing and -fwrapv. There's no use trying to work around C standard problems. Compilers should just be told to define the undefined and to disable everything that…

LLVM's handling of libc is roughly "assume libc always exists and is linked as machine code". This is deeply unhelpful when that is not true, such as when you're implementing libc. -ffreestanding and -fno-builtins (might be spelled differently) should kill the pattern match into memcpy/memset logic, if it doesn't we have another bug. I don't trust the clang -fno-strict-aliasing -fno-pointer-whatever strategy. There's…

> -ffreestanding and -fno-builtins (might be spelled differently) should kill the pattern match into memcpy/memset logic, if it doesn't we have another bug.

Not only does it not kill that pattern, gcc preempts that bug report by documenting it.

It forces us to link to some weird libgcc.a thing too.

https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html

> The compiler may generate calls to memcmp, memset, memcpy and memmove.

> These entry points should be supplied through some other mechanism when this option is specified.

> In most cases, you need libgcc.a even when you want to avoid other standard libraries.

I can't even find clang's documentation for the nostdlib option. Not sure that documentation even exists. I only found documentation for nostdlib++ which suggests they don't really care about C and the things people use it for.

There's just no way to get these compilers to generate a clean binary with only the symbols provided in the source code and without any of this external stuff. Even with all these options I might run readelf on my binary and find a ton of random double underscore gcc stuff in there.

The whole point of writing a freestanding project was to escape from the libc nonsense which actually makes C a much better language but then the compiler just forces all those things back in. Extremely frustrating!!

> I don't trust the clang -fno-strict-aliasing -fno-pointer-whatever strategy. There's too many ways for that to go wrong.

Well that strategy seems to be good enough for the Linux kernel.

https://lwn.net/Articles/316126/

https://lkml.org/lkml/2003/2/26/158

Re: Passing nothing is surprisingly difficult

#79

Post by anderskaseorg »

matheusmoreira wrote: post #68
anderskaseorg wrote: post #14

Earlier quoted context omitted.

It is extremely common to have a collection that might or might not be empty at runtime, and we don’t want to force every programmer who allocates a slice to manually write an alternate code path for the empty case.

Are memory blocks collections though? An empty list or table makes intuitive sense. Does a zero sized memory block make sense? I'm having trouble understanding that. If the memory has size zero, then by definition there is nothing to point to, nothing whose address can be taken. I definitely see the benefits of well-defined arithmetic on null pointers. As a data type though it seems to me that any pointer could be a…

When you allocate memory for an empty array with malloc(num * size) where num == 0, you get a zero-sized memory block. As discussed in the article, representing this with a null pointer causes problems, because that results in undefined behavior in memcpy (despite asking it to copy 0 bytes). So we want it to be a real memory block that can be safely passed to free().

Re: Passing nothing is surprisingly difficult

#80

Post by JonChesterfield »

matheusmoreira wrote: post #78
JonChesterfield wrote: post #77

Earlier quoted context omitted.

LLVM's handling of libc is roughly "assume libc always exists and is linked as machine code". This is deeply unhelpful when that is not true, such as when you're implementing libc. -ffreestanding and -fno-builtins (might be spelled differently) should kill the pattern match into memcpy/memset logic, if it doesn't we have another bug. I don't trust the clang -fno-strict-aliasing -fno-pointer-whatever strategy. There's…

> -ffreestanding and -fno-builtins (might be spelled differently) should kill the pattern match into memcpy/memset logic, if it doesn't we have another bug. Not only does it not kill that pattern, gcc preempts that bug report by documenting it. It forces us to link to some weird libgcc.a thing too. https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html > The compiler may generate calls to memcmp, memset, memcpy and mem…

It should be no-builtins for disabling the pattern match in clang and things like no-builtin-memset to disable them individually. However that'll be implemented with branches in the backend codegen and it's totally plausible that'll be buggy.

That at least somewhat works because otherwise the memset implementation in libc is prone to being optimised to a call to memset, and thus to a self call which doesn't terminate, and thus to undef.

I've been bitten by this nonsense on two back ends now, really should move it up the work queue. I think there's an active discussion on discourse about making magic libc functions less hazardous.

Can't comment on gcc. Bare metal broadly works on GPUs on clang. X86 is moderately likely to emit calls to memcpy despite flags, but that would be a bug.

The Linux kernel is tested rather more aggressively than my code but my fear of compiler bugs remains firmly established. Occupational hazard of working on compilers - most bugs I see are in the toolchain, as that's where I'm looking.

Post reply on HN