Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Passing nothing is surprisingly difficult

davidben.net

Post reply on HN
Print view View original

31–40 of 80 posts

Re: Passing nothing is surprisingly difficult

#31

Post by kevingadd »

Joker_vD wrote: post #18
kevingadd wrote: post #5

A fun additional twist to this is that dereferencing nullptr is valid in WebAssembly, and actual data can in fact end up there, though ideally it never will. If you ensure that the 'zero page' (so to speak) is empty you can also exploit this property for optimizations, and in some cases the emscripten toolchain will do so. i.e. if you have struct MyArray { uint length; T items[0]; } you can elide null pointer checks…

Because WASM is not C and there is no "nullptr" in WASM. In WASM, zero is just an address, as valid as any other. And C actually doesn't require the null pointer value to have bit pattern "all zeros", precisely to allow for architectures where treating zero address as invalid would be way too cumbersome. And some implementations actually took that option.

I wasn't aware the spec allowed for nullptr to not be 0, that's fascinating! In that case you could probably use 0xFFFFFFFF as long as you limit the size of the WASM heap to below 4GB, then. You'd risk having addresses wrap-around though.

Re: Passing nothing is surprisingly difficult

#32

Post by Joker_vD »

kevingadd wrote: post #31
Joker_vD wrote: post #18

Earlier quoted context omitted.

Because WASM is not C and there is no "nullptr" in WASM. In WASM, zero is just an address, as valid as any other. And C actually doesn't require the null pointer value to have bit pattern "all zeros", precisely to allow for architectures where treating zero address as invalid would be way too cumbersome. And some implementations actually took that option.

I wasn't aware the spec allowed for nullptr to not be 0, that's fascinating! In that case you could probably use 0xFFFFFFFF as long as you limit the size of the WASM heap to below 4GB, then. You'd risk having addresses wrap-around though.

Nothing stops you from having your null pointer in the middle of the address space. Some C compiler for DOS or early Windows did that IIRC (it was 0xB800 or something?.. so that it wouldn't accidentally corrupt the interrupt table). Also, C explicitly prohibits address wrap-around problems for pointers:

    Moreover, if the expression P points to the last element of an array object, the expression (P)+1 points one past the last element of the array object, and if the expression Q points one past the last element of an array object, the expression (Q)-1 points to the last element of the array object. If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined.
But this is fine, since pointer comparisons (as in, less/greater comparisons) are actually both pretty restricted and required to have reasonable semantics when comparing pointers that point into the same object/array:

    When two pointers are compared, the result depends on the relative locations in the address space of the objects pointed to. If two pointers to object types both point to the same object, or both point one past the last element of the same array object, they compare equal. If the objects pointed to are members of the same aggregate object, pointers to structure members declared later compare greater than pointers to members declared earlier in the structure, and pointers to array elements with larger subscript values compare greater than pointers to elements of the same array with lower subscript values. All pointers to members of the same union object compare equal. If the expression P points to an element of an array object and the expression Q points to the last element of the same array object, the pointer expression Q+1 compares greater than P. In all other cases, the behavior is undefined.
By the way, this means that, among other things, if you use number N to represent a null pointer then number N-1 can not ever be a valid pointer to anything: adding 1 to a valid pointer is always allowed, and this addition should produce a non-null pointer — because the resulting pointer is required to be well-behaved in comparisons, and comparisons with null pointer are UB.

Re: Passing nothing is surprisingly difficult

#33

Post by wruza »

kazinator wrote: post #22
LegionMammal978 wrote: post #19

Earlier quoted context omitted.

N2464 [0]: there was lots of implementation divergence on what realloc(ptr, 0) did (especially with BSD, which allegedly doesn't free the memory at all?), so they just declared it UB. [0] https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf

The BSD people don't understand what little standards they do read. It's unfortunate that we have to spoil the language for their sake. The requirements in C99 and before are perfectly clear. realloc is described as liberating the old pointer, and then allocates a new one as if by malloc. (Except that it magically has access to both objects so it can transfer the necessary bytes that must be transferred from the old…

Just a reminder to myself how happy I am to leave this academic level of snobbery behind and migrate to languages that help getting things done and save you from nonsense like this. I did C for about ten years. Hope I’ll never have to do a single line of it again.

Re: Passing nothing is surprisingly difficult

#34

Post by LegionMammal978 »

torstenvl wrote: post #25
kazinator wrote: post #22

Earlier quoted context omitted.

The BSD people don't understand what little standards they do read. It's unfortunate that we have to spoil the language for their sake. The requirements in C99 and before are perfectly clear. realloc is described as liberating the old pointer, and then allocates a new one as if by malloc. (Except that it magically has access to both objects so it can transfer the necessary bytes that must be transferred from the old…

C99 and C11 have no special treatment for a size of zero. Since "memory for the new object [of size zero] cannot be allocated, the old object is not deallocated and its value is unchanged. " (emphasis added). This is exactly what BSD does. C17 says "If size is zero and memory for the new object is not allocated, it is implementation-defined whether the old object is deallocated" (emphasis added). What standard, exact…

Such behavior violates C89 (they later made it unspecified, likely to match the divergence):

> If size is zero and ptr is not a null pointer, the object it points to is freed.

It also violates every version of the SUS and POSIX up to Issue 6 (they made it unspecified in Issue 7):

> If size is 0 and ptr is not a null pointer, the object pointed to is freed.

Going down to at least the 3rd edition of SVID:

> If size is zero and ptr is not a null pointer, the object pointed to is freed.

Re: Passing nothing is surprisingly difficult

#35

Post by [deleted] »

adrianN wrote: post #28
kazinator wrote: post #22

Earlier quoted context omitted.

The BSD people don't understand what little standards they do read. It's unfortunate that we have to spoil the language for their sake. The requirements in C99 and before are perfectly clear. realloc is described as liberating the old pointer, and then allocates a new one as if by malloc. (Except that it magically has access to both objects so it can transfer the necessary bytes that must be transferred from the old…

I don't think insulting the BSD folks is very nice. They probably had good reasons for their decisions.

[deleted]

Re: Passing nothing is surprisingly difficult

#36

Post by LegionMammal978 »

torstenvl wrote: post #26
kazinator wrote: post #23

Earlier quoted context omitted.

> Is there a rationale for a memory allocator to support zero sized allocations? Mainly, that it has supported that before and programs rely on it. Programs written to the C99 standard can resize a dynamic vector down to empty with a resize(ptr, 0). The pointer coming from that will be the same as if malloc(0) has been called. So now, that has been taken away; those programs can now make demons fly out of your nose.…

> Programs written to the C99 standard can resize a dynamic vector down to empty with a resize(ptr, 0). C99 has no resize() function. Assuming you mean realloc(), C99 does not guarantee you can use realloc() in this manner. See also: https://news.ycombinator.com/item?id=38850575 https://stackoverflow.com/questions/16759849/using-realloc-x... https://wiki.sei.cmu.edu/confluence/plugins/servlet/mobile?c... https://deve…

Indeed, you couldn't reliably free() the old pointer if the realloc(ptr, 0) failed.

But xrealloc(ptr, 0) (or equivalent) would still be perfectly consistent, assuming you trust your implementation to support non-null 0-size allocations in the first place. It's very common to just "leak it all and abort" on a critical error like memory exhaustion. There's a reason most non-C languages expose an infallible allocation API as the default option.

I do think that UB is an overly heavy hammer for realloc(ptr, 0), since the xrealloc(ptr, 0) use case works just as well regardless of how unspecified the values of the old pointer or errno are on failure.

Re: Passing nothing is surprisingly difficult

#37

Post by kazinator »

adrianN wrote: post #28
kazinator wrote: post #22

Earlier quoted context omitted.

The BSD people don't understand what little standards they do read. It's unfortunate that we have to spoil the language for their sake. The requirements in C99 and before are perfectly clear. realloc is described as liberating the old pointer, and then allocates a new one as if by malloc. (Except that it magically has access to both objects so it can transfer the necessary bytes that must be transferred from the old…

I don't think insulting the BSD folks is very nice. They probably had good reasons for their decisions.

Mostly, the BSD people think that POSIX and ISO C are just forks of Unix documentation, that are mainly used for making non-Unix systems look Unix-like. BSD comes from real Unix DNA and so following those is optional; whatever direction BSD takes is Unix by definition, as if it's forever 1983.

Re: Passing nothing is surprisingly difficult

#38

Post by kazinator »

LegionMammal978 wrote: post #34
torstenvl wrote: post #25

Earlier quoted context omitted.

C99 and C11 have no special treatment for a size of zero. Since "memory for the new object [of size zero] cannot be allocated, the old object is not deallocated and its value is unchanged. " (emphasis added). This is exactly what BSD does. C17 says "If size is zero and memory for the new object is not allocated, it is implementation-defined whether the old object is deallocated" (emphasis added). What standard, exact…

Such behavior violates C89 (they later made it unspecified, likely to match the divergence): > If size is zero and ptr is not a null pointer, the object it points to is freed. It also violates every version of the SUS and POSIX up to Issue 6 (they made it unspecified in Issue 7): > If size is 0 and ptr is not a null pointer, the object pointed to is freed. Going down to at least the 3rd edition of SVID: > If size is…

What's obvious about all those Unix-related specifications is that they are not of BSD lineage.

Re: Passing nothing is surprisingly difficult

#39

Post by kazinator »

torstenvl wrote: post #26
kazinator wrote: post #23

Earlier quoted context omitted.

> Is there a rationale for a memory allocator to support zero sized allocations? Mainly, that it has supported that before and programs rely on it. Programs written to the C99 standard can resize a dynamic vector down to empty with a resize(ptr, 0). The pointer coming from that will be the same as if malloc(0) has been called. So now, that has been taken away; those programs can now make demons fly out of your nose.…

> Programs written to the C99 standard can resize a dynamic vector down to empty with a resize(ptr, 0). C99 has no resize() function. Assuming you mean realloc(), C99 does not guarantee you can use realloc() in this manner. See also: https://news.ycombinator.com/item?id=38850575 https://stackoverflow.com/questions/16759849/using-realloc-x... https://wiki.sei.cmu.edu/confluence/plugins/servlet/mobile?c... https://deve…

> C99 does not guarantee you can use realloc() in this manner

Yes it does. It requires support for reallocing down to zero, which results in an object that is like one that comes from malloc(0).

(What some people think is that realloc(x, 0) is equivalent to free(x). It isn't. Resizing down to zero isn't freeing. It might be, if malloc(0) doesn't allocate anything and just returns null. Why some people think realloc(x, 0) is free(x) is that they read realloc man page from the Linux man-pages project which says such a thing.)

realloc(ptr, 0) could fail to free ptr, in the situation that allocating the zero-sized replacement object fails. In that case, null could be returned, leaving the old object valid. This is ambiguous, because null could also be the happy case return value when the old object was freed and the zero-sized allocation deliberately produced null. Under those conditions, the cases in which there is a memory leak are indistinguishable from the ones in which there isn't.

(I'd rather suffer a memory leak in the OOM condition, than have previously defined behavior gratuitously flip to undefined.)

Re: Passing nothing is surprisingly difficult

#40

Post by [deleted] »

LegionMammal978 wrote: post #36
torstenvl wrote: post #26

Earlier quoted context omitted.

> Programs written to the C99 standard can resize a dynamic vector down to empty with a resize(ptr, 0). C99 has no resize() function. Assuming you mean realloc(), C99 does not guarantee you can use realloc() in this manner. See also: https://news.ycombinator.com/item?id=38850575 https://stackoverflow.com/questions/16759849/using-realloc-x... https://wiki.sei.cmu.edu/confluence/plugins/servlet/mobile?c... https://deve…

Indeed, you couldn't reliably free() the old pointer if the realloc(ptr, 0) failed. But xrealloc(ptr, 0) (or equivalent) would still be perfectly consistent, assuming you trust your implementation to support non-null 0-size allocations in the first place. It's very common to just "leak it all and abort" on a critical error like memory exhaustion. There's a reason most non-C languages expose an infallible allocation A…

[deleted]
Post reply on HN