Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neverflow: C macros that guard against buffer overflows

github.com

Post reply on HN
Print view View original

51–60 of 150 posts

Re: Neverflow: C macros that guard against buffer overflows

#51

Post by mtlmtlmtlmtl »

lelanthran wrote: post #46
mtlmtlmtlmtl wrote: post #10

Earlier quoted context omitted.

The calloc part is one of the most common blind spots I see among C programmers. I try to avoid the malloc(n * sizeof (...)) pattern as much as possible. Sure there are lots of cases where it can never overflow, and you might save a bit of overhead from the zeroing and overflow checking, but most of that overhead might also be imaginary depending on allocator internals, and even kernel internals. It's the sort of thi…

> But now every caller in the future has to be aware of this possibility. Can you clarify: what possibility should you be aware off with malloc that you don't need to be aware of with calloc?

Calloc is the function originally intented to allocate arrays. Instead of accepting a number of bytes, it takes two unsigned integers(size_t): the number of array members, and the the size of each member. And it checks whether the result of multiplying them fits in a size_t. If not, it returns NULL, allocating nothing(and also sets errno, iirc). Then you can have your code detect it, crash or report an error, and avoid memory corruption. Even if you sloppily don't check calloc's return value, you're probably just gonna segfault which is unlikely to lead to data leaks or code execution

If you use malloc(n * size), and n is too large, it could wrap around, malloc gets a smaller number than the program thinks it allocated. Which means that even if the program does bounds/null checking on the array later on, it has the wrong bounds. This can be used to access or modify other objects on the heap, or even modify allocator internals in some cases, depends on the implementation details of the allocator.

So what I meant was, you better be careful using malloc(n * size) unless n is a constant. If it's in any way tied to program behaviour or user input, it's a hole waiting to happen.

Re: Neverflow: C macros that guard against buffer overflows

#52

Post by pjmlp »

insanitybit wrote: post #22
humanrebar wrote: post #21

Earlier quoted context omitted.

Availability of C++ tooling is much, much closer to availability of C tooling (often it's the same tool!) compared to Rust. Adopting Rust isn't the same category of conversion at all. For new side projects, pick what you want to use of course. But for existing codebases and projects that aspire to have maximum impact, I recommend fully considering tradeoffs instead of thinking in terms of "clear winners".

> Availability of C++ tooling is much, much closer to availability of C tooling (often it's the same tool!) compared to Rust. Adopting Rust isn't the same category of conversion at all. Which tooling? Just curious, asking entirely in good faith. My recollection is that the majority of tooling I was using with C++ worked with Rust - debuggers, profilers, and sanitizers being the main tools. Although I find that I use…

Basically all the libraries, IDEs, game engines, game console SDKs, HFT, HPC, OS SDKs, embedded OSes, High Integrity Computing certifications, and plenty more stuff deployed into production since C++ ARM [0] was published in 1990, 33 years ago.

[0] - The Annotated C++ Reference Manual

Re: Neverflow: C macros that guard against buffer overflows

#53

Post by mtlmtlmtlmtl »

loeg wrote: post #49
lelanthran wrote: post #46

Earlier quoted context omitted.

> But now every caller in the future has to be aware of this possibility. Can you clarify: what possibility should you be aware off with malloc that you don't need to be aware of with calloc?

Multiplying array length by sizeof(element type) can overflow. Of course, you can write your own malloc_array() that uses __builtin_mul_overflow() and doesn't come with calloc's drawback (the cost of zeroing the allocated memory).

OpenBSD's libc has reallocarray for this, which is realloc with the same bounds checking as calloc, but if the first parameter is NULL, it's just calloc without the zeroing.

And I believe you'll find it in glibc too these day? Or if not, there's always libbsd, which has lots of handy stuff anyways.

Re: Neverflow: C macros that guard against buffer overflows

#54

Post by insanitybit »

twic wrote: post #24
insanitybit wrote: post #22

Earlier quoted context omitted.

> Availability of C++ tooling is much, much closer to availability of C tooling (often it's the same tool!) compared to Rust. Adopting Rust isn't the same category of conversion at all. Which tooling? Just curious, asking entirely in good faith. My recollection is that the majority of tooling I was using with C++ worked with Rust - debuggers, profilers, and sanitizers being the main tools. Although I find that I use…

The parent comment says "compiler tool chain", and i understand "tooling" here as meaning that. So, compiler, linker, assembler, etc. All the major C compilers are also C++ compilers, and none are (yet) Rust compilers, so out of the gate, C++ has similar availability to C.

Are we just talking about portability then? Because "same category of conversion" seems fine - I would say that for 99.9999% of projects the difference in portability is non existent.

Re: Neverflow: C macros that guard against buffer overflows

#56

Post by pjmlp »

epistasis wrote: post #37
twic wrote: post #24

Earlier quoted context omitted.

The parent comment says "compiler tool chain", and i understand "tooling" here as meaning that. So, compiler, linker, assembler, etc. All the major C compilers are also C++ compilers, and none are (yet) Rust compilers, so out of the gate, C++ has similar availability to C.

And yet, even with that, Yann Collet credits Google's use of C++ for the compression library as a critical mistake that allowed him, an unknown, to gain traction with his own compression methods. Google later rewrote their library in C: https://overcast.fm/+LfVPHmBTo Even if the tool chain exists, it must be adopted, unless you can rely on binaries being available for your end users, which will never be the case for…

Unless we are talking about an obscure platform or some PIC CPU, a C++ compiler is available on the same box as the C compiler.

Second, extern "C" exists.

Third, in what concerns clang and MSVC, the C library is actually implemented in C++ with extern "C".

Re: Neverflow: C macros that guard against buffer overflows

#57

Post by pjmlp »

coliveira wrote: post #31
epistasis wrote: post #17

Earlier quoted context omitted.

If one is ready to switch languages, then the clear winner is rust over C++, and I say that as someone who avoided diving into Rust for years because it seemed completely overhyped and with too much cryptic syntax. C still wins by far when writing libraries that will be used by lots of other people. Doesn't matter what language they are using, they will be able to add in a library written in C very easily. However, C…

People using C will not change to your language-du-jour, please stop.

They better improve their error free coding skills when liability laws come for them.

Re: Neverflow: C macros that guard against buffer overflows

#58

Post by adastra22 »

6D794163636F756 wrote: post #50
eqvinox wrote: post #8

The problem with C and buffer overflows isn't that you can't guard against them, or that there is no existing, reusable code to do so — it's that none of this functionality is standardized. Adding another one to the existing 41383 ways of doing this is in fact the exact opposite of what's needed. Ideally C needs one way of doing this, and that would be described in the standard. But that's not how C "rolls", and we'l…

C never has just one way to do something. myArr[5] == 5[myArr] == (insert pointer arithmetic that I won't write here without a compiler check). I think that part of C's beauty is that it gives you freedom. Freedom to shoot yourself in the foot, freedom to write hyper efficient code, and freedom to choose another tool. I agree that this will never be implemented as a standard, but I think that's a good thing. Higher l…

Those are syntactic sugar for the same thing though. Array[5] is just shorthand for *(Array + 5), which is why 5[Array] also works (because addition is commutative).

Note that C does have strong conventions, such as that strings are terminated by a zero byte. Nothing in the language demands that, it’s just a convention! C could adopt better conventions.

Re: Neverflow: C macros that guard against buffer overflows

#59

Post by bheadmaster »

ollien wrote: post #34
ChrisRR wrote: post #26

Earlier quoted context omitted.

The issue with 1 is that it only works until you pass an array into a function by pointer, then the macro no longer works. In my experience it's most likely that a function will write past the bounds of a buffer that's been passed as an argument. In that case, make sure the size of array is always included as an argument as you said in 4.

> The issue with 1 is that it only works until you pass an array into a function by pointer, then the macro no longer works. GCC even has a warning for this. Even worse, even if you specify the argument to be "of the type" array, it will actually still decay to a pointer. Basically, this macro will only work if you use it in the same function the array is defined. https://godbolt.org/z/vr4za73qq

One exception is if you explicitly define argument as array of fixed length.

Downside being, obviously, that it will only work with arrays of that particular length.

Re: Neverflow: C macros that guard against buffer overflows

#60

Post by uecker »

hgs3 wrote: post #16

C23 improved struct compatibility so you might be able to leverage that to craft macros that better emulate slices. [1] There is an RFC proposal for the Clang frontend for adding bounds checking reminiscent of Microsoft's SAL. [2] [1] https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3003.pdf [2] https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-...

You may be interested in this: https://github.com/uecker/noplate.git
Post reply on HN