Live data from Hacker News

Neverflow: C macros that guard against buffer overflows

github.com

111–120 of 150 posts

Re: Neverflow: C macros that guard against buffer overflows

#111
post #52

Earlier quoted context omitted.

> Availability of C++ tooling is much, much closer to availability of C tooling (often it's the same tool!) compared to Rust. Adopting Rust isn't the same category of conversion at all. Which tooling? Just curious, asking entirely in good faith. My recollection is that the majority of tooling I was using with C++ worked with Rust - debuggers, profilers, and sanitizers being the main tools. Although I find that I use…

Basically all the libraries, IDEs, game engines, game console SDKs, HFT, HPC, OS SDKs, embedded OSes, High Integrity Computing certifications, and plenty more stuff deployed into production since C++ ARM [0] was published in 1990, 33 years ago. [0] - The Annotated C++ Reference Manual

That's not C++ tooling. That's tooling written in C++. Two very different things.

Re: Neverflow: C macros that guard against buffer overflows

#112
post #67

Earlier quoted context omitted.

Many of the str functions in the C standard library assume a nul terminator.

Yes, but aside from string literals pointed out by a sibling comment, nothing in the language itself dictates this convention. The C library could be augmented with functions which expect strings structured in other ways.

> nothing in the language itself dictates this convention.

String literals are nul-terminated, e.g.: "foo"[3] == '\0'

Re: Neverflow: C macros that guard against buffer overflows

#113
post #91

Earlier quoted context omitted.

> C could be upgraded to do this in future versions, without too much backwards incompatibility. But I'd hope that doing that would always be optional. There are numerous situations where that would seriously get in the way.

Could you mention one of them?

Strings can point anywhere in the malloc'ed region:

  char buffer[] = "railroad";
  char *s = buffer;
  char *t = buffer + 4;
  printf("mult: %ld\n", strlen(s) * strlen(t));
Suppose I read 100 bytes, formatted as "{name}\t{rank}\t{serial number}\t" using variable length parts.

I can read the data into a single string buffer, replace the commas with NULs, and set up strings pointing to the middle of the buffer;

   typedef struct {char buf[101], char *name, char *rank, char *serialno} person;

   /* 100 bytes formatted as: name\trank\tserial no\t. */
   int read_data(FILE *f, person *p) {
     char *s;
     if (fread(p->buf, 1, 100, f) != 100) return -1;
     p->buf[100] = 0;
     p->name = p->buf;
     if ((s = strchr(p->buf, '\t') == NULL) return -2;
     *s = 0;
     p->rank = s+1;
     if ((s = strchr(s+1, '\t') == NULL)) return -2;
     *s = 0;
     p->serialno = s+1;
     if ((s = strchr(s+1, '\t') == NULL)) return -2;
     *s = 0;
     return 0;
   }

   person subject;
   if (read_data(stdin, &subject)) fail("cannot read.");
   print("Hello %s %s.\n", subject.rank, subject.name);
   ...
Even better, the protocol might have NUL characters already in the code, expecting C strings to point to the correct start.

Re: Neverflow: C macros that guard against buffer overflows

#114

Earlier quoted context omitted.

> Note that C does have strong conventions, such as that strings are terminated by a zero byte Stated the same on HN earlier, but someone pointed out that literal strings are ASCIIZ.

One common trick in safer C libraries is to encode the length of the string one word prior to the beginning of the string. So "hello world" in memory would be 11 'h' 'e' 'l' 'l' 'o' ' ' 'w' 'o' 'r' 'l' 'd' '\0' ptr ^ C could be upgraded to do this in future versions, without too much backwards incompatibility.

From the C99 draft at https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf :

"A string is a contiguous sequence of characters terminated by and including the first null character. .. The length of a string is the number of bytes preceding the null character"

This means, for example, strlen() must always check for the location of the first null character - there's no advantage to checking the length.

How would this work?

  void *x = malloc(8);

   ...
  uint64_t i = 5216694956355289088; // Python: int.from_bytes(b'Hello!\0\0')
  memcpy(x, &i, 8);
  char *s = x;
  puts(s);
Assuming I did it correctly, this should print "Hello!".

When the length get added to the start of the string?

Re: Neverflow: C macros that guard against buffer overflows

#116
post #94

Earlier quoted context omitted.

Calloc is the function originally intented to allocate arrays. Instead of accepting a number of bytes, it takes two unsigned integers(size_t): the number of array members, and the the size of each member. And it checks whether the result of multiplying them fits in a size_t. If not, it returns NULL, allocating nothing(and also sets errno, iirc). Then you can have your code detect it, crash or report an error, and avo…

calloc has its own set of gotchas, though. For instance, it may allocate a different amount of memory than you requested, and it comes with the overhead of zeroing out the allocated memory. Neither of these may matter to you, but when they do, they really matter. So you still have to be thoughtful about using it. Not so different from how you have to be thoughtful about using malloc.

I tend to see zeroed memory as an advantage in the vast majority of cases. And when it's actually significant overhead then s/calloc(/reallocarray(NULL,/

The thing I like about almost always allocating through calloc is this: I know that if my code is somehow not initialising memory properly, the resulting bug will be the same each time, and therefore faster to reproduce and debug. Not that I misinitialise my memory very frequently anymore, it's not that hard to get right.

Surprisingly often, I've found that so much of my data should probably default to zero anyway, so it doesn't really matter all that much.

Calloc can over-allocate, which i always found annoying myself, although at least with calloc, you know that if you only index the pointer modulo the n you passed onto calloc, you won't invoke any demons from the underworld.

But yeah, in general, to really know what you're doing in C, you kind of have to understand memory allocators at a fairly deep level, because the footguns are aplenty. You need to have a mental model of the heap and stack.

Re: Neverflow: C macros that guard against buffer overflows

#117
Here is a different take on it. We can use #define to inform the header about the properties of certain symbols.

Here is my oob.c program. I will show the output, and then the content of "oob.h".

  #include 
  #include 
  #include "oob.h"

  int oob_fail(const char *file, int line)
  {
    fprintf(stderr, "%s:%d:out of bounds array access\n", file, line);
    abort();
  }

  /*
   * Declare properties of array type x
   */
  #define ARRAY_ELTYPE_x int    /* element type is int */
  #define ARRAY_SIZE_x 7        /* number of elements is 7 */

  /*
   * Ensure array type x is fully declared at file scope
   */
  ARRAY_FULLTYPE(x);

  /*
   * Inform the OOB module that the identifiers p and a are
   * used as variables related to type x: either pointers
   * to it or values.
   */
  #define ARRAY_TYPEOF_p x
  #define ARRAY_TYPEOF_a x

  int get_elem(ARRAY_TYPE(x) *p, int i)
  {
     return APREF(p, i);
  }

  int main(void)
  {
     ARRAY_TYPE(x) a = ARRAY_INIT(1, 2, 3);

     for (size_t i = 0; i 
Output:

  $ ./oob
  a[0] == 1
  a[1] == 2
  a[2] == 3
  a[3] == 0
  a[4] == 0
  a[5] == 0
  a[6] == 0
  oob.c:31:out of bounds array access
  Aborted (core dumped)
The content of "oob.h"

  #ifndef OOB_H_435E_FDE9
  #define OOB_H_435E_FDE9

  int oob_fail(const char *file, int line);

  #define OOB_PREFIX oob_ident_
  #define OOB_XCAT(X, Y) X ## Y
  #define OOB_CAT(X, Y) OOB_XCAT(X, Y)

  #define ARRAY_ELTYPE(T) OOB_CAT(ARRAY_ELTYPE_, T)
  #define ARRAY_SIZE(T) OOB_CAT(ARRAY_SIZE_, T)
  #define ARRAY_TAG(T) OOB_CAT(ARRAY_TAG_, T)

  #define ARRAY_FULLTYPE(T)                                                     \
    struct ARRAY_TAG(T) {                                                       \
      ARRAY_ELTYPE(T) a[ARRAY_SIZE(T)];                                         \
    }

  #define ARRAY_TYPE(T) struct ARRAY_TAG(T)

  #define ARRAY_TYPEOF(V) OOB_CAT(ARRAY_TYPEOF_, V)
  #define ARRAY_SIZEOF(V) ARRAY_SIZE(ARRAY_TYPEOF(V))

  #define ARRAY_INIT(...) { { __VA_ARGS__ } }

  #define AREF(ARRAY, I)                                                        \
    (((size_t) (I) >= ARRAY_SIZEOF(ARRAY))                                      \
     ? oob_fail(__FILE__, __LINE__), (ARRAY).a[0]                               \
     : (ARRAY).a[I])

  #define APREF(PARRAY, I)                                                      \
    (((size_t) (I) >= ARRAY_SIZEOF(PARRAY))                                     \
     ? oob_fail(__FILE__, __LINE__), (PARRAY)->a[0]                             \
     : (PARRAY)->a[I])

  #endif
Preprocessor invoked on oob.c (snipped down to the relevant part after the run-time support function oob_fail):

  struct ARRAY_TAG_x { int a[7]; };


  int get_elem(struct ARRAY_TAG_x *p, int i)
  {
     return (((size_t) (i) >= 7) ? oob_fail("oob.c", 31), (p)->a[0] : (p)->a[i]);
  }

  int main(void)
  {
     struct ARRAY_TAG_x a = { { 1, 2, 3 } };

     for (size_t i = 0; i 
It's clean enough to be readable (except, of course, code dense with AREF or APREF calls will be a mess). Uses arrays wrapped in structs, so you can pass arrays by value.

You have to make a list of your variables that are involved and write some #define lines for them.

Same for the array types.

Re: Neverflow: C macros that guard against buffer overflows

#118

Earlier quoted context omitted.

C never has just one way to do something. myArr[5] == 5[myArr] == (insert pointer arithmetic that I won't write here without a compiler check). I think that part of C's beauty is that it gives you freedom. Freedom to shoot yourself in the foot, freedom to write hyper efficient code, and freedom to choose another tool. I agree that this will never be implemented as a standard, but I think that's a good thing. Higher l…

Those are syntactic sugar for the same thing though. Array[5] is just shorthand for *(Array + 5), which is why 5[Array] also works (because addition is commutative). Note that C does have strong conventions, such as that strings are terminated by a zero byte. Nothing in the language demands that, it’s just a convention! C could adopt better conventions.

My copy of the C standard says "A string is a contiguous sequence of characters terminated by and including the first null character."

Re: Neverflow: C macros that guard against buffer overflows

#119

Earlier quoted context omitted.

> Note that C does have strong conventions, such as that strings are terminated by a zero byte Stated the same on HN earlier, but someone pointed out that literal strings are ASCIIZ.

> literal strings are ASCIIZ. If only. In C, it’s a (95+5)-item character set that happens to be a subset of ascii. See https://en.cppreference.com/w/c/language/charset : “The basic literal character set consists of all characters of the basic character set, plus the following control characters” That page also explicitly says: The following characters are not in basic execution character set, but they are required t…

[deleted]

Re: Neverflow: C macros that guard against buffer overflows

#120

Earlier quoted context omitted.

Could you mention one of them?

Sure. For instance, there are times when you need to pack strings tightly together. Adding an extra byte or two before the start of the string would get in the way. You could work around it in many cases, but it makes the code uglier and harder to understand/maintain. One of the things that makes C particularly suitable for certain sorts of tasks is that it's mostly WYSIWYG when it comes to the relationship between d…

I agree on the first paragraph, but the second one applies poorly to strings:

  char *s = "hello";
"hello" has length 6 because there's a hidden \0 even if I never wrote it in the code.
Post reply on HN