Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C Strings and my slow descent to madness

deusinmachina.net

Post reply on HN
Print view View original

161–170 of 329 posts

Re: C Strings and my slow descent to madness

#161

Post by WalterBright »

tialaramex wrote: post #139
athenot wrote: post #103

Earlier quoted context omitted.

The old MacOS (pre-X) did just that. Strings were all "Pascal strings", ie. with the first byte containing the length of the actual string. Building blocks for memory were also very different from stdlib, notably the use of Handles, which were pointers of pointers, so that the OS could move a block of data around to defragment the heap behind your back without breaking the memory addressing.

Pascal strings are also kind of bad though. All sub-string operations need allocation, or have to be defined with intermediate results which aren't "really" strings, so in that sense it's not an improvement on Zero-terminated strings. Equality tests are cheaper which is nice, since strings of different lengths compare unequal immediately, but most things aren't really improved. C++ string_view is closer to the Right…

D's strings were defined to be UTF-8 back in 2000. wstring is UTF-16, and dstring is UTF-32.

Back then it wasn't clear which encoding method would turn out to be dominant, so we did all three. (Java was built on UTF-16.)

As it eventually became clear, UTF-8 is da winnah, and the other formats are sideshows. Windows, which uses UTF-16, is handled by converting UTF-8 to -16 just before calling a Windows function, and converting anything coming back to UTF-8.

D doesn't distinguish between a string and a string view.

Re: C Strings and my slow descent to madness

#163

Post by asveikau »

1vuio0pswjnm7 wrote: post #141
stephc_int13 wrote: post #31

If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own. It is not that difficult in practice. The old C std lib is, in my opinion, outdated, obsolete and a very bad fit for complex string handling, especially on the memory management side. In my own framework, the string management module is using a dedicated memory allocator and a "high l…

"If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own." unsigned int str_len(const char *s) { register const char *t; t = s; for (;;) { if (!*t) return t - s; ++t; if (!*t) return t - s; ++t; if (!*t) return t - s; ++t; if (!*t) return t - s; ++t; } } I still use this instead of stdlib strlen. Of course I also use software everyday that…

Don't do this. libc string functions are usually done with hand tweaked assembly or as a builtin by the compiler. They will be faster than this.

This sort of practice is very outdated. The last time it made sense for performance reasons was probably the early 90s or earlier.

Additionally, strlen is not one of the C string functions you want to replace due to defects, the same way you would want to do with crusty old strcpy. If you're working with C strings there is nothing wrong with strlen. (Just don't call it redundantly in a loop body ...)

Re: C Strings and my slow descent to madness

#164

Post by mikewarot »

WalterBright wrote: post #159
athenot wrote: post #103

Earlier quoted context omitted.

The old MacOS (pre-X) did just that. Strings were all "Pascal strings", ie. with the first byte containing the length of the actual string. Building blocks for memory were also very different from stdlib, notably the use of Handles, which were pointers of pointers, so that the OS could move a block of data around to defragment the heap behind your back without breaking the memory addressing.

> with the first byte containing the length of the actual string And the wheels fall off with the first string longer than 255 characters.

Which is why Free Pascal strings are so awesome. I've personally stuffed a billion bytes on one, without issues. They are automatic reference counted, and as close to magic as you can get. You can return one from a function without issue.

However, Free Pascal has the worst documentation of any major project I've ever encountered (The exact opposite of Turbo Pascal), so I can't link to a good reference. Their Wiki is a black hole of nuance and sucks all useful stuff off the internet.

Re: C Strings and my slow descent to madness

#165

Post by initplus »

WalterBright wrote: post #159
athenot wrote: post #103

Earlier quoted context omitted.

The old MacOS (pre-X) did just that. Strings were all "Pascal strings", ie. with the first byte containing the length of the actual string. Building blocks for memory were also very different from stdlib, notably the use of Handles, which were pointers of pointers, so that the OS could move a block of data around to defragment the heap behind your back without breaking the memory addressing.

> with the first byte containing the length of the actual string And the wheels fall off with the first string longer than 255 characters.

You can fix this issue by using a variable width integer encoding for the size.

Re: C Strings and my slow descent to madness

#166

Post by 1vuio0pswjnm7 »

1vuio0pswjnm7 wrote: post #141
stephc_int13 wrote: post #31

If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own. It is not that difficult in practice. The old C std lib is, in my opinion, outdated, obsolete and a very bad fit for complex string handling, especially on the memory management side. In my own framework, the string management module is using a dedicated memory allocator and a "high l…

"If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own." unsigned int str_len(const char *s) { register const char *t; t = s; for (;;) { if (!*t) return t - s; ++t; if (!*t) return t - s; ++t; if (!*t) return t - s; ++t; if (!*t) return t - s; ++t; } } I still use this instead of stdlib strlen. Of course I also use software everyday that…

Here is the musl stdlib strlen. I do not use glibc.

https://git.musl-libc.org/cgit/musl/plain/src/string/strlen....

   #include 
   #include 
   #include 
   
   #define ALIGN (sizeof(size_t))
   #define ONES ((size_t)-1/UCHAR_MAX)
   #define HIGHS (ONES * (UCHAR_MAX/2+1))
   #define HASZERO(x) ((x)-ONES & ~(x) & HIGHS)
   
   size_t strlen(const char *s)
   {
   const char *a = s;
   #ifdef __GNUC__
   typedef size_t __attribute__((__may_alias__)) word;
   const word *w;
   for (; (uintptr_t)s % ALIGN; s++) if (!*s) return s-a;
   for (w = (const void *)s; !HASZERO(*w); w++);
   s = (const void *)w;
   #endif
   for (; *s; s++);
   return s-a;
   }

Re: C Strings and my slow descent to madness

#167

Post by int_19h »

giantrobot wrote: post #154
int_19h wrote: post #124

Earlier quoted context omitted.

That was my point. Although you can, actually - since literals themselves are array-typed, you can sizeof them to get the character count without relying on null termination. It's even possible to get a non-null-terminated literal if the target array type is not large enough to fit null, e.g.: char s[3] = "foo"; // not null-terminated!

> character count Byte count.

If you want to be pedantic, "an object declared as type char is large enough to store any member of the basic execution character set". It doesn't actually have to be a byte.

In practice, in C context, character == char == byte. Other concepts have to use different names to avoid confusion with the language spec.

Re: C Strings and my slow descent to madness

#168

Post by avar »

TremendousJudge wrote: post #108
tom_ wrote: post #67

Earlier quoted context omitted.

* consumes an int, not a size_t: https://port70.net/~nsz/c/c11/n1570.html#7.21.6.1p5

I love how in every C code snippet on every comment on this thread, somebody got something wrong. I take it as a sign that it's probably best to avoid C as much as possible.

Any C compiler that isn't a trivial toy implementation will warn about that, so it's hardly a C gotcha.

Re: C Strings and my slow descent to madness

#169

Post by PaulDavisThe1st »

bluGill wrote: post #151
PaulDavisThe1st wrote: post #133

Earlier quoted context omitted.

The appearance of strncpy() in any source code is an immediate panic attack for me. It should never be used, and if it is used, it should be removed. Similar rule for sprintf(), all instances of which should be replaced by snprintf().

Unfortunately there often isn't a better replacement in your standard library (embedded systems are weird). I ended up using strncpy followed by automatically setting the last byte of the string to null.

strncpy() in particular is so bad that you're better off (for a rare exception to the rule) just writing your own, that does what most people think strncpy() does (or should do) rather than what it actually does.

Re: C Strings and my slow descent to madness

#170

Post by bruce343434 »

Meh. The w_char stuff is barely C's fault. You use wide (constant width) characters then set the terminal encoding to utf8 (variable length encoding). What did you expect? It's a windows issue. I can copy paste all sorts of utf8 in "normal" string literals, printf and puts them, and it just works in my terminal.

RE counting characters: this is a whole can of worms. Do you want to count grapheme clusters? Code points? Anything other than just the amount of bytes? Use a unicode library.

The latter part of this article is a bit like those articles that make fun of javascript for having floating point numbers behave like, gasp, floating point numbers.

Post reply on HN