Git's list of banned C functions • Hacker News Board

Re: Git's list of banned C functions

#571

Post by schlupa » Fri, Mar 05, 2021, 3:01 PM UTC

WalterBright wrote: post #533

dotancohen wrote: post #520

Earlier quoted context omitted.

Hello Walter! All things considered, you are probably the best person to ask for tips on string handling in C. Would you might sharing the things that you look for, from the obvious to the subtle? I would love to see some rejected push requests if possible. If I were writing C under your direction, what would you drill into me? Thank you, it is an honour to address you here.

1. whenever you see strncpy(), there's a bug in the code. Nobody remembers if the `n` includes the terminating 0 or nor. I implemented it, and I never remember. I always have to look it up. Don't trust your memory on it. Same goes for all the `n` string functions. 2. be aware of all the C string functions that do strlen. Only do strlen once. Then use memcmp, memcpy, memchr. 3. assign strlen result to a const variable…

strncpy() suffers from its naming. It never was a string function in reality. It is a function to write and clear a fixed size buffer. It was invented to write filenames in the 14 character buffer of a directory entry in early Unix. It should have been name mem-something and people would have never come to the idea to use it for general string routines.

Re: Git's list of banned C functions

#572

Post by JoeAltmaier » Fri, Mar 05, 2021, 3:05 PM UTC

schlupa wrote: post #571

WalterBright wrote: post #533

Earlier quoted context omitted.

1. whenever you see strncpy(), there's a bug in the code. Nobody remembers if the `n` includes the terminating 0 or nor. I implemented it, and I never remember. I always have to look it up. Don't trust your memory on it. Same goes for all the `n` string functions. 2. be aware of all the C string functions that do strlen. Only do strlen once. Then use memcmp, memcpy, memchr. 3. assign strlen result to a const variable…

strncpy() suffers from its naming. It never was a string function in reality. It is a function to write and clear a fixed size buffer. It was invented to write filenames in the 14 character buffer of a directory entry in early Unix. It should have been name mem-something and people would have never come to the idea to use it for general string routines.

If it respects null terminator, then it is a string function.

Re: Git's list of banned C functions

#573

Post by schlupa » Fri, Mar 05, 2021, 3:05 PM UTC

WalterBright wrote: post #533

dotancohen wrote: post #520

Earlier quoted context omitted.

Hello Walter! All things considered, you are probably the best person to ask for tips on string handling in C. Would you might sharing the things that you look for, from the obvious to the subtle? I would love to see some rejected push requests if possible. If I were writing C under your direction, what would you drill into me? Thank you, it is an honour to address you here.

1. whenever you see strncpy(), there's a bug in the code. Nobody remembers if the `n` includes the terminating 0 or nor. I implemented it, and I never remember. I always have to look it up. Don't trust your memory on it. Same goes for all the `n` string functions. 2. be aware of all the C string functions that do strlen. Only do strlen once. Then use memcmp, memcpy, memchr. 3. assign strlen result to a const variable…

Also most of the time people have serious performance regressions with strncpy() as the function overwrites with 0 all the test of the buffer.

     char buffer[2000];
     strcpy(buffer, "hello", sizeof buffer);

writes "hello" and 1995 0 to the buffer.

Re: Git's list of banned C functions

#574

Post by harpiaharpyja » Fri, Mar 05, 2021, 3:10 PM UTC

KptMarchewa wrote: post #503

th3sly wrote: post #491

Earlier quoted context omitted.

> Programming requires none of this. But it requires you to refresh your knowledge constantly so from this point of view it's similar

If you're arguing that medicine does not, then I hope you're doing software engineering.

> so from this point of view it's similar

Re: Git's list of banned C functions

#575

Post by JoeAltmaier » Fri, Mar 05, 2021, 3:10 PM UTC

WalterBright wrote: post #533

dotancohen wrote: post #520

Earlier quoted context omitted.

Hello Walter! All things considered, you are probably the best person to ask for tips on string handling in C. Would you might sharing the things that you look for, from the obvious to the subtle? I would love to see some rejected push requests if possible. If I were writing C under your direction, what would you drill into me? Thank you, it is an honour to address you here.

1. whenever you see strncpy(), there's a bug in the code. Nobody remembers if the `n` includes the terminating 0 or nor. I implemented it, and I never remember. I always have to look it up. Don't trust your memory on it. Same goes for all the `n` string functions. 2. be aware of all the C string functions that do strlen. Only do strlen once. Then use memcmp, memcpy, memchr. 3. assign strlen result to a const variable…

So many potential pitfalls to string functions. But memcpy and friends can have pitfalls too.

I was working on a RISC processor and somebody started using various std lib functions like memcpy from a linux tool chain. I got a bug report - it crashed on certain alignments. Made sense - this processor could only copy words on word alignment etc.

So I wrote a test program for memcpy. Copy 0-128 bytes from a source buffer from offsets 0-128 to a destination buffer at offset 0-128, all combinations of that. Faulted on an alignment issue in code that tried to save cycles by doing register-sized load and store without checking alignment. That was easy! Fixed it. Ran again. Faulted again - different issue, different place.

Before I was done, I had to fix 11 alignment issues. A total fail for whomever wrote that memcpy implementation.

What was the lesson? Well, writing exhaustive tests is a good one. Not blindly trusting std intrinsic libraries is another.

But the one I took with me was, why the hell isn't there an instruction in every processor to efficiently copy from arbitrary source to arbitrary destination with maximum bus efficiency? Why was this a software issue at all! I've been facing code issues like this for decades, and it seems like it will never end.

Re: Git's list of banned C functions

#576

Post by schlupa » Fri, Mar 05, 2021, 3:13 PM UTC

ajuc wrote: post #515

anikom15 wrote: post #462

Earlier quoted context omitted.

I’m not sure if it was the original purpose of C, or of it’s what made C popular, but compared to BASIC, processing strings in C was much faster.

Everything was faster in C - it was compiled and BASIC was interpreted. Better comparison would be between C and Turbo Pascal strings in DOS times. TP strings were limited to 255 characters but they were almost as fast as C strings, in some operations (like checking length) they were faster, and you had to work very hard to create a memory leak or security problem using them. I've learnt Pascal before C and the whole…

UCSD and Turbo-Pascal had it easy with the 255 byte strings. They had real strings but these were compiler extensions. Real Pascal didn't have string support and you could only work with packed array of chars of fixed size and as the language was extremely strong types, to packed chars types of different lengths were considered different types, so you had to write procedures and functions for all used packed array sizes.

Re: Git's list of banned C functions

#577

Post by oleganza » Fri, Mar 05, 2021, 3:20 PM UTC

0xbadcafebee wrote: post #432

paultopia wrote: post #17

Its really wild, as a person coming from other languages who has written maybe ten lines of C in his life that the functions that seem to be massive footguns in C are, like, "format a string" or "get time in GMT." That's... really scary.

Well that's input validation for ya. It's not enough to say "give me a string", or even "give me a file path", and then only check that it has ASCII characters. You have to validate that this input could conceivably be a file on this system that someone would use. "../../../../../../../../../../../../../../../../../../../../etc/shadow" is not a file someone would ever reasonably want to access. But is there an easy w…

That example is typical "confused deputy" security vuln irrespective of the language, not a "validate input" one. Meaning, the typical unix interface to the filesystem is such that it's hard/impossible to express "i'm only having access to this folder and interested in paths within that folder". `chroot` is too dramatic sandboxing that cannot be used for all use cases.

BTW, in macOS there are "secure bookmarks" (see NSURL docs) that are effectively capability tokens: when user drags a file, or selects it in an Open File dialog (which runs isolated from the app), the kernel creates an app-specific token that grants access to that file to the app, so it can access it beyond its sandbox.

Re: Git's list of banned C functions

#578

Post by schlupa » Fri, Mar 05, 2021, 3:24 PM UTC

cestith wrote: post #254

pix64 wrote: post #245

Earlier quoted context omitted.

strlcpy() isn't standard. You have to provide your own implementation if you want your code to be portable.

This is something git does. That's why they prefer it - it's available to git everywhere.

It's 4 lines of code to implement it. So even if it is not available on a platform (glibc mmmh because of Dreppers stubborness), it's no problem.

    size_t strlcpy(char *dst, const char *src, size_t dstsize)
    {
       size_t len = strlen(src);
      if(dstsize)
        *((char*)mempcpy(dst, src, min(len, dstsize-1))) = 0;
      return len;
    }

Re: Git's list of banned C functions

#579

Post by commandlinefan » Fri, Mar 05, 2021, 3:26 PM UTC

yetihehe wrote: post #498

commandlinefan wrote: post #149

Earlier quoted context omitted.

Ok, memcpy(dst, src, strlen(src)) it is then!

Yay for errors, it should be memcpy(dst, src, strlen(src)+1). Strlen doesn't count last 0. If your dst is not zeroed already you will have unterminated string.

Should have let me use strncpy then, shouldn't you? ; P

Re: Git's list of banned C functions

#580

Post by harpiaharpyja » Fri, Mar 05, 2021, 3:26 PM UTC

hguant wrote: post #312

cletus wrote: post #300

Earlier quoted context omitted.

If I had to choose a language to teach programmers to absolute beginners, I think I'd actually go with Go. I understand the predilection for Python but there are some parts of Python that are just... odd.

Python is great fun, and you can be really productive with it, but for people first coming into programming, a language with an explicit and strict type system is invaluable. I used to think that everyone should be taught python first, because it lets you focus on the meat of computer science - algorithms, data manipulation, actually _doing_ something - but after helping my girlfriend out with some comp sci 101-104 p…

I use Python a lot professionally these days, after having been a C#/Java developer for a while (and some experience with C and Free Pascal). I absolutely love the language.

I always feel a little iffy when people talk about Python like it's a language ideally suited to beginners.

Dynamic typing puts so much power in your hands to create expressive structures. But it requires discipline to use properly. It's a great trade off for me but I don't think it would be for beginners.