Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No strcpy either

daniel.haxx.se

Post reply on HN
Print view View original

111–120 of 151 posts

Re: No strcpy either

#111

Post by masklinn »

Waterluvian wrote: post #52

> Enforce checks close to code This makes a lot of sense but one time I find this gets messy is when there’s times I need to do checks earlier in a dataset’s lifetime. I don’t want to pay to check multiple times, but I don’t want to push the check up and it gets lost in a future refactor. I’m imagining a metadata for compile time that basically says, “to act on this data it must have been first checked. I don’t care…

> Which I’m imagining is what Rust is doing with a Result type?

Result only carries information about the success / failure of an unspecified operation, it is not a long term signal and furthermore is not resistant to tampering (so a mistake processing the Result can undo the validation).

What you want in this case is a new separate type, which can only be constructed through the check operation. This is the ethos of "parse, don't validate".

And you're correct that in that case you don't need the check to be close to the consumer, in fact you want the opposite, for the check to be as close to the software edge as possible such that tainted data has limited to no presence inside the system and it's difficult or impossible to unwittingly interact with it.

But of course the farther into that direction you head the more expressive a type system you need. And some constraints are not so easily checked as there's a multitude of consumers each with their own foibles, or as in this case you need to check the interaction of multiple runtime objects.

Re: No strcpy either

#112

Post by adrian_b »

amluto wrote: post #108
samshine wrote: post #79

Earlier quoted context omitted.

I haven't seen a strcpy use a scalar loop in ages. Is this an ARM thing?

Modern x86 CPUs have actual instructions for strcpy that work fairly well. There were several false starts along the way, but the performance is fine now.

They have instructions for memcpy/memmove (i.e. rep movs), not for strcpy.

They also have instructions for strlen (i.e. rep scasb), so you could implement strcpy with very few instructions by finding the length and then copying the string.

Executing first strlen, then validating the sizes and then copying with memcpy if possible is actually the recommended way for implementing a replacement for strcpy, inclusive in the parent article.

On modern Intel/AMD CPUs, "rep movs" is usually the optimal way to implement memcpy above some threshold of data size, e.g. on older AMD Zen 3 CPUs the threshold was 2 kB. I have not tested more recent CPUs to see if the threshold has diminished.

On the old AMD Zen 3 there was also a certain size range above 2 kB at sizes comparable with the L3 cache memory where their implementation interacted somehow badly with the cache and using "non-temporal" vector register transfers outperformed "rep movs". Despite that performance bug for certain string lengths, using "rep movs" for any size above 2 kB gave a good enough performance.

More recent CPUs might be better than that.

Re: No strcpy either

#113

Post by gizmo686 »

adrian_b wrote: post #112
amluto wrote: post #108

Earlier quoted context omitted.

Modern x86 CPUs have actual instructions for strcpy that work fairly well. There were several false starts along the way, but the performance is fine now.

They have instructions for memcpy/memmove (i.e. rep movs), not for strcpy. They also have instructions for strlen (i.e. rep scasb), so you could implement strcpy with very few instructions by finding the length and then copying the string. Executing first strlen, then validating the sizes and then copying with memcpy if possible is actually the recommended way for implementing a replacement for strcpy, inclusive in t…

X86-64 has the REP prefix for string operation. When combined with the MOVS instruction, that is pretty much an instruction for strcpy.

Re: No strcpy either

#114

Post by ofalkaed »

masklinn wrote: post #74
amelius wrote: post #62

Earlier quoted context omitted.

We should just move away from null-terminated strings, where we can, as fast as we can.

We have. C is basically the only langage in any sort of widespread use where terminated strings are a thing. Which of course causes issues when languages with more proper strings interact with C but there you go.

>Which of course causes issues when languages with more proper strings interact with C but there you go.

Is is an issue of "more proper strings" or just languages trying to have their cake and eat it too? have their sense of a string and C interoperability. I think this is were we see the strength of Zig, it's strings are designed around and extend the C idea of string instead of just saying our way is better and we are just going to blame C for any friction.

My standard disclaimer comes into play here, I am not a programmer and very much a humanities sort, I could be completely missing what is obvious. Just trying to understand better.

Edit: That was not quite right, Zig has its string literal for C compatibility. There is something I am missing here in my understanding of strings in the broader sense.

Re: No strcpy either

#115

Post by rf15 »

it feels like the arguments' off-by-one buffer size vs string length is horrible ergonomics and will probably lead to further usage errors in the future.

Yes I have a degree in bike shedding, why am I always getting this particular question

Re: No strcpy either

#116

Post by messe »

gizmo686 wrote: post #113
adrian_b wrote: post #112

Earlier quoted context omitted.

They have instructions for memcpy/memmove (i.e. rep movs), not for strcpy. They also have instructions for strlen (i.e. rep scasb), so you could implement strcpy with very few instructions by finding the length and then copying the string. Executing first strlen, then validating the sizes and then copying with memcpy if possible is actually the recommended way for implementing a replacement for strcpy, inclusive in t…

X86-64 has the REP prefix for string operation. When combined with the MOVS instruction, that is pretty much an instruction for strcpy.

No, it's an instruction for memcpy. You still need to compute the string length first, which means touching every byte individually because you can't use SIMD due to alignment assumptions (or lack thereof) and the potential to touch uninitialized or unmapped memory (when the string crosses a page boundary).

Re: No strcpy either

#117

Post by throwaway2037 »

masklinn wrote: post #74
amelius wrote: post #62

Earlier quoted context omitted.

We should just move away from null-terminated strings, where we can, as fast as we can.

We have. C is basically the only langage in any sort of widespread use where terminated strings are a thing. Which of course causes issues when languages with more proper strings interact with C but there you go.

Doesn't C++'s std::string also use a null terminated char* string internally? Do you count that also?

Re: No strcpy either

#118

Post by messe »

Someone wrote: post #70
t43562 wrote: post #18

I've always wondered at the motivatons of the various string routines in C - every one of them seems to have some huge caveat which makes them useless. After years I now think it's essential to have a library which records at least how much memory is allocated to a string along with the pointer. Something like this: https://github.com/msteinert/bstring

> I've always wondered at the motivatons of the various string routines in C This idiom: char hostname[20]; ... strncpy( hostname, input, 20 ); hostname[19]=0; exists because strncpy was invented for copying file names that got stored in 14-byte arrays, zero terminated only if space permitted ( https://stackoverflow.com/a/1454071 )

I've always assumed that the n in strncpy was meant to signify a max length N. Now I'm wondering if it might have stood for NUL padding.

Re: No strcpy either

#119

Post by anal_reactor »

throwaway2037 wrote: post #117
masklinn wrote: post #74

Earlier quoted context omitted.

We have. C is basically the only langage in any sort of widespread use where terminated strings are a thing. Which of course causes issues when languages with more proper strings interact with C but there you go.

Doesn't C++'s std::string also use a null terminated char* string internally? Do you count that also?

This doesn't count because it's implemented in a way "if you don't need null-terminated string, you won't see it".

Re: No strcpy either

#120

Post by ncruces »

messe wrote: post #116
gizmo686 wrote: post #113

Earlier quoted context omitted.

X86-64 has the REP prefix for string operation. When combined with the MOVS instruction, that is pretty much an instruction for strcpy.

No, it's an instruction for memcpy . You still need to compute the string length first, which means touching every byte individually because you can't use SIMD due to alignment assumptions (or lack thereof) and the potential to touch uninitialized or unmapped memory (when the string crosses a page boundary).

You do aligned reads, which can't crash.

Not even musl uses a scalar loop, if it can do aligned reads/writes: https://git.musl-libc.org/cgit/musl/tree/src/string/stpcpy.c

And you don't need to worry about C UB if you do it in ASM.

Post reply on HN