Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Memsetting Structures

anmolsarma.in

Post reply on HN
Print view View original

151–160 of 277 posts

Re: Stop Memsetting Structures

#151

Post by kllrnohj »

johannes1234321 wrote: post #87
kllrnohj wrote: post #82

Earlier quoted context omitted.

Damn straight! That's why I exclusively use punch cards, for that good ol' reliability. Don't need no new fangled language crap! Sugar is for eating, not for syntax! If you're actually stuck with C89 you have my sympathy, like all the people stuck still maintaining FORTRAN and COBOL. But that's obviously a niche case with extreme limitations, it doesn't really need to be brought up every time someone mentions C99. Or…

COBOL and Fortran are indeed still wide used, however I claim there is way more C89 than those two together. The point about C89 is that this covers a large area - all common Unix/Linux APIs and libraries are at least C89 compatible, if not written using C8 and as the GP mentioned embedded often depends heavily on such compilers. Thus usage goes far across industries with new libraries being created each day. COBOL a…

C89 compatible but not written in C89, which is the point. And almost certainly just C89 compatible because C99 didn't change the ABI, so that happened "for free". But the actual source, the thing people actually write in, is not C89. Linux heavily leverages new language and compiler extensions. It doesn't handicap itself with only using C89, because that's just silly.

Re: Stop Memsetting Structures

#152

Post by yuubi »

andrewshadura wrote: post #148
bjourne wrote: post #113

Earlier quoted context omitted.

I'm joking because endless flame wars have already been spent debating this issue. But I realize not everyone has heard them yet, so let's strap on the football shoes and bring out the horse carcass one more time! There are platforms on which NULL, integer 0 and floating point 0 is not the same. Hence memsetting won't do the right thing. On modern platforms they are, but still, you don't want to get the Standard C We…

By the standard, the null pointer must be equivalent to 0.

Only if you use conversion within the language: void *p=0;

If you said memset(&p, 0, sizeof(p)) then that's not guaranteed to work on all implementations.

Re: Stop Memsetting Structures

#153

Post by uryga »

andrewshadura wrote: post #148
bjourne wrote: post #113

Earlier quoted context omitted.

I'm joking because endless flame wars have already been spent debating this issue. But I realize not everyone has heard them yet, so let's strap on the football shoes and bring out the horse carcass one more time! There are platforms on which NULL, integer 0 and floating point 0 is not the same. Hence memsetting won't do the right thing. On modern platforms they are, but still, you don't want to get the Standard C We…

By the standard, the null pointer must be equivalent to 0.

gizmo686's comment [1] explains the subtle difference:

> [...] The technicality of the memset example is that it does not set the bits of the pointer by referring to it as a pointer, so the requirement that 0 behave as if it was a null pointer does not apply.

[1] https://news.ycombinator.com/item?id=19767316

Re: Stop Memsetting Structures

#154

Post by blattimwind »

johncolanduoni wrote: post #123
Someone wrote: post #12

Earlier quoted context omitted.

Sending structures that include padding or that you don’t know the exact layout of over the network is another thing you shouldn’t do, period. Code doing that may break with a compiler change or a compilation flag change on one end of the connection, even if you can guarantee both ends use the same CPU.

This kind of issue happens all the time with structs that a kernel, driver etc. expose to userspace: https://j00ru.vexillium.org/papers/2018/bochspwn_reloaded.pd... Not sending the structure over the network is insufficient, and rigorous memset is a clear solution to this problem. Also, C structure layout for a specific ABI is strictly determined; if that weren’t the case you couldn’t compile two C compilation units…

Also don't expect memset() to actually be a library call at all.

Re: Stop Memsetting Structures

#155

Post by blattimwind »

a_t48 wrote: post #106
repsilat wrote: post #93

Earlier quoted context omitted.

I guess there could be some fun workarounds with ptrdiff_t instead of pointers. "My child is 1000 bytes that way in memory" etc.

That's a pretty classical file serialization trick to pack a bunch of heterogeneous data in one file. A header/manifest starting the file, with an array of names, offsets, and sizes.

Actually works somewhat well for mmap'ed access, too, by using a macro or function to "dereference" "pointers" (offsets).

Re: Stop Memsetting Structures

#156

Post by Dylan16807 »

mehrdadn wrote: post #48
unmole wrote: post #45

Earlier quoted context omitted.

I am aware of the fact that unitilized padding can have nasty side effects: https://lwn.net/Articles/417989/ But yeah, I should have mentioned the caveats too. I will update the post once I'm back at my computer.

I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why…

Technically I don't think memset forces the compiler to clear the padding and keep it clear either though. The magic of the C abstract machine.

Re: Stop Memsetting Structures

#158

Post by mehrdadn »

Dylan16807 wrote: post #156
mehrdadn wrote: post #48

Earlier quoted context omitted.

I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why…

Technically I don't think memset forces the compiler to clear the padding and keep it clear either though. The magic of the C abstract machine.

Why? memset followed by memcpy must return the same byte everywhere. Otherwise what would memset even mean?

Re: Stop Memsetting Structures

#159

Post by boulos »

Dylan16807 wrote: post #156
mehrdadn wrote: post #48

Earlier quoted context omitted.

I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why…

Technically I don't think memset forces the compiler to clear the padding and keep it clear either though. The magic of the C abstract machine.

Do you mean in the sense that a compiler can say "I know what memset is supposed to do" then decides that even though sizeof returns a particular size, that it might ignore that and set fewer bytes when it does a substitution for the memcpy call?

Re: Stop Memsetting Structures

#160

Post by Gibbon1 »

shaklee3 wrote: post #145
Gibbon1 wrote: post #130

Earlier quoted context omitted.

My experience with packed structs is there really aren't any gotcha's. Either works or not. The big annoyance is network people all use big endian probably just to spite everyone else.

Why would it be to spite everyone? Big endian is network order by definition. It's hard-coded into every asic in switches and routers for decades. If you don't use big endian your packets won't even make it to where you want. Of course, your own payload can do whatever you want as long as you know with 100% certainty nobody else will ever want to talk to your application and get confused.

Consider the lorawan spec which was written in the last 5 years. The stack is designed to be implemented in software on small 32 bit microcontrollers which are _all_ little endian. And it's big endians.

The people that wrote that spec knew that and went with big endian anyways.

Also I work with people that do network hardware. They can't care less about little or big endian. Makes no difference to them.

Post reply on HN