Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Banned C standard library functions in Git source code

github.com

Post reply on HN
Print view View original

301–310 of 329 posts

Re: Banned C standard library functions in Git source code

#301

Post by pjmlp »

jstimpfle wrote: post #297
pjmlp wrote: post #279

Earlier quoted context omitted.

Because they have a security culture that you only get errors due to "holding it wrong" or "every good programmer does it right", despite evidence on the contrary. Plus added the fact that early C compilers generated quite lousy code on 8 and 16 bit computers, there is this idea to micro-optimize each line of code as it is being written, without any profiling feedback of it actually matters, rather cargo cult how wri…

> For example, outside 3D rendering and audio software processing, I never saw a visible impact (to the end user) of bounds checking. Indeed the computation overhead on bounds checking is irrelevant for "cold" code, but consider this 1) Pretty sure you can get bounds checking when compilers detect that you're accessing a static array. 2) Otherwise, it's unclear how to devise a system that integrates bounds checking w…

1) Totally optional from ISO C point of view, not all C compilers do do it, usually requires static analysis or specific compiler warnings to be enabled. Which not everyone does.

2) Solaris does it perfectly fine on SPARC thanks to tagged memory (ADI). Which Google in collaboration with ARM will make mandatory on future Android releases as well. [0]

3) It hardly mattered in MS-DOS and Amiga LOB applications developed in across Turbo Basic, Quick Basic, GFA Basic, Turbo Pascal, Clipper, so it matters even less nowadays unless we are speaking about PIC like hardware.

Regarding bounds checking I usually refer to Hoare's turing award speech, back in 1981:

"Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980, language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."

Or for that matter, the DoD Multics's B2 security evaluation[1], with several remarks how PL/I made the system safer with its string handling, pointer integrity validation and bounds checking.

As noted, yes there are niche cases where bounds checking does have an impact, but for a large spectrum of code that gets daily written it isn't the case.

[0] - https://security.googleblog.com/2019/08/adopting-arm-memory-...

[1] - https://multicians.org/multics-fer.pdf

Re: Banned C standard library functions in Git source code

#302

Post by jstimpfle »

pjmlp wrote: post #301
jstimpfle wrote: post #297

Earlier quoted context omitted.

> For example, outside 3D rendering and audio software processing, I never saw a visible impact (to the end user) of bounds checking. Indeed the computation overhead on bounds checking is irrelevant for "cold" code, but consider this 1) Pretty sure you can get bounds checking when compilers detect that you're accessing a static array. 2) Otherwise, it's unclear how to devise a system that integrates bounds checking w…

1) Totally optional from ISO C point of view, not all C compilers do do it, usually requires static analysis or specific compiler warnings to be enabled. Which not everyone does. 2) Solaris does it perfectly fine on SPARC thanks to tagged memory (ADI). Which Google in collaboration with ARM will make mandatory on future Android releases as well. [0] 3) It hardly mattered in MS-DOS and Amiga LOB applications developed…

1) It's not standardized but it's very simple to implement nevertheless.

It is my understanding that 2) doesn't have anything to do with C (so given hardware support, you can have it for free whether working in C or not. I think this kind of invalidates your point).

And also, 2) is not perfect, only probabilistic (a source I found says 94% likelyhood to detect OOB).

And it works only for the most basic situations where you use the system allocator and never subpartition these allocations. So, beyond these simple cases that we can get for free without any involvement from C semantics, I still maintain that it is unclear how to devise a reasonable a useful system to do bounds checking that can be added to the C memory model. We can make up an annotation syntax to cover many of the simpler cases, but these are hardly better than plain assertions (which I regularly use).

I doubt Hoare had a good idea to add general bounds checking on a low-level language like C, otherwise that would be standardized by now.

Re: Banned C standard library functions in Git source code

#303

Post by jstimpfle »

pjmlp wrote: post #298
jstimpfle wrote: post #295

Earlier quoted context omitted.

When compatibility is needed, APIs should consume pointer+length pairs. It doesn't get better than that, in ANY language, in terms of simplicity and modularity. Libraries like Qt with extreme lock-in are at the other end of the spectrum. If it works for them, that's nice. Doesn't work for me. I don't think using a string library is in the spirit of C programming.

Which means either pointer+length get packed into their own structure to avoid mix up errors, thus requiring conversion function calls across libraries, or they are given separately manually, thus opening the door to the copy-paste mistakes from pointer+bad length that they are supposed to protect against. Qt is as locked-in as any LPGL 3 FOSS project is.

> Which means either pointer+length get packed into their own structure to avoid mix up errors, thus requiring conversion function calls across libraries, or they are given separately manually, thus opening the door to the copy-paste mistakes from pointer+bad length that they are supposed to protect against.

No, we were talking about compatibility/modularity. You're shifting the topic.

> Qt is as locked-in as any LPGL 3 FOSS project is.

I'm not speaking about licenses lock-in, but about lock-in from an engineering point of view. Are you aware of any significant Qt projects that don't have "Q" all over their codebase?

(And yes, by contrast to GPL, I believe you can use LGPL libraries without suffering a terrible amount of (license) lock-in)

Re: Banned C standard library functions in Git source code

#305

Post by pjmlp »

jstimpfle wrote: post #303
pjmlp wrote: post #298

Earlier quoted context omitted.

Which means either pointer+length get packed into their own structure to avoid mix up errors, thus requiring conversion function calls across libraries, or they are given separately manually, thus opening the door to the copy-paste mistakes from pointer+bad length that they are supposed to protect against. Qt is as locked-in as any LPGL 3 FOSS project is.

> Which means either pointer+length get packed into their own structure to avoid mix up errors, thus requiring conversion function calls across libraries, or they are given separately manually, thus opening the door to the copy-paste mistakes from pointer+bad length that they are supposed to protect against. No, we were talking about compatibility/modularity. You're shifting the topic. > Qt is as locked-in as any LPG…

How am I shifting the topic?

Compatibility/modularity doesn't happen in the air, rather in written code.

So either one passes structs around, and somehow they need to be compatible.

Or one passes pointer + lenght as two separated variables, with the consequences to keep in sync two unrelated variables, from the compiler point of view.

Re: Banned C standard library functions in Git source code

#306

Post by pjmlp »

jstimpfle wrote: post #302
pjmlp wrote: post #301

Earlier quoted context omitted.

1) Totally optional from ISO C point of view, not all C compilers do do it, usually requires static analysis or specific compiler warnings to be enabled. Which not everyone does. 2) Solaris does it perfectly fine on SPARC thanks to tagged memory (ADI). Which Google in collaboration with ARM will make mandatory on future Android releases as well. [0] 3) It hardly mattered in MS-DOS and Amiga LOB applications developed…

1) It's not standardized but it's very simple to implement nevertheless. It is my understanding that 2) doesn't have anything to do with C (so given hardware support, you can have it for free whether working in C or not. I think this kind of invalidates your point). And also, 2) is not perfect, only probabilistic (a source I found says 94% likelyhood to detect OOB). And it works only for the most basic situations whe…

Pointer validation via hardware memory tagging has everything to do with C, because it is only due to C's shortcommings that millions of research dollars keep being spent to try to make it work.

In what concerns Solaris, and the requirements for future Android with ARM memory tagging, the system allocator is all there is, at least from official support point of view.

Hoare hardly needed to pursue such endevour, because all systems programming languages derived from Algol, like ESPOL, NEWP, PL/I, PL/S, PL/8, BLISS,.... were sane regarding bounds checking.

Hardware validation of memory is the only way to tame C, the alternative is to just dump the language, because as proven by ISO C11 dropping Annex K, very few actually care about making the language safe.

However given UNIX's dependency on C, it is also quite clear to me that in the next couple of decades C will be around, long after I am gone, and business opportunities to create companies on top of CVE exploits due to memory corruption bugs.

Re: Banned C standard library functions in Git source code

#307

Post by jstimpfle »

pjmlp wrote: post #306
jstimpfle wrote: post #302

Earlier quoted context omitted.

1) It's not standardized but it's very simple to implement nevertheless. It is my understanding that 2) doesn't have anything to do with C (so given hardware support, you can have it for free whether working in C or not. I think this kind of invalidates your point). And also, 2) is not perfect, only probabilistic (a source I found says 94% likelyhood to detect OOB). And it works only for the most basic situations whe…

Pointer validation via hardware memory tagging has everything to do with C, because it is only due to C's shortcommings that millions of research dollars keep being spent to try to make it work. In what concerns Solaris, and the requirements for future Android with ARM memory tagging, the system allocator is all there is, at least from official support point of view. Hoare hardly needed to pursue such endevour, becau…

> Pointer validation via hardware memory tagging has everything to do with C, because it is only due to C's shortcommings that millions of research dollars keep being spent to try to make it work.

So, to restate, Solaris does not do it "perfectly fine". Thanks for making my point.

> In what concerns Solaris, and the requirements for future Android with ARM memory tagging, the system allocator is all there is, at least from official support point of view.

That's a pity, because if you're not doing your own allocators then you'll have to accept lock contention, extreme memory overhead (for smaller allocations, say You're just not going to write a large infrastructure (i.e. performance-oriented) system in this way.

Re: Banned C standard library functions in Git source code

#308

Post by [deleted] »

pjmlp wrote: post #305
jstimpfle wrote: post #303

Earlier quoted context omitted.

> Which means either pointer+length get packed into their own structure to avoid mix up errors, thus requiring conversion function calls across libraries, or they are given separately manually, thus opening the door to the copy-paste mistakes from pointer+bad length that they are supposed to protect against. No, we were talking about compatibility/modularity. You're shifting the topic. > Qt is as locked-in as any LPG…

How am I shifting the topic? Compatibility/modularity doesn't happen in the air, rather in written code. So either one passes structs around, and somehow they need to be compatible. Or one passes pointer + lenght as two separated variables, with the consequences to keep in sync two unrelated variables, from the compiler point of view.

[deleted]

Re: Banned C standard library functions in Git source code

#309

Post by hibbelig »

pault wrote: post #172
zzo38computer wrote: post #109

Earlier quoted context omitted.

I don't use setjmp() and longjmp() often, but sometimes I do; I use goto more often than those (but still not all the time). (I want goto in JavaScript too. I figured out a algorithm to do so, but have not implemented it.) One use of setjmp/longjmp I have used is in ZORKMID, to deal with the debugger. At the beginning of the execute() function I have: while(setjmp(exception_buffer)); (The semicolon is correct; the lo…

What can you accomplish with goto that you can't accomplish with try/catch and custom errors?

> What can you accomplish with goto that you can't accomplish with try/catch and custom errors?

You can program in C. C doesn't have try/catch...

Re: Banned C standard library functions in Git source code

#310

Post by gbacon »

chris_wot wrote: post #92
alister wrote: post #26

I'm glad to see that setjmp() and longjmp() are still allowed. I'm just kidding by the way. For those C programmers who haven't encountered these before, it is a powerful way to do a "goto" in C. Powerful in the sense that you can jump anywhere, not limited to the same function. If it's used at all these days, it's used for exception handling. More info: https://en.wikipedia.org/wiki/Setjmp.h

I've always wondered why goto is actually considered harmful. I really should read the original paper by Djikstra...

See also Structured programming with go to statements[0] by Donald E. Knuth.

[0]: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.103...

Post reply on HN