Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing C software without the standard library

weeb.ddns.net

Post reply on HN
Print view View original

141–150 of 188 posts

Re: Writing C software without the standard library

#141

Post by leeter »

bogomipz wrote: post #136
leeter wrote: post #123

A few thoughts: * The space savings are moot as other processes such as the daemons are going to load libc into virtual memory anyway, and the kernel shares libc's page among all processes. * This adds a lot of LOC you have to maintain, instead of shoving it off on the compiler/libc vendor, this increases the chance of bugs. * This will prevent the use of VDSOs to optimize high volume system calls like gettimeofday.…

Can you elaborate on why this "will prevent the use of VDSOs"? Why can't I just call gettimeofday() which will access the vdso page mapped into my address space?

Because the code that knows how to call the VDSOs is in libc, and how VDSOs work is very architecture dependent. Not saying you couldn't in theory do it. But at that point you might as well just load libc.

Re: Writing C software without the standard library

#142

Post by leeter »

jstimpfle wrote: post #135
leeter wrote: post #123

A few thoughts: * The space savings are moot as other processes such as the daemons are going to load libc into virtual memory anyway, and the kernel shares libc's page among all processes. * This adds a lot of LOC you have to maintain, instead of shoving it off on the compiler/libc vendor, this increases the chance of bugs. * This will prevent the use of VDSOs to optimize high volume system calls like gettimeofday.…

He spoke about the space savings of the binary itself (which must be debug things and statically linked code). There is no sharing of that , unless multiple processes run that same binary.

Unless the binary size exceeds VM page size this has no real value even in the unshared case. Because the OS is still going to have to map a full page for the executable to be loaded into.

That said techniques to allow a binary to fit under page size do have value; as the unused extra page can be used for other things.

Re: Writing C software without the standard library

#143

Post by goldfire »

userbinator wrote: post #80

A value in the range between -4095 and -1 indicates an error, it is -errno. The syscall/errno stuff has always seemed unusual, inelegant, and inefficient --- instead of just returning a negative error code directly, the function returns the vague "an error has occurred" -1, and you have to then check errno separately after that. It only adds insult to injury when you realise that the kernel itself isn't doing it, but…

> Unfortunately the Windows syscalls are not officially documented and even less stable than on Linux, changing even between service packs.

You can get pretty close though; it's possible to skip the C runtime and most of the other user space libraries and call into ntdll directly. Many of the functions it exports are fairly thin wrappers over the system calls.

Re: Writing C software without the standard library

#144

Post by bluecalm »

pawadu wrote: post #38

I think this is unnecessary when you got : typedef unsigned long int u64; typedef unsigned int u32; ... if you define your own types like this you may need to revise them when you switch architecture or even compiler. Now you could argue that this is part of the standard library, but I actually see it as a part of the standard C language.

A lot of good programmers do that and the reason given is that types in stdint.h have ugly long names like uint32_t. "_t" thing rubs a lot of people the wrong way so I am not surprised they change it to something more pleasant. I like using uint64, int32 etc. but the convention where it's u64, i32, f32, f64 is something I would get behind.

Re: Writing C software without the standard library

#145

Post by taeric »

leeter wrote: post #133
taeric wrote: post #127

Earlier quoted context omitted.

I'd be interested to see benchmarks to consider point one. At face value, I fully agree with your point and doubt it has any benefit. However, it is less that has to be swapped in for your program to run. Would be curious if this has an odd cache friendliness for an application. The tooling answer to this, it seems, would be to support statically linked libraries. But again, I would want to see numbers before persona…

So a few things to consider in regards to point 1: * Your executable is going to be loaded as a whole page regardless of how large it is, on most platforms this means you'll need at least 4k of User VM. * You'll need a page table, which has its own overhead. If someone was going to push back on the libc assertion I'd expect it here, a the PTEs for libc and and the VDSOs cannot be shared between processes (as far as I…

Right, my question is along the lines of avoiding the pages of libc. An easy question here would be how many pages libc takes up. I'm assuming not many, but more than one.

I think there is a strong argument that this page is often already paged into memory from everyone using it. However, if the function you used from it would have fit in the pages you were already using for your application, I could imagine some benefit.

I continue to stress, though, that this is just imagined. Numbers would be first thing I would have to collect before acting on this. (And I hope it doesn't sound like I am tasking you or anyone else with this. That is not my intent.)

Re: Writing C software without the standard library

#146

Post by jstimpfle »

leeter wrote: post #142
jstimpfle wrote: post #135

Earlier quoted context omitted.

He spoke about the space savings of the binary itself (which must be debug things and statically linked code). There is no sharing of that , unless multiple processes run that same binary.

Unless the binary size exceeds VM page size this has no real value even in the unshared case. Because the OS is still going to have to map a full page for the executable to be loaded into. That said techniques to allow a binary to fit under page size do have value; as the unused extra page can be used for other things.

That's totally true, and there is more overhead associated to processes. 8K executables are negligible. Some other languages produce executables in the dozens of megabytes.

Re: Writing C software without the standard library

#147

Post by jblow »

sincerel wrote: post #140
DSMan195276 wrote: post #86

Earlier quoted context omitted.

You're getting down-voted (Which is unnecessary IMO) but you're really not wrong. `gcc` itself provides `stdint.h`, it's not actually part of libc - or rather, you can use it without actually having a libc in place. Generally this is a good move, because you can always write a `stdint.h` replacement on arch's that don't have one, but on ones that do you're guaranteed to get the types correct.

I use stdint.h too, but I'm honestly curious if there is any common platform around today where one of the following asserts fails: int main() { assert(sizeof(signed char) == 1); assert(sizeof(short) == 2); assert(sizeof(int) == 4); assert(sizeof(long long) == 8); return 0; } I'm not interested in the language lawyering, because yes I know the standard provides more freedom to compilers. I just think those definition…

I think int is 8 bytes on the PS4. (I just got bitten by this...)

Re: Writing C software without the standard library

#148

Post by sqeaky »

lolisamurai wrote: post #124
vxNsr wrote: post #4

He claims that your code will be easy to port but then goes straight to Linux system calls. Still I like the idea. This is something that should be covered in a CS 102 type course. I know way to many cs guys who have no idea how to debug, let alone how their is being implemented.

Suppose you want to port to some architecture not supported by libc. If you were using libc you would have to find a replacement that works and targets that arch or port libc yourself. If you wrote everything from scratch, instead, you just have to read the specification and add support to your code. That's what I meant. Of course, if your target archs are all supported by libc, porting is much easier with libc.

Depending on the scope of your project porting libc itself might be good idea. Then you simplify porting any tools you need that also use libc. Then you can port a few extra dependencies, then more tools...

Re: Writing C software without the standard library

#149

Post by iainmerrick »

zimpenfish wrote: post #126
iainmerrick wrote: post #125

Earlier quoted context omitted.

Wow, that seems like a really bad approach for Go to take. So is anybody actually using Go on OS X? I guess maybe not, if all the real Go deployments are on servers running Linux.

I do a bunch of development work using Go on OSX but I do deploy the resulting solutions on Linux.

Aha, that's kind of what I'd figured. Work on Mac, deploy on Linux.

It makes me think twice about e.g. using Go as part of the build workflow in a big project if it's going to randomly stop working on various OS X versions.

Re: Writing C software without the standard library

#150

Post by leeter »

taeric wrote: post #145
leeter wrote: post #133

Earlier quoted context omitted.

So a few things to consider in regards to point 1: * Your executable is going to be loaded as a whole page regardless of how large it is, on most platforms this means you'll need at least 4k of User VM. * You'll need a page table, which has its own overhead. If someone was going to push back on the libc assertion I'd expect it here, a the PTEs for libc and and the VDSOs cannot be shared between processes (as far as I…

Right, my question is along the lines of avoiding the pages of libc. An easy question here would be how many pages libc takes up. I'm assuming not many, but more than one. I think there is a strong argument that this page is often already paged into memory from everyone using it. However, if the function you used from it would have fit in the pages you were already using for your application, I could imagine some ben…

At least two is the best answer I can give without specifying a specific libc and architecture. Something to think about is that the binary size of libc is only half the story. Even if the executable portion of libc fits in a single page. A libc implementation has a lot of per thread and per process statics it holds onto.

A good libc developer could actually put these into separate pages based on how often they change. In other words, if a static only is ever set once then coalesce it into a page with other statics that are only set once and are not process dependent.

Why that's important: because of fork, when fork creates a new process it sets the parent processes pages read only and then preforms copy on write when they are modified. In theory you can share both the binary and some of the statics between all the processes.

Post reply on HN