Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does C have the best file API

maurycyz.com

Post reply on HN
Print view View original

81–90 of 169 posts

Re: Why does C have the best file API

#81

Post by socalgal2 »

amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

it's not a terrible idea. It has it's uses. You just have to know when to use it and when not to use it.

For example, to have fast load times and zero temp memory overhead I've used that for several games. Other than changing a few offsets to pointers the data is used directly. I don't have to worry about incompatibilities. Either I'm shipping for a single platform or there's a different build for each platform, including the data. There's a version in the first few bytes just so during dev we don't try to load old format files with new struct defs. But otherwise, it's great for getting fast load times.

Re: Why does C have the best file API

#83

Post by pjmlp »

Negitivefrags wrote: post #66
amluto wrote: post #61

Earlier quoted context omitted.

Lots of reasons: The code is not portable between architectures. You can’t actually define your data structure. You can pretend with your compiler’s version of “pack” with regrettable results. You probably have multiple kinds of undefined behavior. Dealing with compatibility between versions of your software is awkward at best. You might not even get amazing performance. mmap is not a panacea. Page faults and TLB flu…

I've written a lot of code using that method, and never had any portability issues. You use types with number of bits in them. Hell, I've slung C structs across the network between 3 CPU architectures. And I didn't even use htons! Maybe it's not portable to some ancient architecture, but none that I have experienced. If there is undefined behavior, it's certainly never been a problem either. And I've seen a lot of ta…

Some people also don't use protective gear when going downhill biking, it is a matter of feeling lucky.

Re: Why does C have the best file API

#84

Post by torginus »

gopalv wrote: post #64
amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

> correspond to a binary format in accordance with the C ABI on your particular system. We're so deep in this hole that people are fixing this on a CPU with silicon. The Graviton team made a little-endian version of ARM just to allow lazy code like this to migrate away from Intel chips without having to rewrite struct unpacking (& also IBM with the ppc64le). Early in my career, I spent a lot of my time reading Java b…

I thought all iterations of ARM are little endian, even going back as far to ARM7. same as x86?

The only big-endian popular arch in recent memory is PPC

Re: Why does C have the best file API

#85

Post by pjmlp »

amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

Yeah, and as you well put it, it isn't even some snowflake feature only possible in C.

The myth that it was a gift from Gods doing stuff nothing else can make it, persists.

And even on the languages that don't, it isn't if as a tiny Assembly thunk is the end of the world to write, but apparently at a sign of a plain mov people run to the hills nowadays.

Re: Why does C have the best file API

#86

Post by pjmlp »

Const-me wrote: post #44
SvenL wrote: post #31

Earlier quoted context omitted.

I think you don’t need to be unsafe, they have normal API for it. https://learn.microsoft.com/en-us/dotnet/standard/io/memory-...

Indeed, but these normal APIs have runtime costs for bounds checking. For some use cases, unsafe can be better. For instance, last time I used a memory-mapped file was for a large immutable Bloom filter. I knew the file should be exactly 4GB, validated that in the constructor, then when testing 12 bits from random location of the mapped file on each query, I opted for unsafe codes.

It is a matter of the deployment scenario, in the days people ship Electron, and deploy production code in CPython, those bounds checking don't hurt at all.

When they do, thankfully there is unsafe if needed.

Re: Why does C have the best file API

#88

Post by qalmakka »

ajross wrote: post #20
seba_dos1 wrote: post #3

mmap is not a C feature, but POSIX. There are C platforms that don't provide mmap, and on those that do you can use mmap from other languages (there's mmap module in the Python's standard library, for example).

I think this is sort of missing the point, though. Yes, mmap() is in POSIX[1] in the sense of "where is it specified". But mmap() was implemented in C because C is the natural language for exposing Unix system calls and mmap() is a syscall provided by the OS. And this is true up and down the stack. Best language for integrating with low level kernel networking (sockopts, routing, etc...)? C. Best language for async I…

> C is the natural language for exposing Unix system calls

No, C is the language _designed_ to write UNIX. Unix is older than C, C was designed to write it and that's why all UNIX APIs follow C conventions. It's obvious that when you design something for a system it will have its best APIs in the language the system is written in.

C has also multiple weird and quirky APIs that suck, especially in the ISO C libc.

Re: Why does C have the best file API

#89

Post by masklinn »

torginus wrote: post #84
gopalv wrote: post #64

Earlier quoted context omitted.

> correspond to a binary format in accordance with the C ABI on your particular system. We're so deep in this hole that people are fixing this on a CPU with silicon. The Graviton team made a little-endian version of ARM just to allow lazy code like this to migrate away from Intel chips without having to rewrite struct unpacking (& also IBM with the ppc64le). Early in my career, I spent a lot of my time reading Java b…

I thought all iterations of ARM are little endian, even going back as far to ARM7. same as x86? The only big-endian popular arch in recent memory is PPC

AFAIK ARM is generally bi-endian, though systems using BE (whether BE32 or BE8) are few and far between.

Re: Why does C have the best file API

#90

Post by flohofwoe »

seba_dos1 wrote: post #3

mmap is not a C feature, but POSIX. There are C platforms that don't provide mmap, and on those that do you can use mmap from other languages (there's mmap module in the Python's standard library, for example).

And it's not just mmap(), all the functions in the code snippet except printf() are not actually C stdlib functions.
Post reply on HN