Live data from Hacker News

Why does C have the best file API

maurycyz.com

51–60 of 169 posts

Re: Why does C have the best file API

#51
post #32

I guess the author didn't use that many other programming languages or OSes. You can do the same even in garbage collected languages like Java and C# and on Windows too. https://docs.oracle.com/javase/8/docs/api/java/nio/MappedByt... https://learn.microsoft.com/en-us/dotnet/api/system.io.memor... https://learn.microsoft.com/en-us/windows/win32/memory/creat... Memory mapping is very common.

And since Java 4 no less.

Re: Why does C have the best file API

#52
post #14

Aside from what https://news.ycombinator.com/item?id=47210893 said, mmap() is a low-level design that makes it easier to work with files that don't fit in memory and fundamentally represent a single homogeneous array of some structure. But it turns out that files commonly do fit in memory (nowadays you commonly have on the order of ~100x as much disk as memory, but millions of files); and you very often want to read…

mmap is also relatively slow (compared to modern solutions, io_uring and friends), and immensely painful for error handling. It's simple, I'll give it that.

Page faults are slower than being deliberate about your I/O but mapped memory is no faster or slower than "normal" memory, its the same mechanism.

Re: Why does C have the best file API

#53
post #32

I guess the author didn't use that many other programming languages or OSes. You can do the same even in garbage collected languages like Java and C# and on Windows too. https://docs.oracle.com/javase/8/docs/api/java/nio/MappedByt... https://learn.microsoft.com/en-us/dotnet/api/system.io.memor... https://learn.microsoft.com/en-us/windows/win32/memory/creat... Memory mapping is very common.

[deleted]

Re: Why does C have the best file API

#54
post #33

Earlier quoted context omitted.

Address space size and RAM are two different things.

What they said is correct regardless of that though?

The point the article makes is that a 32GB file can be mmapped even if you only have 8GB of memory available - it wasn't talking about the address space. So the response is irrelevant even if technically correct

Re: Why does C have the best file API

#55
I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes, but:

> (as far as I'm aware) C is the only language that lets you specify a binary format and just use it.

I assume they mean:

    struct foo { fields; };
    foo *data = mmap(…);
And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea. And C doesn’t even let you specify a binary format — it lets you write a struct that will correspond to a binary format in accordance with the C ABI on your particular system.

If you want to access a file containing a bunch of records using mmap, and you want a well defined format and good performance, then use something actually intended for the purpose. Cap’n Proto and FlatBuffers are fast but often produce rather large output; protobuf and its ilk are more space efficient and very widely supported; Parquet and Feather can have excellent performance and space efficiency if you use them for their intended purposes. And everything needs to deal with the fact that, if you carelessly access mmapped data that is modified while you read it in any C-like language, you get UB.

Re: Why does C have the best file API

#56
It may have a tidy mmap api, but Smalltalk has a much better file api through its Streams hierarchy IMHO. You can create a stream on a diskfile, you can create a stream on a byteArray, you can create a stream on standard Unix streams, you can create a stream on anything where "next" makes sense.

Re: Why does C have the best file API

#57
post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

Had the same thought. Also confused at the backhanded compliment that pickle got:

> Just look at Python's pickle: it's a completely insecure serialization format. Loading a file can cause code execution even if you just wanted some numbers... but still very widely used because it fits the mix-code-and-data model of python.

Like, are they saying it's bad? Are they saying it's good? I don't even get it. While I was reading the post, I was thinking about pickle the whole time (and how terrible that idea is, too).

Re: Why does C have the best file API

#58
post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

Why is it such a terrible idea?

No need to add complexity, dependancies and reduced performance by using these libraries.

Re: Why does C have the best file API

#60
post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

Why is it such a terrible idea? No need to add complexity, dependancies and reduced performance by using these libraries.

No defined binary encoding, no guarantee about concurrent modifications, performance trade-offs (mmap is NOT always faster than sequential reads!) and more.
Post reply on HN