Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does C have the best file API

maurycyz.com

Post reply on HN
Print view View original

91–100 of 169 posts

Re: Why does C have the best file API

#91

Post by bloppe »

jraph wrote: post #62
Negitivefrags wrote: post #58

Earlier quoted context omitted.

Why is it such a terrible idea? No need to add complexity, dependancies and reduced performance by using these libraries.

Because a struct might not serialize the same way from a CPU architecture to another. The sizes of ints, the byte order and the padding can be different for instance.

C has had fixed size int types since C99. And you've always been able to define struct layouts with perfect precision (struct padding is well defined and deterministic, and you can always use __attribute__(packed) and bit fields for manual padding).

Endianness might kill your portability in theory. but in practice, nobody uses big endian anymore. Unless you're shipping software for an IBM mainframe, little endian is portable.

Re: Why does C have the best file API

#92

Post by nly »

ddtaylor wrote: post #79
Negitivefrags wrote: post #66

Earlier quoted context omitted.

I've written a lot of code using that method, and never had any portability issues. You use types with number of bits in them. Hell, I've slung C structs across the network between 3 CPU architectures. And I didn't even use htons! Maybe it's not portable to some ancient architecture, but none that I have experienced. If there is undefined behavior, it's certainly never been a problem either. And I've seen a lot of ta…

C allows most of this, whereas C++ doesn't allow pointer aliasing without a compiler flag, tricks and problems. I agree you can certainly just use bytes of the correct sizes, but often to get the coverage you need for the data structure you end up writing some form of wrapper or fixup code, which is still easier and gives you the control versus most of the protobuf like stuff that introduces a lot of complexity and t…

__attribute__((may_alias, packed)) right on the struct.

Re: Why does C have the best file API

#93

Post by fc417fc802 »

bloppe wrote: post #91
jraph wrote: post #62

Earlier quoted context omitted.

Because a struct might not serialize the same way from a CPU architecture to another. The sizes of ints, the byte order and the padding can be different for instance.

C has had fixed size int types since C99. And you've always been able to define struct layouts with perfect precision (struct padding is well defined and deterministic, and you can always use __attribute__(packed) and bit fields for manual padding). Endianness might kill your portability in theory. but in practice, nobody uses big endian anymore. Unless you're shipping software for an IBM mainframe, little endian is…

It can be made to work (as you point out), and the core idea is great, but the implementation is terrible. You have to stop and think about struct layout rules rather than declaring your intent and having the compiler check for errors. As usual C is a giant pile of exquisitely crafted footguns.

A "sane" version of the feature would provide for marking a struct as intended for ser/des at which point you'd be required to spell out every last alignment, endianness, and bit width detail. (You'd still have to remember to mark any structs used in conjunction with mmap but C wouldn't be any fun if it was safe.)

Re: Why does C have the best file API

#94

Post by cb321 »

socalgal2 wrote: post #81
amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

it's not a terrible idea. It has it's uses. You just have to know when to use it and when not to use it. For example, to have fast load times and zero temp memory overhead I've used that for several games. Other than changing a few offsets to pointers the data is used directly. I don't have to worry about incompatibilities. Either I'm shipping for a single platform or there's a different build for each platform, incl…

To support your point, it's also used in basically every shared library / DLL system. While usually used "for code", a "shared pure data library" has many applications. There are also 3rd party tools to make this convenient from many PLangs like HDF5, https://github.com/c-blake/nio with its FileArray for Nim, Apache Arrow, etc.

Unmentioned so far is that defaults for max live memory maps are usually much higher than defaults for max open files. So, if you are careful about closing files after mapping, you can usually get more "range" before having to move from OS/distro defaults. (E.g. for `program foo*`-style work where you want to keep the foo open for some reason, like binding them to many read-only NumPy array variables.)

Re: Why does C have the best file API

#95

Post by johannes1234321 »

nly wrote: post #92
ddtaylor wrote: post #79

Earlier quoted context omitted.

C allows most of this, whereas C++ doesn't allow pointer aliasing without a compiler flag, tricks and problems. I agree you can certainly just use bytes of the correct sizes, but often to get the coverage you need for the data structure you end up writing some form of wrapper or fixup code, which is still easier and gives you the control versus most of the protobuf like stuff that introduces a lot of complexity and t…

__attribute__((may_alias, packed)) right on the struct.

That is not C, but a non-standard extension and thus not portable.

Re: Why does C have the best file API

#96

Post by Surac »

After reading the comments here it boils down to: But my language is better then yours. mmap is not a feature of C. Some more modern languages try to prevent people form shooting in there feet and only allow byte wise access to such mmaped regions. The have a point doing this, but on the other hand also the C-Users have a valid point. Safety and Speed are 2 Factors you have to consider using the tools you use. From a Hardware point of view C might be more direct but it also enables you to make "stupid" errors fast. More Modern languages prevent you from the "stupid" errors but make you copy or transform the data more. Scotty from the Enterprise sayed once: Allways use the fitting tool

Re: Why does C have the best file API

#97

Post by fc417fc802 »

fulafel wrote: post #80
tremon wrote: post #46

Earlier quoted context omitted.

C doesn't have exceptions, do you mean signals? If not, I don't see how that is that any different from having to handle I/O errors from write() and/or open() calls.

It's very different since at random points of your program your signal handler is caleld asynchronously, and you can only do a very limited signal-safe things there, and the flow of control in your i/o, logic etc code has no idea it's happening. tldr; it's very different.

Well at least in this case the timing won't be arbitrary. Execution will have blocked waiting on the read and you will (AFAIK) receive the signal promptly in this case. Since the code in question was doing IO that you knew could fail handling the situation can be as simple as setting a flag from within the signal handler.

I'm unclear what would happen in the event you had configured the mask to force SIGBUS to a different thread. Presumably undefined behavior.

> If multiple standard signals are pending for a process, the order in which the signals are delivered is unspecified.

That could create the mother of all edgecases if a different signal handler assumed the variable you just failed to read into was in a valid state. More fun footguns I guess.

Re: Why does C have the best file API

#98

Post by nixon_why69 »

namibj wrote: post #70
nixon_why69 wrote: post #52

Earlier quoted context omitted.

Page faults are slower than being deliberate about your I/O but mapped memory is no faster or slower than "normal" memory, its the same mechanism.

Nah, usually can't have huge pages. Almost certainly can't have giant pages. Can't even fit all L3$ capacity into the L2 TLB if done via 4k pages...

I hadn't thought of that but apparently Linux at least has had support for a while, according to manpage? https://man7.org/linux/man-pages/man2/mmap.2.html

Re: Why does C have the best file API

#99

Post by josefx »

johannes1234321 wrote: post #95
nly wrote: post #92

Earlier quoted context omitted.

__attribute__((may_alias, packed)) right on the struct.

That is not C, but a non-standard extension and thus not portable.

> non-standard extension and thus not portable

Modern versions of standard C aren't very portable either, unless you plan to stick to the original version of K&R C you have to pick and choose which implementations you plan to support.

Re: Why does C have the best file API

#100

Post by ErroneousBosh »

pjmlp wrote: post #85
amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

Yeah, and as you well put it, it isn't even some snowflake feature only possible in C. The myth that it was a gift from Gods doing stuff nothing else can make it, persists. And even on the languages that don't, it isn't if as a tiny Assembly thunk is the end of the world to write, but apparently at a sign of a plain mov people run to the hills nowadays.

> And even on the languages that don't, it isn't if as a tiny Assembly thunk is the end of the world to write, but apparently at a sign of a plain mov people run to the hills nowadays.

Use the right tool for the job. I've always felt it's often the most efficient thing to write a bit of code in assembler, if that's simpler and clearer than doing anything else.

It's hard to write obfuscated assembler because it's all sitting opened up in front of you. It's as simple as it gets and it hasn't got any secrets.

Post reply on HN