Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does C have the best file API

maurycyz.com

Post reply on HN
Print view View original

161–169 of 169 posts

Re: Why does C have the best file API

#161

Post by ozgrakkurt »

amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

Mapping a struct from binary buffers is actually a very good idea if you know how it works.

Flatbuffers etc. is cool but they can be very bloaty and clunky.

Re: Why does C have the best file API

#162

Post by ozgrakkurt »

gopalv wrote: post #64
amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

> correspond to a binary format in accordance with the C ABI on your particular system. We're so deep in this hole that people are fixing this on a CPU with silicon. The Graviton team made a little-endian version of ARM just to allow lazy code like this to migrate away from Intel chips without having to rewrite struct unpacking (& also IBM with the ppc64le). Early in my career, I spent a lot of my time reading Java b…

I would question why is it big endian in the first place. Little endian is obviously more popular, why use big endian at all?

Re: Why does C have the best file API

#163

Post by direwolf20 »

amluto wrote: post #61
Negitivefrags wrote: post #58

Earlier quoted context omitted.

Why is it such a terrible idea? No need to add complexity, dependancies and reduced performance by using these libraries.

Lots of reasons: The code is not portable between architectures. You can’t actually define your data structure. You can pretend with your compiler’s version of “pack” with regrettable results. You probably have multiple kinds of undefined behavior. Dealing with compatibility between versions of your software is awkward at best. You might not even get amazing performance. mmap is not a panacea. Page faults and TLB flu…

"Portable" has originally meant "able to be ported" and not "is already ported"

Re: Why does C have the best file API

#164

Post by rerdavies »

amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

mmap is not part of ISO C. mmap is part of POSIX 2008, but MSVC/Windows does not support it.

Re: Why does C have the best file API

#165

Post by alanfranz »

okanat wrote: post #34
alanfranz wrote: post #21

Well... I'm not sure what the author really wants to say. mmap is available in many languages (e.g. Python) on Linux (and many other *nix I suppose). C provides you with raw memory access, so using mmap is sort-of-convenient for this use case. But if you use Python then, yes, you'll need a bytearray, because Python doesn't give you raw access to such memory - and I'm not sure you'd want to mmap a PyObject anyway? The…

Creating memory mapped files is a very common OS feature since 90s. Many high level languages have it as OS agnostic POSIX or not.

And?

Did I claim something different? I just didn’t use that feature on other OSes.

Re: Why does C have the best file API

#166

Post by hyc_symas »

amluto wrote: post #55

I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes , but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea . And C doesn’t…

How often does anyone care about using data on a different system than it was created on?

These days, any C struct you built on amd64 will work identically on arm64. There really aren't any other architectures that matter.

And yes, managing concurrent access to shared resources requires care and cooperation. That has always been true, and has nothing specific to do with mmap.

Re: Why does C have the best file API

#167

Post by hyc_symas »

phoronixrly wrote: post #37
Dwedit wrote: post #12

Using mmap means that you need to be able to handle memory access exceptions when a disk read or write fails. Examples of disk access that fails includes reading from a file on a Wifi network drive, a USB device with a cable that suddenly loses its connection when the cable is jiggled, or even a removable USB drive where all disk reads fail after it sees one bad sector. If you're not prepared to handle a memory acces…

Ah, reminds me of 'Are You Sure You Want to Use MMAP in Your Database Management System? (2022)' https://db.cs.cmu.edu/mmap-cidr2022/

Ah yes, the ever popular "mongoDB's developers were incompetent therefore mmap is bad" paper.

Pure tripe. https://www.symas.com/post/are-you-sure-you-want-to-use-mmap...

Re: Why does C have the best file API

#168

Post by ozgrakkurt »

fc417fc802 wrote: post #97
fulafel wrote: post #80

Earlier quoted context omitted.

It's very different since at random points of your program your signal handler is caleld asynchronously, and you can only do a very limited signal-safe things there, and the flow of control in your i/o, logic etc code has no idea it's happening. tldr; it's very different.

Well at least in this case the timing won't be arbitrary. Execution will have blocked waiting on the read and you will (AFAIK) receive the signal promptly in this case. Since the code in question was doing IO that you knew could fail handling the situation can be as simple as setting a flag from within the signal handler. I'm unclear what would happen in the event you had configured the mask to force SIGBUS to a diff…

Signals are extremely bad to work with. Would rather do error handling in javascript. It feels like trying to write low level primitives in rust or trying to learn c++. There are so many edge cases that I start questioning what am I doing with my life

Re: Why does C have the best file API

#169

Post by namibj »

nixon_why69 wrote: post #98
namibj wrote: post #70

Earlier quoted context omitted.

Nah, usually can't have huge pages. Almost certainly can't have giant pages. Can't even fit all L3$ capacity into the L2 TLB if done via 4k pages...

I hadn't thought of that but apparently Linux at least has had support for a while, according to manpage? https://man7.org/linux/man-pages/man2/mmap.2.html

mmap is also for non-disk-backed memory:

Your link refers to https://www.kernel.org/doc/Documentation/admin-guide/mm/huge... ,

which contains this tidbit:

> If the user applications are going to request huge pages using mmap system call, then it is required that system administrator mount a file system of type hugetlbfs::

Note this otherwise has semantics similar to tmpfs; notably, it's usage is mutually exclusive with being able to supply a disk file fd to mmap!

Post reply on HN