Live data from Hacker News

Are you sure you want to use MMAP in your database management system? [pdf]

db.cs.cmu.edu

131–137 of 137 posts

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#131

Earlier quoted context omitted.

Why are you constantly ignoring what I'm writing? > It doesn't matter what the OS does or doesn't do in the background, atomically reading and writing to memory is unaffected. That's not true. If this thing is file backed there is usually no guarantee that the page of virtual memory (i.e. a page of the file data) you're accessing is present in physical memory. You'll cause page faults and data transfers to/from disk.…

This can delay the execution of an atomic read or write You can play "what if" all you want if you don't know what else running, but this was always about lock free interprocess communication, which is not broken by a page fault or process suspension. An atomic instruction by design will do everything it needs to when the instruction runs. Saying the OS can ultimately control the execution of a process is a nonsense…

Dude, the example I gave you with shm_open() is creating anonymous memory. That's just what non-file-backed mappings are called, no matter how long you want to keep obsessing about any "file paths".

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#132

Earlier quoted context omitted.

I'm starting to think you're even more confused than I had assumed. You were literally given a reasonable possible answer to your question multiple times (MAP_ANONYMOUS). And if there wasn't a big confusion you wouldn't be asking these questions in the first place because you could just make up your own answer. I'm also left uncertain if you're assuming Linux and not talking about it. At least your objections to gene…

"MAP_ANONYMOUS|MAP_SHARED mapped memory can only be accessed by the process which does that mmap() call or its child processes. There is no way for another process to map the same memory because that memory can not be referred to from elsewhere since it is anonymous." https://stackoverflow.com/questions/4991533/sharing-memory-b... misunderstanding what I meant by "file-backed memory" No, it started by talking about u…

> This thread is about mmap - it says it in the title.

I was asking what YOU are talking about. And also, this thread is actually about the approach of memory-mapped file I/O, not about POSIX mmap() specifically.

That's why I was (clearly) making statements that are not tied to any particular OS or platform, from the beginning.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#133

Earlier quoted context omitted.

This just goes back to the same question - what do two processes use to map the same memory into their memory space if it isn't a path to a file? I'm not saying there isn't anything, I'm just seeing an extreme avoidance to an actual answer. The other guy went down a rabbit hole of syncing that memory to storage, which has nothing to do with anything.

Please just RTFM on `mmap(... MAP_SHARED | MAP_ANONYMOUS, -1, 0);`. There is no file path involved. That's all I can say to you now, apart from "this is not StackOverflow".

But then that isn't interproccess communication.

If your boss said "we need these two programs to have lock free IPC through memory" and you said "use MAP_ANONYMOUS" they would say "that is local to the process tree and won't work".

You can try to ignore the context of this thread, but if someone wants IPC, this doesn't work.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#134

Earlier quoted context omitted.

This can delay the execution of an atomic read or write You can play "what if" all you want if you don't know what else running, but this was always about lock free interprocess communication, which is not broken by a page fault or process suspension. An atomic instruction by design will do everything it needs to when the instruction runs. Saying the OS can ultimately control the execution of a process is a nonsense…

Dude, the example I gave you with shm_open() is creating anonymous memory. That's just what non-file-backed mappings are called, no matter how long you want to keep obsessing about any "file paths".

This doesn't even seem like a reply to what I said.

If you map memory anonymously you aren't doing interprocess communication.

If you don't, you have a file path that the other program can use to map the same memory.

That's it, there is nothing wrong with this. I don't know why this is so upsetting. Mapping memory anonymously is local to the process tree and doesn't work for two different programs communicating.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#135

Earlier quoted context omitted.

Dude, the example I gave you with shm_open() is creating anonymous memory. That's just what non-file-backed mappings are called, no matter how long you want to keep obsessing about any "file paths".

This doesn't even seem like a reply to what I said. If you map memory anonymously you aren't doing interprocess communication. If you don't, you have a file path that the other program can use to map the same memory. That's it, there is nothing wrong with this. I don't know why this is so upsetting. Mapping memory anonymously is local to the process tree and doesn't work for two different programs communicating.

[deleted]

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#136

Earlier quoted context omitted.

Dude, the example I gave you with shm_open() is creating anonymous memory. That's just what non-file-backed mappings are called, no matter how long you want to keep obsessing about any "file paths".

This doesn't even seem like a reply to what I said. If you map memory anonymously you aren't doing interprocess communication. If you don't, you have a file path that the other program can use to map the same memory. That's it, there is nothing wrong with this. I don't know why this is so upsetting. Mapping memory anonymously is local to the process tree and doesn't work for two different programs communicating.

Ok, I'm extremely embarassed but it looks like I got the terminology wrong with regards to "Anonymous memory". And sorry for being so upset, at least I finally got something out of it.

It's also a fact that if I'm using disk swap space on a Unix, the same performance and stability issues apply as for disk backed file mappings. In that sense, there really is no difference.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#137

Earlier quoted context omitted.

Please just RTFM on `mmap(... MAP_SHARED | MAP_ANONYMOUS, -1, 0);`. There is no file path involved. That's all I can say to you now, apart from "this is not StackOverflow".

But then that isn't interproccess communication. If your boss said "we need these two programs to have lock free IPC through memory" and you said "use MAP_ANONYMOUS" they would say "that is local to the process tree and won't work". You can try to ignore the context of this thread, but if someone wants IPC, this doesn't work.

> But then that isn't interproccess communication.

It is. It may not be _generic_ IPC, but it is IPC all the same. E.g., this is how postgres does IPC across its processes.

> that is local to the process tree and won't work

Isn't that what SHM is for? But, oh I see, you're willfully ignoring the fact that SHM keys _are not file paths_. So, yeah, I guess in _your_ world, non-file-backed IPC can't work.

> If your boss said ...

Sucks to be your boss, since _you_ don't get the fact that SHM keys and the filesystem are entirely separate namespaces.

Post reply on HN