Earlier quoted context omitted.
No, I didn't. An fst is a compressed data structure, which means you use it in its compressed form without decompressing it first. If you ported the fst crate to C, it would use the same technique. And in C, you have to design your data structures to be mmap friendly anyway. Same deal in Rust. But this is moving the goal posts. This thread started with "you can't do this." But you can. And I have. Multiple times. And…
> which means you use it in its compressed form without decompressing it first. So your code operates directly on a block of raw bytes? I can see how that can work with mmap without much problems. My argument was more about structured data (created using the type system), which is a level higher than raw bytes.
Correct. It's a finite state machine. The docs of the crate give links to papers if you want to drill down.
> My argument was more about structured data (created using the type system), which is a level higher than raw bytes.
Yes. You should be able to do in Rust whatever you would do in C. You can tag your types with `repr(C)` to get a consistent memory layout equivalent to whatever C does. But when you memory map stuff like this, you need to take at least all the same precautions as you would in C. That is, you need to build your data structures to be mmap friendly. The most obvious thing that is problematic for mmap structures like this that is otherwise easy to do is pointer indirection.
With that said, this technique is not common in Rust because it requires `unsafe` to do it. And when you use `unsafe`, you want to be sure that it's justified.
This is all really besides the point. You'd have the same problems if you read a file into heap memory. The main problem in Rust land with memory maps is that they don't fit into Rust's safety story in an obvious way. But this in and of itself doesn't make them inaccessible to you. It just makes it harder to reason about safety.