11–20 of 23 posts
For such a problem, I really recommend https://kaitai.io/ From their website: > Kaitai Struct is a declarative language used to describe various binary data structures, laid out in files or in memory: i.e. binary file formats, network stream packet formats, etc. They even have a Rust interface: https://github.com/kaitai-io/kaitai_struct_rust_runtime
I really wish there was a safe way to initialize structs with arbitrary binary data in rust. Perhaps I'm missing something, but it seems intuitively safe, especially if the struct only contains integer primitives.
I really wish there was a safe way to initialize structs with arbitrary binary data in rust. Perhaps I'm missing something, but it seems intuitively safe, especially if the struct only contains integer primitives.
How would it be possible to make it safe? The compiler can't verify that N random bytes input from an arbitrary source are going to be valid for the types in the struct - you have to tell tell the compiler "trust this even if you can't verify it from the source code" - hence the unsafe keyword.
I really wish there was a safe way to initialize structs with arbitrary binary data in rust. Perhaps I'm missing something, but it seems intuitively safe, especially if the struct only contains integer primitives.
How would it be possible to make it safe? The compiler can't verify that N random bytes input from an arbitrary source are going to be valid for the types in the struct - you have to tell tell the compiler "trust this even if you can't verify it from the source code" - hence the unsafe keyword.
Earlier quoted context omitted.
The rust structure is defined as repr(C) so there is no reordering.
Does the C spec actually specify it needs to be in that order? Been awhile since I messed that that sort of thing. I know in practice they typically just put them in the same order and align on some sort of default packing depending on CPU. Usually to minimize on extra instructions. I have eeked out some extra perf on some CPUs and structs just by moving stuff around and making sure it fits into a cacheline. Sometime…
I think c struct is missing alignment attribute? This idea has been explored in flatbuffers, capnproto and probably others.
C structs are generally assumed to be laid out in the order defined with proper padding automatically inserted (as most platforms at best dislike unaligned accesses). Alignment information should only be necessary if you want to pack the structure or need to apply wider than standard alignment (e.g. need to align a 32 or 64b type to 128 bytes). The one possible issue is that reading padding is UB (i think). But here…
How does one calculate this offset based on the fields in the struct?
Earlier quoted context omitted.
How would it be possible to make it safe? The compiler can't verify that N random bytes input from an arbitrary source are going to be valid for the types in the struct - you have to tell tell the compiler "trust this even if you can't verify it from the source code" - hence the unsafe keyword.
For integer primitives, all possible values are valid. Just allowing binary initialization for structs that consist of integer types would allow compatibility with C structure. As it is, I have to either resort to unsafe code or manually deserialize every struct
Earlier quoted context omitted.
How would it be possible to make it safe? The compiler can't verify that N random bytes input from an arbitrary source are going to be valid for the types in the struct - you have to tell tell the compiler "trust this even if you can't verify it from the source code" - hence the unsafe keyword.
Enter safe transmute - https://github.com/rust-lang/project-safe-transmute/blob/mas...