Guess it is similar to Apache Daffodil.
Kaitai: Describe the structure of data, not how you read or write it
31–40 of 66 posts
Re: Kaitai: Describe the structure of data, not how you read or write it
#32The first thing that comes to my mind in this context is that "kaitai" means "want to write" (書いたい) in Japanese. It also has many other meanings: https://jisho.org/search/kaitai . Maybe the authors had disassembly (解体) in mind.
Re: Kaitai: Describe the structure of data, not how you read or write it
#33This is a use-case for which a DSL is well-suited. They would be wise to abandon YAML.
Re: Kaitai: Describe the structure of data, not how you read or write it
#34Earlier quoted context omitted.
I might be missing something - why would this be a breakthrough? It sounds complicated to generate the interfaces, sure, but is there a theoretical problem blocking this, or just practical?
I've looked at this problem quite a bit over the years...I agree with you completely. there isn't anything fundamental here, just the normal cultural adoption issues, usability, etc. There may be some compilation/complexity issues around formats with variable length fields and self-description, but certainly less problematic than general purpose programming. I really wish though that there were more traction here, as…
It's a really interesting idea! I'm also surprised there hasn't been much traction here. I've started a Rust library for this: https://github.com/sharksforarms/deku
It's a declarative bit-level symmetrical reader writer generator library.
Re: Kaitai: Describe the structure of data, not how you read or write it
#35The first thing that comes to my mind in this context is that "kaitai" means "want to write" (書いたい) in Japanese. It also has many other meanings: https://jisho.org/search/kaitai . Maybe the authors had disassembly (解体) in mind.
Just a correction to point away from want to write. It's 書きたい (kakitai) for want to write (and it's probably not want to buy 買いたい).
Re: Kaitai: Describe the structure of data, not how you read or write it
#36Re: Kaitai: Describe the structure of data, not how you read or write it
#37Earlier quoted context omitted.
I've looked at this problem quite a bit over the years...I agree with you completely. there isn't anything fundamental here, just the normal cultural adoption issues, usability, etc. There may be some compilation/complexity issues around formats with variable length fields and self-description, but certainly less problematic than general purpose programming. I really wish though that there were more traction here, as…
Through this thread I really feel like I’m missing something. Are we not talking about writing binary files that conform to the spec? Like, in the case of a GIF: simply writing valid garbage data should produce a file that presents as a valid GIF with noise for the image. Similarly: reading the file through the parser and writing it out unmodified should create an identical file (assuming no stenography). Right?
You're right. The person saying this would be a breakthrough doesn't understand what this is doing.
There are already similarly declarative tools which can accomplish this. Haskell has binary parsing libraries which work similarly and give you both reading and writing capabilities.
Re: Kaitai: Describe the structure of data, not how you read or write it
#38Earlier quoted context omitted.
Yikes, huge limitation. Guess I won't be looking into this any further.
It would be a CS breakthrough if it could... You're asking a lot...
This breakthrough happened already. This is called bidirectional or invertible parsing. See it discussed here https://news.ycombinator.com/item?id=16392654
And this paper https://dl.acm.org/doi/10.1145/1863523.1863525 "Invertible syntax descriptions: unifying parsing and pretty printing"
And this Haskell library https://hackage.haskell.org/package/roundtrip among others
Re: Kaitai: Describe the structure of data, not how you read or write it
#39This is very interesting, but just curious, when would you use something like this versus, say, protobufs?
Kaitai Struct can read any 'ol random data format while protobufs (I assume) are only able to read/write a specific protocol. I was playing with it a while back along with wasm and got it to decode all the individual opcodes (along with the rest of the file) but it turned out to be really, really slow in the generated python version. C++ probably has much better performance but I haven't actually tested the differenc…
Last time I looked at it, it generated sub-objects as pointers allocated with new, which was a bit meh.
Re: Kaitai: Describe the structure of data, not how you read or write it
#40I do love kaitai and recently contributed a grammar, but both the title and the copy talk about writing. > Reading and writing binary formats is hard... Kaitai Struct tries to make this job easier... Kaitai cannot write data back out. [1] This is a major limitation for me. It would be nice to use it as a mutation engine for fuzzing, but without being able to write it back out, it is mostly just beneficial for analysi…
I guess the submission title is incorrect then:
>Kaitai: Describe the structure of data, not how you read or write it