Live data from Hacker News

Kaitai: Describe the structure of data, not how you read or write it

kaitai.io

11–20 of 66 posts

Re: Kaitai: Describe the structure of data, not how you read or write it

#11
post #5

I do love kaitai and recently contributed a grammar, but both the title and the copy talk about writing. > Reading and writing binary formats is hard... Kaitai Struct tries to make this job easier... Kaitai cannot write data back out. [1] This is a major limitation for me. It would be nice to use it as a mutation engine for fuzzing, but without being able to write it back out, it is mostly just beneficial for analysi…

Any other lib that also generates Encoder/Writer code?

Re: Kaitai: Describe the structure of data, not how you read or write it

#12

This is very interesting, but just curious, when would you use something like this versus, say, protobufs?

If I understand it correctly, katai generates for you the reader as well, while protobuf generates only the data container. Katai is more a sort of protobuf + grpc for reading data.

Protobufs generate readers/writers for various languages (i.e. serialization/deserializiation).

gRPC is an RPC framework that uses Protobufs.

Re: Kaitai: Describe the structure of data, not how you read or write it

#13
post #5

I do love kaitai and recently contributed a grammar, but both the title and the copy talk about writing. > Reading and writing binary formats is hard... Kaitai Struct tries to make this job easier... Kaitai cannot write data back out. [1] This is a major limitation for me. It would be nice to use it as a mutation engine for fuzzing, but without being able to write it back out, it is mostly just beneficial for analysi…

Yikes, huge limitation. Guess I won't be looking into this any further.

It would be a CS breakthrough if it could... You're asking a lot...

Re: Kaitai: Describe the structure of data, not how you read or write it

#15
post #5

I do love kaitai and recently contributed a grammar, but both the title and the copy talk about writing. > Reading and writing binary formats is hard... Kaitai Struct tries to make this job easier... Kaitai cannot write data back out. [1] This is a major limitation for me. It would be nice to use it as a mutation engine for fuzzing, but without being able to write it back out, it is mostly just beneficial for analysi…

Any other lib that also generates Encoder/Writer code?

The Construct library for Python can do it:

https://construct.readthedocs.io/en/latest/intro.html#exampl...

I’ve long searched for something better than Construct, but so far I have yet to find even an equal.

Re: Kaitai: Describe the structure of data, not how you read or write it

#16

Earlier quoted context omitted.

Yikes, huge limitation. Guess I won't be looking into this any further.

It would be a CS breakthrough if it could... You're asking a lot...

Is this a known theoretical CS problem?

Re: Kaitai: Describe the structure of data, not how you read or write it

#17

Earlier quoted context omitted.

Yikes, huge limitation. Guess I won't be looking into this any further.

It would be a CS breakthrough if it could... You're asking a lot...

I might be missing something - why would this be a breakthrough? It sounds complicated to generate the interfaces, sure, but is there a theoretical problem blocking this, or just practical?

Re: Kaitai: Describe the structure of data, not how you read or write it

#18

Earlier quoted context omitted.

Yikes, huge limitation. Guess I won't be looking into this any further.

It would be a CS breakthrough if it could... You're asking a lot...

Are we talking about different things?

ASN.1 does more or less what I was hoping Kaitai can do. Where's the breakthrough?

I just want to be able to describe network protocols in some "language" and generate code that can serialize/deserialize it.

Re: Kaitai: Describe the structure of data, not how you read or write it

#20
I love Kaitai, and use it extensively (outside my day job) for exploring binary file formats of things like DOS games and quilting patterns. About a year ago I wrote a blog entry about using it for scientific data: https://matthewturk.github.io/post/kaitai-struct-scientific-... .

Recently I learned about some pretty cool work using it with respect to high-energy physics:

https://osf.io/2sner/

Post reply on HN