21–30 of 73 posts
Earlier quoted context omitted.
does that invalidate the criticisms?
recency trolling; it's trivial to find the date of publication, but some people seem to think they can get karma points by their selfless public service...
I think one of the most desirable and under-appreciated goals of schema languages and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs. My read of Cap’n’Proto didn’t make it sound that safety was the highest priority. At least not above perfo…
Having said that, DFDL fails pretty miserably by the standards set in the article. The main design goal was to be able to describe as many existing data formats as possible, which means the spec is massive and supports a lot of bad ideas.
Despite having its 1.0 release in 2015, and being the most complete implementation, Apache Daffodil still does not fully implement the DFDL spec. And it is not an easy code base to jump into and understand.
Earlier quoted context omitted.
right, and it should not be left to the serialization layer for that.
it depends on what type of safety. The schema language might for example allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer, same for array/list/vector length.
Earlier quoted context omitted.
recency trolling; it's trivial to find the date of publication, but some people seem to think they can get karma points by their selfless public service...
No, some people go to Hacker News to read things that are new . But people love reposting so the compromise is to put the date in the title for old posts so we can easily skip over them.
Rest in peace, Ian, my friend.
For context: it seems that the author passed away on last July https://www.winchesteruu.org/2023/07/25/joys-sorrows-and-tra...
Earlier quoted context omitted.
Safety means: garbage in, error out.
right, and it should not be left to the serialization layer for that.
A while ago I read a great article about how the Adobe PDF serialization format is nearly impossible to secure because it allows inherently unsafe constructs.
For example, it allows cross-references that are basically just arbitrary unaligned pointers. It uses many different alignment and padding algorithms. It has length-prefixed and not-length prefixed sections. Etc, etc...
Apparently it was a serious research exercise to make a safe PDF parser, and they only covered a fraction of the full spec!
To put things in perspective: Originally, PDF allowed arbitrary code execution as a core feature, allowing the output of shell commands to be used as document content.
Most people like the Chromium and Firefox teams have just given up and now parse PDF using a sandboxed JavaScript VM because it's too hard to do it safely with C++. They parse HTML and JavaScript with C++, but not PDF. Think about that.
A similar issue caused Log4j, where a "format string parser" contained a vulnerability because it was too flexible and allowed network requests to be triggered by user-controlled data.
Even trivial, "surely it must be safe" formats like XML and JSON are riddled with security issues, such as different layers in a microservice architecture having different handling semantics for duplicate keys, null values, etc... This can result in exploits such as authentication and authorization tokens being interpreted by a system one way, but a different way by a different system. For real-world attacks along these lines, search for "request smuggling".
Serialization and parsing are security minefields and it is dangerously naive to just hand-wave that away.
I think one of the most desirable and under-appreciated goals of schema languages and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs. My read of Cap’n’Proto didn’t make it sound that safety was the highest priority. At least not above perfo…
I agree entirely, and this is one of my single greatest frustrations with the majority of the current popular IDLs/schema languages. ASN.1 is hilariously bad in a lot of ways, but one thing it gets absolutely right is strong typing and being able to express constraints (ranges, values dependent on other values). That combined with a canonicalized encoding form (DER) goes a long way in making various error states unre…
Earlier quoted context omitted.
it depends on what type of safety. The schema language might for example allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer, same for array/list/vector length.
It feels like a check against an input size of 10MB is something you would do well before deserialization, no?
Earlier quoted context omitted.
right, and it should not be left to the serialization layer for that.
Security is a concern for every layer. It's not magic pixie dust that' can be sprinkled on top of software to renders it secure! A while ago I read a great article about how the Adobe PDF serialization format is nearly impossible to secure because it allows inherently unsafe constructs. For example, it allows cross-references that are basically just arbitrary unaligned pointers. It uses many different alignment and p…
when execution of user-supplied code is allowed (in the examples that you have outlined above), surely, the layer _executing_ the code cannot really do anything about it ! perhaps you actually did intend to `rm -rf /` ?
policy checking, enforcement etc. has to happen at a higher / different layer. i am not sure why mechanism and policy are being conflated here.
in the same way, you gave the serialization layer a 10mb or whatever sized input to serialize, sure...you get an valid serialized output etc. maybe there is a genuine usecase for that in some context or another f.e. when serializing say image files, or something else etc. etc.
[edit] : minor comment.
Earlier quoted context omitted.
right, and it should not be left to the serialization layer for that.
it depends on what type of safety. The schema language might for example allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer, same for array/list/vector length.
why ? are there no cases where serializing even larger file is valid ?