Earlier quoted context omitted.
An alternative approach might be to use an existing popular serialization format such as Protocol Buffers, Apache Thrift, or Cap'N'Proto and create or improve tools that convert to/from human-readable text formats to the serialized binary format. For example: - Protocol buffers have a text format mode: https://medium.com/@nathantnorth/protocol-buffers-text-forma... - Thrift has readable-thrift which is a human-friend…
Yes, I had a look at these formats before embarking on my venture. I listed the things I found important in the comparison matrix: https://github.com/kstenerud/concise-encoding#-compared-to-o... To your points: - Protobufs is not an ad-hoc format, which is a big reason why low-friction formats like JSON are popular. There are many use cases where formats like protobufs are clearly the superior choice, but CE doesn't…
This looks like a very ambitious project, and I can see that you've put a lot of thought, time, and effort into it! You clearly have a lot of interesting ideas (the graph idea is really cool) and significant experience with data formats.
If this is a security-oriented application, then with cyclic data structures there is the risk of blowing out your server's memory using something like a fork bomb when processing untrusted user input (https://en.wikipedia.org/wiki/Fork_bomb).
There are some systems like DHall that guarantee termination by putting upper bounds on computation: https://dhall-lang.org/
I'm also a bit concerned with how the different features can interact, for example it's not super clear how to distinguish between UTC offset (-130, or do these always have to be 4 digits?) and global coordinates (-130/-172). An attacker could specify a comment inside the media type (eg: application/* which would require special logic to filter out).
My concern is that the parser will become extremely complicated and require a lot of special-case logic and validation (eg: there must be at least one digit on each side of a radix point) which is more prone to errors and unexpected behaviors.
Rather than using slash delimiters, I'd recommend splitting the time formats into subfields, eg: { date: "2022-01-01" time: "21:14:10" offset_is_negative: true offset: "10:30" }
This does make the text format more verbose, but it reduces ambiguity and makes the parsing faster as well since you don't need to descend into branches and backtrack when they don't match, and also might permit more code/logic reuse.
It's also not clear how easy it is to add new data types to the grammar. Based on the project description, it seems like you're using ANTLDR parser.
Since you seem to be quite interested in parsing, you might also be interested in parser combinators which are a somewhat different approach with different tradeoffs: - https://softwareengineering.stackexchange.com/questions/3386... - https://fsharpforfunandprofit.com/posts/understanding-parser...