New better alterative to XML, JSON and YAML

> I believe there is no other feasible encoding than UTF-8 for textual formats now.

I disagree, although I do not believe that this disagreement should have to affect some file formats, since some file formats should not need to care about the character encoding, except perhaps specific details, e.g. that ASCII bytes have ASCII meaning and non-ASCII bytes have non-ASCII meaning (which is true of UTF-8 and of some others).

> IEEE 754 doesn't define any textual format while it does have binary decimal formats. The correct wording should have been that numeric scalars follow a specific grammar to be interpreted as an IEEE 754 binary64 number in the data model.

OK. Although it seems clearly enough, probably the document should specify explicitly its working anyways, like you mention.

> There are several formats that do try to support native graph types, including YAML and Concise [1]. So that is hardly new.

I had seen Concise as well, and I have criticisms of that too. I did forget it though as I was writing my message.

I did have a idea to add a reference type too (you can see my other comment about TER and ASN.1X), although I would have it to not have the same meaning as the data it references (therefore meaning that it cannot result in cyclic graphs, even if it references itself). I think I should not have remote references, though. Concise encoding seems to have it have the same meaning as the data it references, and I think it is useful to not do this (you could use compression if you want to deal with many of duplicated data).

> The flexibility in name is to support the plethora of programming languages.

As far as aware, there is no language that allows a tab in its identifier. I'm aware of some (uncommon) languages that allow a space in identifiers though.

After all, the goal of language is firstly to set what's valid or not and secondly to give a meaning to valid one. The valid set should be as maximally different from the invalid set as possible, but allowing virtually invalid characters in names is against this goal. Consider the approach taken by XML 1.1 (not 1.0) if you need an idea.

> Commas are for readability as per English.

A lot of English-speaking countries actually have `.` and `,` swapped: 3,141.592 vs. 3.141,592 for example. Due to this ambiguity, a comma as a grouping separator is heavily discouraged. You can instead use a underscore `_` (very common in programming languages) or a space (more preferred in human texts) without such concerns.

> Three digits are recommended for the encoder.

Also, some English-speaking countries use different group sizes (notably India). I'm personally fine with three-digit groupings despite of that, but that should be clearly specified at the very least.

> Textual data is decoded into binary information in software so setting expectations, e.g. supporting ∞, facilitates interoperability by reducing impedance mismatches.

This procedure should have been made explicit. In fact, the single worst thing you can do in serialization formats is an unclear definition of data model.

> If the implementation uses ɪᴇᴇᴇ doubles the rounding shall be as expected.

There is nothing like "as expected" in IEEE 754. The current rounding mode is a part of the execution state, so leaving it unspecified risks a non-deterministic interpretation even in the same execution. Either you should specify some rounding mode (most likely round-to-even), or you should state that encoders should pick a long enough decimal representation to avoid any such issue.

> We are following the Base64 standard exactly.

Which base64 standard in [1]? The padding in base64 is vestigial anyway and serves no practical need by now, so there is no strong reason to use a particular standard with the required padding. It is much more important to decide what to do with incorrect padded bits.

[1] https://en.wikipedia.org/wiki/Base64#Variants_summary_table

> It is essential for data to be structured as a graph, it simply occurs. Serialization on most data formats has the required support for graphs awkwardly layered on top of the encoding.

I don't question that graph structures often occur naturally and existing schemes are often awkward, but I think that's more of the lack of co-developed standards. I have outlined my rationale for layering in other comments.

> The C# implementation defines the data model; essentially objects, array and scalars with multiple parents for nodes. The Formats section is to, as stated, facilitate interoperability.

The data model should be abstract enough to be truly interoperable. JSON suffered a lot from having no defined data model to this day, even when there was a soft-of-reference implementation by Crockford. It is not too hard to define a data model in prose rather than code.

> Limiting whitespace to spaces and tabs does speed processing. The ᴜᴛꜰ-8 bytes can be left in the input buffer rather than copied. All significant bytes are ᴀꜱᴄɪɪ.

May have been true in the past, but it's no longer true since SIMD-based parsers. Also the very existence of escape sequence prevents the true non-destructive zero-copy (aka in-situ) parsing anyway. With such sequences, zero-copy/in-situ parsing has to be destructive to be performant and that can preclude some use cases. Allowing additional space characters is much easier than that.

> There are no unmatched angle brackets (that was a typo).

I meant to refer to `...`. Multiple grouping characters can allow for simpler syntaxes.

> I disagree, although I do not believe that this disagreement should have to affect some file formats, [...]

Ah, if you just want to make the UTF-8 requirement less strict for simpler decoders, that can be actually okay. My belief is more about legacy encodings like Shift_JIS.

I never seriously tried to fit any graph struture into a serialization format, so I don't yet have a very concrete opinion besides from what was already said. (I'm not even sure Concise had a good approach for that...)

Sorry but I would never use this format for both manual or programmatic approach.

* I've tried to read the data this format describes without reading its documentation and I just failed: the format is amazingly counter-intuitive. I never had a readability and understanding issues with XML/HTML, JSON or even YAML (that I think is overly complicated) when I saw them for the first time.

* Terse does not mean cryptic. Basic notation is just weird: why would it need unbalanced the less-than symbol to open the array? Why `` for delimiting elements? Why `` but not `>` at least just to be more readable by human and look balanced? The syntax goes more weird for arrays containing objects: indents (okay to some extent), `` and `` (`{` and `}`?).

* Auto-removing whitespaces may hurt. If the format offers this, would it also offer a heredoc-style text like `cat * Native support for arrays. I mentioned a few above. `` and `` -- guess what these two mean if you see this first time? You would never guess. It's an empty array and an empty element, you've just failed.

* Graphs... Another weird syntax comes into the room: `#id;` but `@id` (no semicolon?). Okay, these seem to be first-class ids and refs, not necessarily designed for graphs (I'm not sure if the `#ID;` and `@` would play perfect with any non-empty names.) But what does graphs make first-class citizens here and why? Graphs can be expressed, I believe, in any data/markup format/language and then processed with a particular application if graphs are needed. By the way, arrays and objects are not necessarily trees from the semantic point of view. More graph processing issues were mentioned in other comments to this topic. What about the first-class support for sets? I'm kidding

* Comments. Another symbol here to come: `%`. To be honest, I can't recall any instance I could see the percent sign elsewhere for this purpose. What if the comments would start with a well-known `#` at least with a space right after it so that it wouldn't be considered a "graph id" (or, don't get me wrong, with another `* Just got to the Escaping section and now I see how the characters are escaped. Perhaps this is okay.

* Scalars. Crazy number formatting and locale issues are waiting. The never-on-keyboard infinity symbol would be great for APL, but why not just Inf(inity)? Whatever the scalar value is, no need to cover all existing primitive scalars -- just let them be processed by an application since all scalars are text semantically. Another crazy things: what does make UUIDs that special for this format?; why does make Base64 that special so that it has native support (would it support Base16 for human-readable message digests; or Base58 to remove visually lookalike Base64 characters)?

* CR/LF? I can understand its semantic purpose, but why not LF to make it even more "blazingly" fast? Say good-bye to UNIX users.

* The cognitive load for the markup syntax absolutely does not make it efficient in typing. Believe me, it does not.

What I would do, I would probably enhance the widely used formats, say make JSON, which I find almost perfect from the syntax point of view, not require quotes for object property names if the names would not contain special characters like `:` just like it goes in JavaScript. And perhaps make XML "v2" move away from SGML hence loosening its syntax to get rid of the closing tags with shorter notation, first-class array support and fixing syntax issues especially for CDATA and comments that can't support `--`. You would blame me, but I love XML the most: it just has the richest set of standardized amazing well-designed extensions to operate XML with regardless the heavy XML syntax.

P.S. How does it look like in the document it marks up is minified (e.g., no whitespaces)?

> Which?

Oh, I just realized that I carelessly put "English-speaking" there. (AFAIK the exact reversal does exist in English, but is much rarer and not entirely domestic.) But that doesn't really justify the use of comma in numbers.

> .net for example has round trip encoding to achieve this.

That was never guaranteed to my knowledge, and there also seems an apparent difference in .NET Framework and .NET Core/Runtime versions according to some searches. Many enough standardized languages have no strong requirement as well (for example, see [1] for ECMAScript), so this should be clearly specified to be truly portable.

[1] https://tc39.es/ecma262/multipage/abstract-operations.html#s...

> As linked in the document, we refer to ʀꜰᴄ 4648 for Base64 using the simplest version [...].

You are free to rephrase any variant of base64 standard into simple statements. (Ideally the standard itself should be also cited, though.) I was asking why that particular variant was used.

> Layering graphs on top of the encoding in a separate standard is not practical. As stated serialization requires a graph structure.

The separateness here is not binary. The Unicode standard (ISO/IEC 10646) for example has multiple standard annexes [2] that are synchronized with the core standard but published and developed separately. For all practical purposes they form a single unified standard, but you can (and almost likely should) ignore some irrelevant annexes. I'm arguing for a similar structure to be clear, do you think that's also a no-go?

[2] https://www.unicode.org/reports/

> I have done that.

The only explicit thing in your data model is the following definition:

    // Read as: "document" is either "object", "array" or "scalar".
    document = object | array | scalar
    // Read as: "object" is a distinct type "Object" with "name" and zero or more "field"s. (And so on)
    object = Object(name, field*)
    array = Array(name, item*)
    scalar = Scalar(name, value)

This is not what's implied by following sections. To begin with, you have no explicit definition of `item` or `field`. Yes, it is kinda obvious that both should be `document` and also can be somehow inferred that `scalar` in the `item` place should be a nameless object, but not only such relations are not explicit but now we have another class of objects not mentioned in the first place! The exact nature of `value` is also very unclear.

In my understanding, your data model is more like this:

    document = object | array | scalar

    // `...?` for zero or one copy of the preceding data.
    optional-name = Name(name)?
    optional-ref-id = RefId(ref-id)?
    optional-type = Type(type)?

    name = string             // Can't be empty
    ref-id = string           // Globally unique
    type = string             // Interpretation up to clients

    object = Object(optional-name, optional-ref-id, optional-type, field*)
    field = ObjectField(name, optional-ref-id, optional-type, field*)
          | scalar

    array = Array(optional-name, item*)
    item = object
         | ScalarItem(optional-ref-id, field*)  // Implicitly converted to object
         | ReferenceItem(reference-id)

    scalar = Scalar(name, value)
           | Reference(name, ref-id)
    value = string    // Additional interpretation rules may exist after parsing

For example, the following document:

...is thought to parsed into the following underlying data:

    Object(Name("Person"),
        Scalar(Name("Name"), "Bonnie"),
        ObjectField(Name("Spouse"), RefId("jack-smith"),
            Scalar(Name("Name"), "Jack")
        ),
        Reference(Name("Doctor"), "jack-smith")
    )

This abstraction clearly demonstrates many important things coming up in implementations. For example, every `field` has to provide `name` because all possible branches (`ObjectField`, `Scalar` or `Reference`) have one---at least in my understanding. I shouldn't be figuring out such data model myself to be honest; it's your job to provide one.