Live data from Hacker News

JSON with Commas and Comments

nigeltao.github.io

171–180 of 251 posts

Re: JSON with Commas and Comments

#171
post #124

I've been finding plain JavaScript as a very nice alternative to the many config file formats out there. It already has the trailing commas + comments bells and whistles, plus the ability to do computation to generate repetitive elements. Of course there's always the danger of the user creating a monstrosity of a 'config file' but if they're the ones using the software that consumes the config file, that's on them to…

Turing complete config files makes me a bit nervous.

Not just turing complete, but impure. Configuration files shouldn't be able to perform arbitrary IO.

Starlark to solve this exact problem, and is a Python subset so that people don't have to learn something totally new either. It's still turing complete, but at least guaranteed to not have any side effects, or even be able to access anything but an explicitly given execution context.

Re: JSON with Commas and Comments

#172

What's nice is that this can essentially be ignored by API's and browsers. Computer-generated data never needs to include trailing commas or comments. Really the principal use case here is JSON as human-edited configuration files. In which case I suppose it's nice to have a name like "JWCC", but really it's just two flags for JSON decoding libraries to add (or a single nice combined flag). So that really would be gre…

It doesn’t improve the dev experience if the compilers don’t produce it, and this is really about dev comfort. You need to be able to modify JSON programatically and keep the comments, if the output is pretty-printed. Also, NaN numbers are an entire features, which they talk about above.

But why should re-jsoning keep //-comments? I never met the need for that personally. Comments help initial filename.json to be parsed by human eyes, and when it is sucked into a program (and possibly transformed), they lose meaning, even when pretty printed. For them to stay, one can put comments into real keys and/or structure their data appropriately.

Even if one ultimately needs to keep json document’s human structure (like key order, comments, whitespace nuances), they may create and use more syntax structure-aware library to do that.

Re: JSON with Commas and Comments

#173

> Yes, Doug Crockford deliberately removed comments from JSON but people keep putting them back in. If we’re going to have comment-enriched JSON (e.g. for human-editable configuration files), we might as well have a standard one. The text "deliberately removed comments from JSON" links to https://web.archive.org/web/20150105080225if_/https://plus.g... where Doug not only explains the reason, but also a solution which…

If the solution is to store config in a format with comments and strip them off before handing to JSON, what should that format be? Shouldn't it be formalized? Couldn't JWCC be that format?

Re: JSON with Commas and Comments

#174

Earlier quoted context omitted.

ARPANET, the direct predecessor for the internet, was created for collaborating, including socializing and reading. It was NOT created to "have a war-proof network for army use". The latter is a pernicious falsehood. Quoting https://en.wikipedia.org/wiki/ARPANET : > It was from the RAND study that the false rumor started, claiming that the ARPANET was somehow related to building a network resistant to nuclear war. Th…

From the very page you link to: Nonetheless, according to Stephen J. Lukasik, who as Deputy Director and Director of DARPA (1967–1974) was "the person who signed most of the checks for Arpanet's development: "The goal was to exploit new computer technologies to meet the needs of military command and control against nuclear threats, achieve survivable control of US nuclear forces, and improve military tactical and man…

That quote was immediately after a quote from Charles Herzfeld, ARPA Director:

> "The ARPANET was not started to create a Command and Control System that would survive a nuclear attack, as many now claim. To build such a system was, clearly, a major military need, but it was not ARPA's mission to do this; in fact, we would have been severely criticized had we tried."

History's complicated, isn't it?

So, we look at other things: Licklider became Program Director at ARPA in 1962 and ARPANET started in 1966 (which is when Lukasik joined as Director of Nuclear Test Detection before becoming A.D. the next year, then Director in 1971). And as Lukasik writes in his paper, the late 1960s were a different funding era than the early 1960s when Herzfeld's "foundling" started.

Recall that the 1968 Mansfield Amendment prohibited military funding of research that lacked "a direct or apparent relationship to specific military function" - far different than the Ruina years where office directors and program managers had significant autonomy and funding authority. An effective ARPA director after the Mansfield Amendment was passed is going to be someone who is good at viewing ARPA projects through that military support lens, yes? Which might be different than the lens used earlier?

Next, quoting https://en.wikipedia.org/wiki/Robert_Taylor_(computer_scient... :

> Taylor hoped to build a computer network to connect the ARPA-sponsored projects together, if nothing else, to let him communicate to all of them through one terminal. By June 1966, Taylor had been named director of IPTO; in this capacity, he shepherded the ARPANET project until 1969.[11] Taylor had convinced ARPA director Charles M. Herzfeld to fund a network project earlier in February 1966, and Herzfeld transferred a million dollars from a ballistic missile defense program to Taylor's budget.

It therefore seems very much like collaboration, at the very least, was indeed part of ARPANET's goals when it started in 1966.

FWIW, as Martin Campbell-Kelly and Daniel D Garcia-Swartz point out, at https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58...

> ARPANET network was one among a myriad of (commercial and non-commercial) networks that developed over that period of time – the integration of these networks into an internet was likely to happen, whether ARPANET existed or not.

They further consider it "Whig history" to think ARPANET plays a critical role in the modern day internet.

And I agree with that assessment.

Re: JSON with Commas and Comments

#175
post #129

Let's stop pretending JSON isn't popular because it lets us be loosey-goosey with our specifications. Writing specifications is hard and time consuming, and getting people to follow them is almost impossible. JSON is something we can (almost) all agree on for dumping loosely specified, human readable representations of data structures. It lets users and client application developers be lazy and not have to learn a ne…

Json does have schemas

The JSON schema ecosystem is a mess. json-schema.org is one thing, openapi is another (it uses an 'extended subset' of json-schema.org, which is another way of saying 'incompatible'), there’s also typeschema, and a bunch of other pet projects.

There is no official JSON schema RFC, there is no official linking between json-schema.org and the JSON RFC. There is no push to make every language that supports JSON to also support a JSON schema system.

Finally, there isn't even a guarantee that all human-readable JSON document specifications can be expressed within a JSON schema in any sensible way.

Re: JSON with Commas and Comments

#176
post #129

Let's stop pretending JSON isn't popular because it lets us be loosey-goosey with our specifications. Writing specifications is hard and time consuming, and getting people to follow them is almost impossible. JSON is something we can (almost) all agree on for dumping loosely specified, human readable representations of data structures. It lets users and client application developers be lazy and not have to learn a ne…

I think the reason JSON caught on outside of the javascript community is because it provided a succinct way to represent tree structures of common types (numbers, strings, dictionaries, and lists), using a syntax familiar to anyone who has used C, Perl, Python, Ruby, or Java.

S-expressions might have caught on for this purpose, but they lacked a killer app (jquery, and later rails). Like JSON, they are easy to parse and easy to generate, but being more loosely-specified than JSON, it's less clear how to map a given S-expression to a native type. As a glue format, JSON feels just right, even if it's a bit picky sometimes (e.g. trailing commas).

For anyone who wants extra features like comments, YAML is the oldest popular format I am aware of that is a superset of JSON. For any new format to succeed, IMO it needs to sufficiently distinguish itself from both YAML and JSON, and not just support a feature set that happens to lie somewhere between the two.

Re: JSON with Commas and Comments

#177

> Yes, Doug Crockford deliberately removed comments from JSON but people keep putting them back in. If we’re going to have comment-enriched JSON (e.g. for human-editable configuration files), we might as well have a standard one. The text "deliberately removed comments from JSON" links to https://web.archive.org/web/20150105080225if_/https://plus.g... where Doug not only explains the reason, but also a solution which…

> A new format is a really weird way to go about fixing a problem that already has a solution. Stripping comments before handing off to a JSON parser is not a full solution. For example, if the file has to be augmented with additional data and written out again, the comments would be lost.

Formatting would be lost too (multiple newlines, array formatting, wrap limits, etc), and the order of keys. There are two separate tasks: comment json for a human and ignore that on parse(), and maintain pretty-json with all its usefulness programmatically with parseWithSyntaxStructure(). We shouldn’t mix them into one single problem, because in one case you just want multiple readers (a human and a computer) and in the other you want to rich-edit a human representation on its own.

Re: JSON with Commas and Comments

#178
post #129

Let's stop pretending JSON isn't popular because it lets us be loosey-goosey with our specifications. Writing specifications is hard and time consuming, and getting people to follow them is almost impossible. JSON is something we can (almost) all agree on for dumping loosely specified, human readable representations of data structures. It lets users and client application developers be lazy and not have to learn a ne…

Json does have schemas

Sure, they are just not that widely used. As in, I've not really encountered much json with schema in the wild in my career. I know it exists (in several forms). Support for schemas in mainstream parsers is pretty much non existent making their use very optional as you pretty much have to jump through hoops to do anything with them. Most people seem to opt not to do that.

I never got anything out of xml validation either. It just lead to this ridiculously verbose garbage xml that was even less readable with all the namespace, namespace declarations, etc. Very tedious to write manually as well. Also lots of weirdness doing e.g. xpath or xsl transformations against that. The whole SOAP / web services bubble eventually imploded when people figured that you could send tiny json objects instead via REST.

I have many issues. People sending invalid json to my APIs is not one of them. You get a bad request if you do. End of story. Don't don that if you don't like bad requests. It's not a problem that justifies a lot of over engineering.

Json, yaml, toml, hocon, etc. are basically all just variants of attempting to send human editable blobs of information. Fine for apis where all of the requests are created by programs. Unfortunately people also abuse them for things like DSLs that are authored by humans. The real problem there is not using a strongly and statically typed language that simply does not allow illegal things. A schema is just a stop gap solution when you don't have that.

Kotlin is actually great for creating proper DSLs. You get IDE auto-completion and red squiggly lines when you do it wrong. I think Rust also has some nice syntactical constructs for creating DSLs. Typescript might also emerge as a language that is very suitable for that (with maybe a few more features borrowed from Kotlin). Using a non compiled language with weak typing kind of defeats the purpose. Hence the endless ruby but not quite ruby like DSLs for things like puppet.

Re: JSON with Commas and Comments

#179

Earlier quoted context omitted.

> I don't think there is a native immmutable array in javascript. I'm here to tell you that `Object.freeze()` works just fine on arrays.

Nice. Though I don't think that solves the rest of the issues I outlined sadly. I guess you could have this json+ convert into some custom javascript class that allows arrays/objects as map keys rather than the normal javascript object that only accepts strings as map keys.

> I guess you could have this json+ convert into some custom javascript class that allows arrays/objects as map keys rather than the normal javascript object that only accepts strings as map keys.

Sure - why not? JSON doesn't have to map directly to JavaScript objects.

Re: JSON with Commas and Comments

#180

Earlier quoted context omitted.

it's true JS doesn't support this, and it's also true that any JSON that supported it would therefore no longer be _JavaScript_ object notation. However, it's equally true that a better language and object notation would support non-string scalar keys.

JavaScript wouldn't be able to consume those resulting objects, so it would be pretty diminished as an alternative to JSON. Object keys are all strings.

Not that I find this idea useful for otherwise limited DTO format, which JSON is, but JS has Map, which may be used for such maps.
Post reply on HN