Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JSON with Commas and Comments

nigeltao.github.io

Post reply on HN
Print view View original

221–230 of 251 posts

Re: JSON with Commas and Comments

#221

Post by flohofwoe »

q3k wrote: post #195
flohofwoe wrote: post #184

Earlier quoted context omitted.

What programming languages and platforms will your SDK be compatible with, will you also be around to port the SDK to new platforms in twenty or thirty years? Those are good reasons to agree on a standardized data format instead of a specific implementation. Pretty much every language on the planet has JSON parser libraries, and if not, it's fairly trivial to write your own (or convert the JSON data to something else…

> Pretty much every language on the planet has JSON parser libraries. Yes, but they all implement the JSON RFC slightly differently, or implement it in a totally non-compliant way.

Then it's totally fine to pick another commonly used data exchange format which is better defined, that's still much better than hiding the data format behind an SDK.

However, we're coming from XML, which was better defined, and there are also binary "quasi standard" formats like protobuf, so those problems had already been solved.

Yet still "the world" has moved to JSON, which seems to indicate that those problems probably were not all that important for most use cases.

Re: JSON with Commas and Comments

#222

Post by flohofwoe »

jonathanaird wrote: post #139
nly wrote: post #129

Let's stop pretending JSON isn't popular because it lets us be loosey-goosey with our specifications. Writing specifications is hard and time consuming, and getting people to follow them is almost impossible. JSON is something we can (almost) all agree on for dumping loosely specified, human readable representations of data structures. It lets users and client application developers be lazy and not have to learn a ne…

I really want something that’s strongly typed but doesn’t require code generation like protobufs do. Yaml doesn’t do it for me. The closest I can get is putting the type guarantees in the database and using GraphQL.

There are protobuf libraries without code generation, for instance: https://github.com/cloudwu/pbc (you lose the connection to the language's type system though).

Re: JSON with Commas and Comments

#223

Post by junke »

cout wrote: post #176
nly wrote: post #129

Let's stop pretending JSON isn't popular because it lets us be loosey-goosey with our specifications. Writing specifications is hard and time consuming, and getting people to follow them is almost impossible. JSON is something we can (almost) all agree on for dumping loosely specified, human readable representations of data structures. It lets users and client application developers be lazy and not have to learn a ne…

I think the reason JSON caught on outside of the javascript community is because it provided a succinct way to represent tree structures of common types (numbers, strings, dictionaries, and lists), using a syntax familiar to anyone who has used C, Perl, Python, Ruby, or Java. S-expressions might have caught on for this purpose, but they lacked a killer app (jquery, and later rails). Like JSON, they are easy to parse…

> Like JSON, they are easy to parse and easy to generate, but being more loosely-specified than JSON, ...

They would not have been loosely-specified if they were specified, like JSON was :-) I mean this is taking things a bit backward. When tools use a data format based on S-exprs, they define more clearly what is or isn't valid (OCaml Dune, Guix, etc.)

Re: JSON with Commas and Comments

#224

Post by telekid »

q3k wrote: post #171
jobigoud wrote: post #152

Earlier quoted context omitted.

Turing complete config files makes me a bit nervous.

Not just turing complete, but impure. Configuration files shouldn't be able to perform arbitrary IO. Starlark to solve this exact problem, and is a Python subset so that people don't have to learn something totally new either. It's still turing complete, but at least guaranteed to not have any side effects, or even be able to access anything but an explicitly given execution context.

@borkdude’s sci actually seems like a nice contender for this as well.

Re: JSON with Commas and Comments

#225

Post by xpe »

pseudalopex wrote: post #51
xpe wrote: post #45

Earlier quoted context omitted.

JSON is extremely closely linked with JavaScript. Trying to move it in a different direction, in my opinion, would require a lot of effort, without much benefit, and is unlikely to succeed.

JSON is used lots of places JavaScript isn't. It hasn't been closely linked with JavaScript since people realized eval is dangerous. People are more interested in JSON derivatives like JSON5 or Ion than more complex languages like Dhall in my experience.

All fair points.

Dhall is new to me. Would you tell me more about other configuration languages you seen? I often use TOML myself.

Re: JSON with Commas and Comments

#226

Post by deepakarora3 »

secondcoming wrote: post #158
nly wrote: post #129

Let's stop pretending JSON isn't popular because it lets us be loosey-goosey with our specifications. Writing specifications is hard and time consuming, and getting people to follow them is almost impossible. JSON is something we can (almost) all agree on for dumping loosely specified, human readable representations of data structures. It lets users and client application developers be lazy and not have to learn a ne…

Json does have schemas

Yes it does but as someone else mentioned, it can be really challenging to work with. Its difficult to visualize the structure of the document. Its difficult to program against. This was one of the problems which I wanted to solve when I created unify-jdocs. Agreed that it may not do all the things that JSON schema does but it does all those which are required 90% of the time. Along with other things like not having to generate any model classes (code generation) and being able to read and write any path in a single line of code. And being able to merge JSON documents into another. Strong typing has its pros and cons. In my opinion, for complex JSON documents, going the strongly typed way makes making change difficult in the long run. Think of JSON documents the size of 3000 JSON paths going 10 levels deep etc. You can read more about this Java library at https://github.com/americanexpress/unify-jdocs.

Re: JSON with Commas and Comments

#227

Post by fanf2 »

kstenerud wrote: post #156
jonathanaird wrote: post #139

Earlier quoted context omitted.

I really want something that’s strongly typed but doesn’t require code generation like protobufs do. Yaml doesn’t do it for me. The closest I can get is putting the type guarantees in the database and using GraphQL.

I made https://concise-encoding.org/ to deal with this: - strongly typed - ad-hoc or schema (your choice) - no code generation step - edit in text, send in binary

Why would someone choose this rather than msgpack or CBOR or protobuf or any of the other existing things in that space?

Re: JSON with Commas and Comments

#228

Post by quietbritishjim »

ydant wrote: post #218
dak1 wrote: post #205

Earlier quoted context omitted.

That looks a bit like NDJSON, assuming that the space between the objects is actually a newline. There are libraries[1] that support parsing it and it's not too hard to do yourself, either. Some fairly popular projects use it to represent multiple responses in a single response body[2]. [1] http://ndjson.org/libraries.html [2] ElasticSearch uses it for msearch response bodys, for example.

Or JSON lines - whose website looks very similar to NDJSON's: https://jsonlines.org/ This format works great when using Amazon Athena (Presto) against log files written with one JSON object per line.

Both the websites are fancy ways of saying "one JSON object per line". The idea of laying out data that way was in common use before either of the website, they're just recording the practice.

Re: JSON with Commas and Comments

#229

Post by kstenerud »

fanf2 wrote: post #227
kstenerud wrote: post #156

Earlier quoted context omitted.

I made https://concise-encoding.org/ to deal with this: - strongly typed - ad-hoc or schema (your choice) - no code generation step - edit in text, send in binary

Why would someone choose this rather than msgpack or CBOR or protobuf or any of the other existing things in that space?

Because there isn't anything else in this space that:

- supports ad-hoc data structures or schemas per your preference

- supports all common types natively (doesn't require special string encoding like base64 or such nonsense)

- supports comments, metadata, references (for recursive/cyclical data), custom types

- doesn't require an extra compilation step or special definition files

- Has parallel binary and textual forms so that you're not wasting CPU and bandwidth serializing/deserializing text. Everything stays in binary except in the rare cases where humans want to look or edit.

Re: JSON with Commas and Comments

#230

Post by crazygringo »

laurent92 wrote: post #206
wruza wrote: post #172

Earlier quoted context omitted.

But why should re-jsoning keep //-comments? I never met the need for that personally. Comments help initial filename.json to be parsed by human eyes, and when it is sucked into a program (and possibly transformed), they lose meaning, even when pretty printed. For them to stay, one can put comments into real keys and/or structure their data appropriately. Even if one ultimately needs to keep json document’s human stru…

> they may use more structure-aware library to do that Sure. I’m particularly thinking about mvn upgrades or “npm update”, which modify pom.xml/package.json files to upgrade the libraries, after checking rules (non-breaking changes or not, vulnerabilities or latest, etc). For mvn, libraries never succeeded to modify the pom in-place without wrecking the file format, so mvn upgrades never became a thing. It’s also a d…

That sounds like an entirely separate issue from what JWCC tries to achieve though.

For programatically upgrading JSON config files, it's not just comments but whitespace, indentation, etc. that need to be preserved. Honestly that needs an entirely separate tool/library from normal JSON encoding and decoding -- e.g. special jwcc_insert() and jwcc_delete() functions that guarantee the file remains untouched (exact formatting preserved) except for the modified part.

It's really more akin to when your apache.conf file gets modified programatically -- everything is preserved exactly except for the specific lines that get touched/added.

Post reply on HN