Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FracturedJson

github.com

Post reply on HN
Print view View original

161–170 of 173 posts

Re: FracturedJson

#161

Post by damnitbuilds »

Xymist wrote: post #6
damnitbuilds wrote: post #2

Nice. And BTW, thanks for supporting comments - the reason given for keeping comments out of standard Json is silly ( "they would be used for parsing directives" ).

It's a pretty sensible policy, really. Corollary to Hyrum's Law - do not permit your API to have any behaviours, useful or otherwise, which someone might depend on but which aren't part of your design goals. For programmers in particular, who are sodding munchkins and cannot be trusted not to do something clever but unintended just because it solves a problem for them, that means aggressively hamstringing everything.…

"do not permit your API to have any behaviours, useful or otherwise, which someone might depend on but which aren't part of your design goals"

I can not follow this law by making my API depend, say, the contents of a string value. Preventing APIs depending on the value of a comment is no different, so your argument is not a reason for not having comments.

Re: FracturedJson

#162

Post by Y-bar »

minitech wrote: post #148
Y-bar wrote: post #27

Earlier quoted context omitted.

We stopped having this problem over ten years ago when spec 1.1 was implemented. Why are people still harking on about it?

Current PyYAML: >>> import yaml >>> yaml.safe_load("country: NO") {'country': False} Other people did not stop having this problem. It might be that there’s some setting that fixes this or some better library that everyone should be switching to, but YAML has nothing that I want and has been a repeated source of footguns, so I haven’t found it worth looking into. (I am vaguely aware that different tools do configure…

The ancient rule of ”use software that is updated with bugfixes” certainly applies here.

Re: FracturedJson

#163

Post by Y-bar »

merelysounds wrote: post #112
Y-bar wrote: post #101

Earlier quoted context omitted.

The 1.1 spec was released about _twenty_ years ago, I explicitly used the word _implemented_ for a reason. As in: Our Yaml lib vendor had begun officially supporting that version more than ten years ago.

Note that you reference 1.1, I think that version still had the norway behavior.

1.1 partially fixed it, so that strings (quoted ”no”) did not become Boolean false. 1.2 strengthened it to remove unquoted no from list of tokens which could be interpreted as Boolean false.

Re: FracturedJson

#164

Post by zzo38computer »

kstenerud wrote: post #155
zzo38computer wrote: post #119

Earlier quoted context omitted.

I think JSON is too limited and has some problems, so BONJSON has mostly the same problems. There are many other formats as well, some of which add additional types beyond JSON and some don't. Also, a few programs may expect (and possibly require) that files may contain invalid UTF-8, even though it is not proper JSON (I think it would be better that they should not use JSON, due to this and other issues), so there i…

Yup, and that's perfectly valid. I'm OK with BONJSON not fitting everyone's use case. For me, safety is by far more important than edge cases for systems that require bad data representations. Anyone who needs unsafe things can just stick with JSON (or fix the underlying problems that led to these requirements). Safe, sane defaults, and some configurability for people who (hopefully) know what they're doing. Falling…

> Safe, sane defaults, and some configurability for people who (hopefully) know what they're doing.

Yes, I agree (if you want to use it at all, which as I have mentioned you should consider if you should not use JSON or something related), although some of the things that you specify as not having options will make it more restrictive than JSON will be, even if those restrictions might be reasonable by default. One of these is mismatched surrogates (although matched surrogates should always be disallowed, an option to allow mismatched surrogates should be permitted (but not required)). Also, I think checking for duplicate names probably should not use normalized Unicode. Furthermore, the part that says that names MUST NOT be null seems redundant to me, since it already says that names MUST be strings (for compatibility with JSON) and null is not a string.

> Mandating safety and consistency within the spec is a MAJOR help towards raising the safety of all implementations and avoiding these security vulnerabilities in your infrastructure.

OK, this is a valid point, although there is still the possibility of incorrect implementations (adding test cases would help with that problem, though).

Re: FracturedJson

#165

Post by zzo38computer »

merelysounds wrote: post #19
silvestrov wrote: post #13

Earlier quoted context omitted.

Just say Norway to YAML.

This is a reference to YAML parsing the two letter ISO country code for Norway: country: no As equivalent to a boolean falsy value: country: false It is a relatively common source of problems. One solution is to escape the value: country: “no” More context: https://www.bram.us/2022/01/11/yaml-the-norway-problem/

I think it would be better to require quotation marks around all string values, in order to avoid this kind of problems. (It is not the only problem with YAML, but it is my opinion of how any format with multiple types should require explicitly mentioning if it is a string type, but YAML (and some other formats) doesn't.) (If keys are required to strings, then it can be reasonable to allow keys to be unquoted if the set of characters that unquoted keys can contain is restricted (and disallowing unquoted empty strings as keys).)

Re: FracturedJson

#166

Post by zzo38computer »

miguelbemartin wrote: post #70

Is JSON a format that needs improvement for human readability? I think there are much better ways to present data to users, and JSON is a format that should be used to transfer data from system to system.

I think JSON is not really so good either way, due to problems with the data and with the file format.

Re: FracturedJson

#167

Post by wizzwizz4 »

boxed wrote: post #151
wizzwizz4 wrote: post #73

Earlier quoted context omitted.

You can guarantee that all the cases in the code are tested. That doesn't necessarily mean that all the behaviour is tested. If two implementations use very different approaches, which happen to have different behaviour on the Mersenne primes (for deep mathematical reasons), but one of them special-cases byte values using a lookup table generated from the other, you wouldn't expect mutation testing to catch the discr…

> but one of them special-cases byte values using a lookup table generated from the other, you wouldn't expect mutation testing to catch the discrepancy Sure you would. If the mutation tester mutates that lookup table. Which is quite easy to do, and which mutmut will do (if that lookup table is inside a function, because mutmut is based on mutant schemata).

If the mutation tester mutates that lookup table, then that will eventually lead to all entries in the lookup table being tested. That does not mean that the four divergent values outside the lookup table will end up being tested.

Re: FracturedJson

#168

Post by zzo38computer »

kayhantolga wrote: post #38

These JSON files are actually readable, congrats. I’m wondering whether this could be handled via an additional attached file instead. For example, I could have mycomplexdata.json and an accompanying mycomplexdata.jsonfranc. When the file is opened in the IDE, the IDE would merge the two automatically. That way, the original JSON file stays clean and isn’t polluted with extra data.

> That way, the original JSON file stays clean and isn’t polluted with extra data.

FracturedJson does not add any extra data; it only changes the formatting (it is a way of automatically formatting JSON data, not a new file format). However, the documentation mentions that in some cases it does reorder or rewrite things (such as the order of keys, the number of decimal places, etc).

If you set CommentPolicy=TreatAsError then programs that convert it to a canonical form (whether or not that canonical form is JSON or some binary format intended to be like JSON) should (hopefully) result in the same output with the original and the one converted by FracturedJson, depending on what things are considered to be significant. (I tested this with a program I wrote, which converts JSON to DER (which is a canonical form (and, in my opinion, usually the only good one) of ASN.1), and does not consider the order of keys or the representation of numbers to be significant (although the conversion of numbers does not lose any precision and is converted exactly, but e.g. "1.2" is considered the same as "1.200" and "80e1" is considered the same as "800").)

Re: FracturedJson

#169

Post by merelysounds »

Y-bar wrote: post #163
merelysounds wrote: post #112

Earlier quoted context omitted.

Note that you reference 1.1, I think that version still had the norway behavior.

1.1 partially fixed it, so that strings (quoted ”no”) did not become Boolean false. 1.2 strengthened it to remove unquoted no from list of tokens which could be interpreted as Boolean false.

> 1.1 partially fixed it, so that strings (quoted ”no”) did not become Boolean false.

Do you have a source? Afaik v1.1 didn’t introduce such a change, v1.0 specified the same behavior for quoted strings, i.e. in v1.0 a quoted “no” would remain a string “no” as well.

Post reply on HN