JSON Patch – a format for describing changes to a JSON document • Hacker News Board

Re: JSON Patch – a format for describing changes to a JSON document

#51

Post by nacs » Thu, Oct 30, 2014, 4:22 AM UTC

chacham15 wrote: post #45

I think that this idea is missing one key point of patching: the patch should be smaller than the size of the new document, or at the least the sum of both the new and old documents (in case you want the patch for record keeping). In the example given the original text is 34 bytes long, the patch is 151 bytes, and the result is 42 bytes. Storing both the new text and the old text is 76 bytes which is about half of th…

I use this patch system in production and I agree. For small JSON documents, this system actually can result in larger patch files than the original document.

Instead of using this everywhere we output JSON however, we only use this patch system on large documents. It cuts our average transfer size from ~60KB (full JSON data) to ~2KB (patch data) per API request after the initial load since only a handful of values change over the polling interval.

Re: JSON Patch – a format for describing changes to a JSON document

#52

Post by nacs » Thu, Oct 30, 2014, 4:29 AM UTC

function_seven wrote: post #30

DannyBee wrote: post #13

Earlier quoted context omitted.

1. Errr, to apply any delta, including these, you need the document on hand. If your goal is to store deltas, this is no better than anything else. 2. This is in no way better at handling concurrent changes than anything other diff format. In fact, the extra operations it has do nothing to help you here (it is as good and bad as copy/add, or insert/delete diffs). I'd love to know why you seem to think otherwise. JSON…

> Errr, to apply any delta, including these, you need the document on hand. If your goal is to store deltas, this is no better than anything else. Doesn't this allow the sender to send updates without knowledge of the existing values, though? With a string diff, the existing value would have to be known. > This is in no way better at handling concurrent changes than anything other diff format. In fact, the extra oper…

> Doesn't this allow the sender to send updates without knowledge of the existing values, though?

This system requires knowledge of existing values/nodes to create the diff/patch document.

However for JSON data, it does have the advantage of not clobbering data unlike a string diff as you mentioned.

Re: JSON Patch – a format for describing changes to a JSON document

#53

Post by MBlume » Thu, Oct 30, 2014, 4:29 AM UTC

This is cool! My only substantive comment is criticism, but this is cool.

I think it's a problem that remove and replace don't indicate what they're removing or replacing, the same way .patch files indicate what lines they're removing. This makes patch files reversible, which JSON patches appear not to be. It also provides some redundancy/error correction, which is handy.

Re: JSON Patch – a format for describing changes to a JSON document

#54

Post by logn » Thu, Oct 30, 2014, 4:31 AM UTC

drderidder wrote: post #49

Dashron wrote: post #40

Earlier quoted context omitted.

This proposal is more in line with the system you describe (except null instead of undefined). https://tools.ietf.org/html/rfc7386 I prefer it, and think it's much easier to understand.

Oh, good - someone wrote an RFC for it - thanks for the link! It looks a lot like a post on partial updates in REST [1] where I recommended using null in the context of relational databases like mysql. Now I kinda feel undefined might be a better way to explicitly delete properties, especially if using a NoSQL database. 1. http://51elliot.blogspot.ca/2014/05/rest-api-best-practices-...

I think they used null for deletion because undefined isn't in the JSON spec. It's more of a Javascript standard.

Re: JSON Patch – a format for describing changes to a JSON document

#55

Post by icholy » Thu, Oct 30, 2014, 4:35 AM UTC

I was experimenting with the idea of adding 'slices'. They let you do map like transformations on arrays. https://github.com/icholy/JsonPatch.js

    /foo/1:3/bar

translates to

    foo.slice(1, 3).map(function (obj) { return obj.bar; });

Re: JSON Patch – a format for describing changes to a JSON document

#56

Post by tracker1 » Thu, Oct 30, 2014, 4:40 AM UTC

MBlume wrote: post #53

This is cool! My only substantive comment is criticism, but this is cool. I think it's a problem that remove and replace don't indicate what they're removing or replacing, the same way .patch files indicate what lines they're removing. This makes patch files reversible, which JSON patches appear not to be. It also provides some redundancy/error correction, which is handy.

I think that patch updates could be closer to the mongo syntax myself... I also don't like the "path" following slash notation over dot notation.

I'm also not entirely sure about adjustments to arrays with this syntax.

Re: JSON Patch – a format for describing changes to a JSON document

#57

Post by ioedward » Thu, Oct 30, 2014, 4:41 AM UTC

azinman2 wrote: post #7

We (Empirical) have implemented a similar couple years ago in our json-backed model. I see two big problems with this standard: 1. Why express paths as a string if you have the ability to encode it as an array in json? Otherwise you run up against the escaping problem for which they use ~0 and ~1 (of all things?!) which is only going to cause bugs. 2. Indexing into arrays is a problem in distributed computing because…

I emailed you, but also wanted to express in the comments that I would be definitely interested in idempotent json patches.

Re: JSON Patch – a format for describing changes to a JSON document

#58

Post by EGreg » Thu, Oct 30, 2014, 4:42 AM UTC

I once wrote JSON.diff and JSON.patch functions to do this thing.

Then I kind of stopped because I didn't see the point

Re: JSON Patch – a format for describing changes to a JSON document

#59

Post by EGreg » Thu, Oct 30, 2014, 4:44 AM UTC

https://www.npmjs.org/package/json-diff-patch

Re: JSON Patch – a format for describing changes to a JSON document

#60

Post by tracker1 » Thu, Oct 30, 2014, 4:45 AM UTC

drderidder wrote: post #34

This spec has come up a couple times; the first time I wondered whether a simpler approach might be acceptable: * setting properties to be updated * omitting properties to be unchanged * explicitly setting 'undefined' properties to be deleted I've used that approach and github seems to have done something similar. I like the idea of a patch spec for JSON - it's required by a strict implementation of REST with HTTP PA…

Most serializers will omit undefined (I don't believe it's allowed as a JSON value. That said, I think null should be sufficient. Though, MongoDB already defines update syntax pretty well, and would just assume follow that syntax.

http://docs.mongodb.org/manual/tutorial/modify-documents/

It's not perfect, but it is pretty widely used, and well thought out enough for this purpose.