Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mendoza: Use stack machines to compute efficient JSON diffs

sanity.io

Post reply on HN
Print view View original

1–10 of 35 posts

Re: Mendoza: Use stack machines to compute efficient JSON diffs

#3

Post by anonymousblip »

Interesting approach! But aren't JSON arrays a pretty wasteful encoding?

Since this is an opaque serialization of an instruction set, why not try to encode more bits per number (JSON floats support lossless integers of many more bits), and moving the "symbol table" (string data) to the end?

This way you could also compress redundant symbols into single strings.

Re: Mendoza: Use stack machines to compute efficient JSON diffs

#5

Post by kmelve »

dreamer7 wrote: post #4

On a tangential topic, how does one go about integrating Sanity Studio with Hugo so non-developers can create content more quickly?

Hugo doesn't have an easy way of making pages/paginating from an API yet, so you'd have to first generate the markdown files. That can be done though. Here’s a demo: https://codesandbox.io/s/rj4y72j34n

Re: Mendoza: Use stack machines to compute efficient JSON diffs

#6

Post by AaronFriel »

anonymousblip wrote: post #3

Interesting approach! But aren't JSON arrays a pretty wasteful encoding? Since this is an opaque serialization of an instruction set, why not try to encode more bits per number (JSON floats support lossless integers of many more bits), and moving the "symbol table" (string data) to the end? This way you could also compress redundant symbols into single strings.

Now that you have .TEXT and .DATA sections, you're only a few sentences away from suggesting that the compression/diff algorithm generate and send WASM's binary encoding.

Re: Mendoza: Use stack machines to compute efficient JSON diffs

#7

Post by judofyr »

anonymousblip wrote: post #3

Interesting approach! But aren't JSON arrays a pretty wasteful encoding? Since this is an opaque serialization of an instruction set, why not try to encode more bits per number (JSON floats support lossless integers of many more bits), and moving the "symbol table" (string data) to the end? This way you could also compress redundant symbols into single strings.

Author of the article here.

The format itself is not strictly speaking coupled to JSON: https://github.com/sanity-io/mendoza/blob/master/docs/format.... If you can encode int8 and string more efficiently, then you can save some bytes with a custom binary encoding. However, you always need to be able to represent JSON as well. If a part of the JSON file isn't present in the old version, then you encode that part by the plain JSON.

> and moving the "symbol table" (string data) to the end? … you could also compress redundant symbols into single strings.

Sounds interesting, but in my experience these types of tricks are usually not paying off compared to just sending it through Gzip afterwards.

Post reply on HN