Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's like JSON. but fast and small.

msgpack.org

Post reply on HN
Print view View original

61–70 of 103 posts

Re: It's like JSON. but fast and small.

#61

Post by NelsonMinar »

judofyr wrote: post #20
pooriaazimi wrote: post #15

It looks cool, but be very careful about using it (in a browser, at least). Browsers have super-fast JSON parsers and parsing msgpack is much, much slower than parsing good old JSON. This great comment (from 201 days ago) is exactly about this issue: http://news.ycombinator.com/item?id=4091051 I quote a bit of it here: JSON size: 386kb MsgPack size: 332kb Difference: -14% Encoding with JSON.stringify 4 ms Encoding wi…

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

I did a quick test on some JSON data I use in my webapps; ~100k-1M blobs, mostly numbers. gzipped MessagePack data was just about the same size as gzipped JSON data. If you care about size on the wire over HTTP, then MessagePack may not be an improvement. It's amazing how well gzip compresses the stupid ASCII encoding of numbers in JSON.

Re: It's like JSON. but fast and small.

#62

Post by andrewvc »

gilgoomesh wrote: post #10

This is just an implementation of a binary encoded Property Tree. And not the best implementation out there since it lacks the redundant key/value elimination that more compact serialization formats use. Consider an array of maps where each of the maps have the same keys. By eliminating the redundant keys (have each subsequent occurrence simply reference earlier occurrences) you can halve the encoded data size relati…

As far as redundant kv encoding isn't gzip good enough? Sure, it's more generalized, but tailored compression schemes often aren't the win they're touted as. Additionally, they can't exploit scenarios where multiple pairs are repeated (for instance gzip would compress [{x: 1}, {x: 1}, {x: 1}, ...] far more efficiently.

Re: It's like JSON. but fast and small.

#63

Post by ttflee »

jarin wrote: post #21

Seems like this could be useful as an alternate API format. I'd like to see sites offering plist (bigger than JSON but dead simple parsing in Cocoa) and serialized formats as well as JSON and XML and all that.

By 'plist', did you mean plist xml1 format or binary1 format?

Re: It's like JSON. but fast and small.

#64

Post by Too »

NelsonMinar wrote: post #61
judofyr wrote: post #20

Earlier quoted context omitted.

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

I did a quick test on some JSON data I use in my webapps; ~100k-1M blobs, mostly numbers. gzipped MessagePack data was just about the same size as gzipped JSON data. If you care about size on the wire over HTTP, then MessagePack may not be an improvement. It's amazing how well gzip compresses the stupid ASCII encoding of numbers in JSON.

Doesn't have to be stupid. A normal integer in binary representation ALWAYS consumes 4 bytes. With string representation you have 1000 numbers that will beat that as they only need <=3 characters. Numbers in that range are actually used very frequently.

Re: It's like JSON. but fast and small.

#65

Post by smosher »

judofyr wrote: post #20
pooriaazimi wrote: post #15

It looks cool, but be very careful about using it (in a browser, at least). Browsers have super-fast JSON parsers and parsing msgpack is much, much slower than parsing good old JSON. This great comment (from 201 days ago) is exactly about this issue: http://news.ycombinator.com/item?id=4091051 I quote a bit of it here: JSON size: 386kb MsgPack size: 332kb Difference: -14% Encoding with JSON.stringify 4 ms Encoding wi…

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

Why do we always want to declare a winner?

To be fair, MessagePack seems to be throwing down the gauntlet with their tagline. The parent comment also makes some important points that anyone considering the two should be aware of.

Re: It's like JSON. but fast and small.

#66

Post by gmac »

collinvandyck76 wrote: post #53
creationix wrote: post #40

Earlier quoted context omitted.

How is it binary unfriendly? It's designed to be super easy to parse using C. The lengths and types are in the first byte(s) and most conversions can be done using typecasts. And as far as human readable binary format, it's not that bad. I can usually read a hex dump of msgpack if I've been working with it all day.

But binary, and also, it is human-unfriendly.

Exactly (or I'd have written "... binary- and ...").

Re: It's like JSON. but fast and small.

#67

Post by falien »

millerm wrote: post #50
benatkin wrote: post #36

Earlier quoted context omitted.

> Using it in the browser was never the use case for MessagePack. Hopefully it will be soon, with degradation for old browsers. I'd like to be able to load an avatar as binary png data with the rest of the data.

Assuming you are using a browser (other clients support this too)... For now, can't you just encode the image in your json using base 64 and use the 'data' uri scheme to decode it? Sure the encoded data can't be more than 32k in most browsers, but that is huge avatar.

IE8 has the 32k limitation. I don't think anything else does.

Re: It's like JSON. but fast and small.

#68

Post by pjscott »

marcosdumay wrote: post #56
judofyr wrote: post #20

Earlier quoted context omitted.

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

> Why do we always want to declare a winner? You are right. But I came to that page after reading about a faster and smaller JSON at HN, thus I was expecting to find a faster and smaller JSON. It's just another case of bad marketing hurting a brand.

Oh for crying out loud. In a lot of use cases, msgpack can be treated as a smaller, faster JSON. Let's do a quick experiment. Here's a small, very ordinary JSON document:

http://graph.facebook.com/btaylor

Using Python's json and simplejson modules, which are quite zippy, the encoding time was about 11.4 us, and the decoding time was about 8 us. With msgpack, the encoding and decoding times were 2.7 us and 1.7 us, respectively. The encoded size was 187 bytes with JSON, and 140 bytes with msgpack. Zlib compression brought those down to 127 and 125 bytes, while bringing the total encode-and-compress time up to 25 us for JSON and 16 us for msgpack.

Re: It's like JSON. but fast and small.

#69

Post by pjscott »

sudhirj wrote: post #22
pooriaazimi wrote: post #15

It looks cool, but be very careful about using it (in a browser, at least). Browsers have super-fast JSON parsers and parsing msgpack is much, much slower than parsing good old JSON. This great comment (from 201 days ago) is exactly about this issue: http://news.ycombinator.com/item?id=4091051 I quote a bit of it here: JSON size: 386kb MsgPack size: 332kb Difference: -14% Encoding with JSON.stringify 4 ms Encoding wi…

The Pintrest testimonial on the msgpack website is a very good example of use case: serializing and de-serializing for memcache. The don't claim to be using it for browser-server communication.

Another use is storing a lot of small items in Redis. It's compact in memory, and can be manipulated using Redis 2.6's Lua scripting engine, which is a very handy combination.

Re: It's like JSON. but fast and small.

#70

Post by Tloewald »

marcosdumay wrote: post #56
judofyr wrote: post #20

Earlier quoted context omitted.

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

> Why do we always want to declare a winner? You are right. But I came to that page after reading about a faster and smaller JSON at HN, thus I was expecting to find a faster and smaller JSON. It's just another case of bad marketing hurting a brand.

My first thought was "maybe it would be a useful JSON replacement for Unity" (which still lacks native or even aolid third party JSON support). I'm not sure how big a deal the lack of seamless UTF8 supportwill be in practice (in most cases no big deal i think) but it does look promising owing to its C# implementation.

I do agree the tagline is bound to annoy.

Post reply on HN