It's like JSON. but fast and small. • Hacker News Board

Re: It's like JSON. but fast and small.

#11

Post by ArchD » Sat, Dec 29, 2012, 11:05 AM UTC

stock_toaster wrote: post #4

json is kind of a lingua franca these days. For some kinds of things though, I do like tnetstring (has some nice qualities). msgpack (and bson too) always seemed a bit odd to me though. If you need binary packing, why not use protobufs or thrift?

Within the space of {schema-ful, schemaless} x {binary, text}, protobuf, {BSON, MessagePack}, JSON each occupy a distinct position. The position (schema-ful, text) is not very meaningful combination in practice and not covered by these, whereas the position (schema-less, binary) is a valid practical use case supported by {BSON, MessagePack}. For example, you don't know the schema before-hand but still want to minimize data size.

Re: It's like JSON. but fast and small.

#12

Post by Mooby » Sat, Dec 29, 2012, 11:06 AM UTC

gmac wrote: post #6

Equally, "it's like JSON, but binary and human-unfriendly".

Quite true. In fact, comparing MessagePack to JSON can only be interpreted as comparing $own_project with $most_popular_project, with hopes of getting some attention.

Re: It's like JSON. but fast and small.

#13

Post by scubaguy » Sat, Dec 29, 2012, 11:09 AM UTC

Joeri wrote: post #5

Someone at ebay already compared all the protocols and ended up advising json instead: http://news.ycombinator.com/item?id=3967157

Except when performance is important.

Re: It's like JSON. but fast and small.

#14

Post by oelmekki » Sat, Dec 29, 2012, 11:20 AM UTC

EDIT : Comment based on wrong assumption : string are actually smaller than in JSON, see judofyr comment.

I have to disagree on that news title, for the "It's like JSON" part.

As pointed out in a link[1] of Someone in an other comment here :

> A major use case of MessagePack is to store serialized objects in memcached.

This also seems very relevant (in article linked from here):

> typical short strings only require an extra byte in addition to the strings themselves.

So, stored strings are actually bigger (even only by one bit) than plain text json.

Of course, I'm biased because I'm a webdeveloper, but I see JSON mainly as a message exchange format, meant to communicate across languages and, most of the time, to communicate between server side and client side. Typically, I'll use server side helpers to format strings like value with currency, i18n messages, etc, which makes a lot of strings in any JSON document transmitted.

The original author made it clear it was not intended for server/client side communication : it's a very specific format intended to offer better performances on very specific use cases (I would rather know how it compares to BSON than to JSON).

JSON, on the other hand, is a generalist exchange format. Messagepack is definitely not like JSON.

[1] http://news.ycombinator.com/item?id=4092969

Re: It's like JSON. but fast and small.

#15

Post by pooriaazimi » Sat, Dec 29, 2012, 11:22 AM UTC

It looks cool, but be very careful about using it (in a browser, at least). Browsers have super-fast JSON parsers and parsing msgpack is much, much slower than parsing good old JSON.

This great comment (from 201 days ago) is exactly about this issue:

http://news.ycombinator.com/item?id=4091051

I quote a bit of it here:

    JSON size: 386kb
    MsgPack size: 332kb
    Difference: -14%

    Encoding with JSON.stringify 4 ms
    Encoding with msgpack.pack 73 ms
    Difference: 18.25x slower

    Decoding with JSON.parse 4 ms
    Decoding with msgpack.unpack 13 ms 
    Difference: 3.25x slower

and also this comment: http://news.ycombinator.com/item?id=4093077

   MessagePack is not smaller than gzip'd JSON
   MessagePack is not faster than JSON when a browser is involved
   MessagePack is not human readable
   MessagePack has issues with UTF-8
   Small packets also have significant TCP/IP overhead

Re: It's like JSON. but fast and small.

#16

Post by judofyr » Sat, Dec 29, 2012, 11:24 AM UTC

oelmekki wrote: post #14

EDIT : Comment based on wrong assumption : string are actually smaller than in JSON, see judofyr comment. I have to disagree on that news title, for the "It's like JSON" part. As pointed out in a link[1] of Someone in an other comment here : > A major use case of MessagePack is to store serialized objects in memcached. This also seems very relevant (in article linked from here): > typical short strings only require a…

> > typical short strings only require an extra byte in addition to the strings themselves.

> So, stored strings are actually bigger (even only by one bit) than plain text json.

Strings in JSON require two bytes (the quotes) in addition to the string itself.

Re: It's like JSON. but fast and small.

#17

Post by oelmekki » Sat, Dec 29, 2012, 11:27 AM UTC

judofyr wrote: post #16

oelmekki wrote: post #14

EDIT : Comment based on wrong assumption : string are actually smaller than in JSON, see judofyr comment. I have to disagree on that news title, for the "It's like JSON" part. As pointed out in a link[1] of Someone in an other comment here : > A major use case of MessagePack is to store serialized objects in memcached. This also seems very relevant (in article linked from here): > typical short strings only require a…

> > typical short strings only require an extra byte in addition to the strings themselves. > So, stored strings are actually bigger (even only by one bit) than plain text json. Strings in JSON require two bytes (the quotes) in addition to the string itself.

Oh, I see. I thought it meant "one bit more than JSON". Thanks for fixing it.

Re: It's like JSON. but fast and small.

#18

Post by rwmj » Sat, Dec 29, 2012, 11:34 AM UTC

XDR? ASN.1? Everything old is new.

Re: It's like JSON. but fast and small.

#19

Post by tarabukka » Sat, Dec 29, 2012, 11:39 AM UTC

gilgoomesh wrote: post #10

This is just an implementation of a binary encoded Property Tree. And not the best implementation out there since it lacks the redundant key/value elimination that more compact serialization formats use. Consider an array of maps where each of the maps have the same keys. By eliminating the redundant keys (have each subsequent occurrence simply reference earlier occurrences) you can halve the encoded data size relati…

Although if you're running this over HTTP, you might end up compressing it with DEFLATE (either directly or with gzip), which partly involves adding references to duplicated data... I'm not sure what the use case for this format is meant to be, though, so that might not apply.

Re: It's like JSON. but fast and small.

#20

Post by judofyr » Sat, Dec 29, 2012, 11:41 AM UTC

pooriaazimi wrote: post #15

It looks cool, but be very careful about using it (in a browser, at least). Browsers have super-fast JSON parsers and parsing msgpack is much, much slower than parsing good old JSON. This great comment (from 201 days ago) is exactly about this issue: http://news.ycombinator.com/item?id=4091051 I quote a bit of it here: JSON size: 386kb MsgPack size: 332kb Difference: -14% Encoding with JSON.stringify 4 ms Encoding wi…

sigh. Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there.

> MessagePack is not smaller than gzip'd JSON

But gzip'd MessagePack is smaller than gzip'd JSON, so?

> MessagePack is not faster than JSON when a browser is involved

Using it in the browser was never the use case for MessagePack. JSON these days are used for far more than browser-server-communication.

> MessagePack is not human readable

Fair enough. Although I don't consider JSON without newlines to be "human readable" either. I'll have to pipe it through a beautifier to read it.

> MessagePack has issues with UTF-8

MessagePack only stores bytes. It's up to you to decide how to interpret those bytes.

> Small packets also have significant TCP/IP overhead.

And?