It's like JSON. but fast and small. • Hacker News Board

Re: It's like JSON. but fast and small.

#81

Post by ianb » Sat, Dec 29, 2012, 8:22 PM UTC

Well, with Python it doesn't seem particularly fast. I put together a small test: http://svn.colorstudy.com/home/ianb/msgpack_json_comparison....

It uses some random JSON data I found. msgpack is more compact (87% of the JSON representation), though not dramatically so.

json encode: 5.54msec simplejson encode: 8.27msec msgpack encode: 11.4msec json decode: 16.4msec simplejson decode: 4.06msec msgpack decode: 2.84msec

I'm confused about why json is faster at encoding and simplejson is faster at decoding. simplejson is fastest when you combine encoding and decoding.

Re: It's like JSON. but fast and small.

#82

Post by josephlord » Sat, Dec 29, 2012, 8:40 PM UTC

mikeash wrote: post #73

josephlord wrote: post #38

Earlier quoted context omitted.

>> MessagePack has issues with UTF-8 > MessagePack only stores bytes. It's up to you to decide how to interpret those bytes. Then it's not really "like JSON" is it? I'm not saying that it is a bad thing or that it isn't a better option for many current uses of JSON but the "like JSON but fast and small" is a misleading pitch.

How's that? Just use UTF-8 for your strings and you're on par with JSON, no?

For me JSON's strength is that you can throw typed data in simple structures straight at it and be confident of getting the same out the other end without worrying about the exact details of the encoding. If you exceed the supported types you have to work harder but in great many cases that is not necessary.

If MsgPack doesn't let me throw strings, numbers plus arrays and dictionaries containing arbitrary further structures without me having to worry about defining the encoding it isn't on par.

Obviously any sane data format can contain any data but not equally easily or efficiently.

Re: It's like JSON. but fast and small.

#83

Post by masklinn » Sat, Dec 29, 2012, 8:48 PM UTC

judofyr wrote: post #20

pooriaazimi wrote: post #15

It looks cool, but be very careful about using it (in a browser, at least). Browsers have super-fast JSON parsers and parsing msgpack is much, much slower than parsing good old JSON. This great comment (from 201 days ago) is exactly about this issue: http://news.ycombinator.com/item?id=4091051 I quote a bit of it here: JSON size: 386kb MsgPack size: 332kb Difference: -14% Encoding with JSON.stringify 4 ms Encoding wi…

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

> sigh. Why do we always want to declare a winner?

You may want to ask whoever copywrote messagepack's website, they kind-of are the one declaring:

> It's like JSON. but fast and small.

Re: It's like JSON. but fast and small.

#84

Post by masklinn » Sat, Dec 29, 2012, 8:53 PM UTC

ianb wrote: post #81

Well, with Python it doesn't seem particularly fast. I put together a small test: http://svn.colorstudy.com/home/ianb/msgpack_json_comparison.... It uses some random JSON data I found. msgpack is more compact (87% of the JSON representation), though not dramatically so. json encode: 5.54msec simplejson encode: 8.27msec msgpack encode: 11.4msec json decode: 16.4msec simplejson decode: 4.06msec msgpack decode: 2.84msec…

> It uses some random JSON data I found. msgpack is more compact (87% of the JSON representation), though not dramatically so.

And you can probably end up with a draw if you gzip both

Re: It's like JSON. but fast and small.

#85

Post by stcredzero » Sat, Dec 29, 2012, 8:58 PM UTC

ville wrote: post #45

zurn wrote: post #33

Earlier quoted context omitted.

A lot of these bit shaving goose chases would be short circuited if browsers exposed zlib !

What would be the advantage of exposing it instead of compressing the data transparently in the background. (Like it is done already.)

Freeing up the time of compulsive bit chasers?

Re: It's like JSON. but fast and small.

#86

Post by masklinn » Sat, Dec 29, 2012, 9:02 PM UTC

justincormack wrote: post #27

jarin wrote: post #21

Seems like this could be useful as an alternate API format. I'd like to see sites offering plist (bigger than JSON but dead simple parsing in Cocoa) and serialized formats as well as JSON and XML and all that.

Thats a large amount of testing, potential bugs for what purpose?

Ease of consumption from Cocoa (mostly Touch) applications?

Re: It's like JSON. but fast and small.

#87

Post by newishuser » Sat, Dec 29, 2012, 9:04 PM UTC

MessagePack Has nothing to do with JSON.

This page is a good example of how marketing copy can undermine your product.

MessagePack is a data serialization format, it has about as much to do with JSON as honey has to do with milk, but they went and started a flame war with no more than 7 tiny words, "It's like JSON. but fast and small."

Bam, ruined their own message. Now instead of reading that page thinking "What can message pack do for me?" you read the page thinking "Better than JSON huh? I'll be the judge of that."

Marketers could do us all an enormous favor, themselves included, if they stuck to promoting the positive aspects of their products instead of the negatives of the competition. It puts the consumer in the wrong mindset. Why would you want someone wandering around what is essentially your store looking for negative aspects of your product. Seriously, just stop being negative.

Re: It's like JSON. but fast and small.

#88

Post by frsyuki » Sat, Dec 29, 2012, 9:43 PM UTC

Here is a benchmark result of msgpack, json, bson and protobuf: https://gist.github.com/4348013

An important point we should consider here is that actual performance depends on implementation.

Re: It's like JSON. but fast and small.

#89

Post by frsyuki » Sat, Dec 29, 2012, 9:47 PM UTC

kzk_mover wrote: post #30

sudhirj wrote: post #22

Earlier quoted context omitted.

The Pintrest testimonial on the msgpack website is a very good example of use case: serializing and de-serializing for memcache. The don't claim to be using it for browser-server communication.

For those who are interested in Pinterest usecase (Memcached + MessagePack), please access to this url too. > http://engineering.pinterest.com/posts/2012/memcache-games/

Did you guys see this video?: http://vimeo.com/53261709 Github uses MessagePack for their internal RPC protocol.

Re: It's like JSON. but fast and small.

#90

Post by jspthrowaway2 » Sat, Dec 29, 2012, 10:18 PM UTC

pjscott wrote: post #72

Too wrote: post #64

Earlier quoted context omitted.

Doesn't have to be stupid. A normal integer in binary representation ALWAYS consumes 4 bytes. With string representation you have 1000 numbers that will beat that as they only need <=3 characters. Numbers in that range are actually used very frequently.

Funny you should mention that. Msgpack uses a variable-length binary encoding for integers. It can, for example, fit integers in the interval [-32, 127] into a single byte. And unlike that fixed four-byte encoding you mention, it can also handle 64-bit integers. (I'm not disagreeing with you, or anything. I just like talking about binary encodings.)

There is also zigzag encoding, something Protocol Buffers (I believe) introduced for general usage. Thrift later copied it for the compact encoding.

It's pretty interesting.