Live data from Hacker News

It's like JSON. but fast and small.

msgpack.org

71–80 of 103 posts

Re: It's like JSON. but fast and small.

#71
post #31

and why not just use protocol buffers?

MessagePack is schema-less, whereas protobuf is not.

It's also almost API-compatible with JSON encoders, so if you wrote a program that uses JSON for serialization, switching to msgpack is often as easy as doing a global find-and-replace.

Re: It's like JSON. but fast and small.

#72
post #64

Earlier quoted context omitted.

I did a quick test on some JSON data I use in my webapps; ~100k-1M blobs, mostly numbers. gzipped MessagePack data was just about the same size as gzipped JSON data. If you care about size on the wire over HTTP, then MessagePack may not be an improvement. It's amazing how well gzip compresses the stupid ASCII encoding of numbers in JSON.

Doesn't have to be stupid. A normal integer in binary representation ALWAYS consumes 4 bytes. With string representation you have 1000 numbers that will beat that as they only need <=3 characters. Numbers in that range are actually used very frequently.

Funny you should mention that. Msgpack uses a variable-length binary encoding for integers. It can, for example, fit integers in the interval [-32, 127] into a single byte. And unlike that fixed four-byte encoding you mention, it can also handle 64-bit integers.

(I'm not disagreeing with you, or anything. I just like talking about binary encodings.)

Re: It's like JSON. but fast and small.

#73
post #20

Earlier quoted context omitted.

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

>> MessagePack has issues with UTF-8 > MessagePack only stores bytes. It's up to you to decide how to interpret those bytes. Then it's not really "like JSON" is it? I'm not saying that it is a bad thing or that it isn't a better option for many current uses of JSON but the "like JSON but fast and small" is a misleading pitch.

How's that? Just use UTF-8 for your strings and you're on par with JSON, no?

Re: It's like JSON. but fast and small.

#74
post #20

Earlier quoted context omitted.

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

I think developers like to declare winners for very, very good reasons typically. Winners are often very useful. For example in settling on a standard once a good or great solution is found so a community can thrive around it, lots of open tools and open code can be built for it, developers can build upon it with a roadmap in mind, sites can publish with it knowing they can reach a maximum audience, etc etc etc etc.…

Choice is a good thing.. do you use a fork, to open doors, eat,scratch your foot, kill flies, start a fire, just because you have to use the one-and-ultimate tool ?

json are not the best for every scenario, as also msgpack does not have the best fitness for everything..

Nature is pretty wise on that matter, it "launch" several possibilities, cause the environment is always changing.. so in a particular giver scenario, one of the possibilities (biological agents) can succed..

but that one that succeded in a particular scenario, will fail in another, where now, another subject who has failed is the better option, adaptation..

so.. choise is a good thing.. why people are so lazy, and dont like to think? everything must be ready..

its our duty to choose the best tool to solve a given problem.. its cool that i can even use something "obsolet" from the 1800´s to solve a 2012 problem..

So if we keep thinking.. "oh that old unwanted thing from the past", we will never see it..

Winners are only winners in a given scenario.. its good to have choice, even against all boards and standards i prefer to decide from myself, than to have others to do that decision for me..

This is technology, not fashion.

Re: It's like JSON. but fast and small.

#76

Earlier quoted context omitted.

I noticed that opportunity too. If anyone is looking for a fun project... The next thing would be to have practices and/or libraries that switch techniques based on browser support.

Is was a fun project actually. https://github.com/creationix/msgpack-js-browser

Question out of curiosity: did you notice any performance improvements from using DataView and ArrayBuffer?

Re: It's like JSON. but fast and small.

#77
post #67
post #50

Earlier quoted context omitted.

Assuming you are using a browser (other clients support this too)... For now, can't you just encode the image in your json using base 64 and use the 'data' uri scheme to decode it? Sure the encoded data can't be more than 32k in most browsers, but that is huge avatar.

IE8 has the 32k limitation. I don't think anything else does.

Doh, you are correct. I had that piece of info reversed. Thanks for clarifying.

Re: It's like JSON. but fast and small.

#78
post #73

Earlier quoted context omitted.

>> MessagePack has issues with UTF-8 > MessagePack only stores bytes. It's up to you to decide how to interpret those bytes. Then it's not really "like JSON" is it? I'm not saying that it is a bad thing or that it isn't a better option for many current uses of JSON but the "like JSON but fast and small" is a misleading pitch.

How's that? Just use UTF-8 for your strings and you're on par with JSON, no?

JSON can't carry binary data because it must be UTF-8. Glancing over the spec, msgpack commits the complementary sin, it can't carry UTF-8 data because it can only carry binary data. Yes, it is possible to stuff some UTF-8 bytes into a binary, but you still semantically have a binary, which could be anything. You must bring additional external information to the party to know what that binary hunk actually is.

In theory this may sound like no big deal; in practice I've observed in similar cases it's a disaster. Average developers routinely muck this up (and that's just me being conservative, above-average ones can choke on this problem too). msgpack really ought to have a dedicated string type, and either declare that this string is always a particular encoding, or give a way to declare what encoding the string is in. (The second is more flexible and arguably more correct, but in something like this where there's going to be dozens of libraries trying to implement it, it is virtually guaranteed that a number of them will muck the variable encoding support up badly, so in practice I'd go with mandatory UTF-8 too.)

Re: It's like JSON. but fast and small.

#79
post #20

It looks cool, but be very careful about using it (in a browser, at least). Browsers have super-fast JSON parsers and parsing msgpack is much, much slower than parsing good old JSON. This great comment (from 201 days ago) is exactly about this issue: http://news.ycombinator.com/item?id=4091051 I quote a bit of it here: JSON size: 386kb MsgPack size: 332kb Difference: -14% Encoding with JSON.stringify 4 ms Encoding wi…

sigh . Why do we always want to declare a winner? Anyone who has tried to store binary data in JSON knows that MessagePack does indeed have a use case. And anyone who is working with browsers knows that JSON/JS is the king there. > MessagePack is not smaller than gzip'd JSON But gzip'd MessagePack is smaller than gzip'd JSON, so? > MessagePack is not faster than JSON when a browser is involved Using it in the browser…

sigh Why do we always have to misconstrue messages that conflict with our personal beliefs? :)

The GP was not declaring a winner. He was responding to overreaching title of the link. Yes there are similarities to JSON, and yes it may be faster and smaller, but only in certain cases. It is very relevant for the GP to point out that in the most common JSON cases (browser communication), those claims are basically false or only marginal.

This is reasoned argumentation, not arbitrary judgement to declare The One.

Post reply on HN