There is also Thrift.
You're Using JSON, Why not MessagePack?
41–50 of 66 posts
Re: You're Using JSON, Why not MessagePack?
#42Zip both resulting strings and I bet you it's a wash. Since JSON is mostly used (at least by me) for sending data to client side javascript, it doesn't make a difference. JSON is just easier.
Also - I'd have to assume that the performance benefit of native JSON parsing in modern browsers is non-negligible...
Re: You're Using JSON, Why not MessagePack?
#43Earlier quoted context omitted.
They're different types of thing. ZeroMQ doesn't replace JSON, it replaces HTTP. If you're using ZeroMQ it will help you get some bytes from one place to another, but you still need a serialization format. You might well send JSON over ZeroMQ.
Here's my specific doubt - my application uses arrays, maps, lists, etc. Can I not use zeromq to send these data-structures as binary data rather than serializing them to JSON ?
You're going to have to send them as a sequence of bytes somehow. MessagePack converts your data structures into a straightforward binary format, or lets you roll your own custom serializers easily (I've used it, and it's good stuff), and JSON does something similar in a less compact but more human-readable form. If your data structures are very simple, you could roll your own, e.g. send an array of ints by just converting them to network byte order and sending them.
Once you've got your data expressed as a sequence of bytes, you can send that with ZeroMQ. Or with HTTP, or raw sockets, or whatever. The serialization format and the transport protocol are pretty much completely independent.
Re: You're Using JSON, Why not MessagePack?
#44I'm not using JSON or MessagePack because I need to send arbitrary binary data. I understand why people use JSON, and it has its domain (sending data to and from the browser), but in other situations I don't think it is a good protocol. Another poster mentioned tnetstrings, those look interesting, however I am not sure how well those would handle binary data.
I wouldn't say JSON's main domain is sending data to the browser - I'd say it's exchanging common data structures (lists, hashes, strings, numbers) between environments, including between different languages. Sure, it doesn't make sense if you're shipping binary blobs around, but most of the time you're probably dealing with lists, hashes, strings and numbers.
JSON is good for edge labeled trees. XML is good for node labeled trees. But the data structures we want to transport are most often edge labeled graphs. You usually end up having to invent a mechanism for canonical node references to turn the tree into a possibly cyclic graph and vice versa. ISTM to improve on JSON, focus should be here.
Re: You're Using JSON, Why not MessagePack?
#45* fucking use JSON
* if needed: use snappy for blob storage (ie for key value stores)
* if you have an need for very high perfomance in your application that is mature to some degree, evaluate some binary protocols or compression and see how they perform.
the bad thing is that i already see full stack frameworks popping up "now with " and everybody will scream "YEAH!". Most YEAH!-sayers will seriously be butt-hurt by the plain fact that it isn't human readable. Switching between JSON/binary won't matter, there will be situations that are not debugable.
Trivial performance optimizations won't fix a broken design.
Re: You're Using JSON, Why not MessagePack?
#46Re: You're Using JSON, Why not MessagePack?
#47The "4x faster than Protocol Buffers" claim is misleading, as I have explained before: http://news.ycombinator.com/item?id=2146147 I'm working on a Protocol Buffer library that can serialize/deserialize to either JSON or Protocol Buffers. That way you can do all your development with JSON, but if you ever find you need the efficiency improvements of a binary format, you can just change your Serialize() call. Having a…
You already said it, but just wanted to back you up: The MessagePack benchmark is completely useless!
It consists of serializing 3 integers and a string 200,000 times.
MessagePack defines 27 different types¹ (excluding reserved ones) with variable bit length for the type identifier, length somewhat correlated with frequency of use.
A benchmark should therefor test real life data and a lot of it.
Their inability to produce such benchmark makes me question the sanity of splitting up e.g. the type marker for “array” into 3 different types depending on the size of the array — this adds complexity, so it would be good to know what exactly the authors based this design choice on, hopefully not that it made it faster to serialize a 3 element array 200,000 times.
¹ http://wiki.msgpack.org/display/MSGPACK/Format+specification...
Re: You're Using JSON, Why not MessagePack?
#48What about bencoding?
http://wiki.theory.org/BitTorrentSpecification#bencoding
Seriously, this is not an either or question. Just use the right tool for the right job. I wouldn't serialize my data to XML on a micro controller but so I wouldn't drive my JS frontend with binary serialized data.
Re: You're Using JSON, Why not MessagePack?
#49Earlier quoted context omitted.
Rather than down vote you for a simple mistake, I thought I'd let you know that xml is just as human readable as json.
Most people find formats like JSON and YAML much easier to read than XML.
It doesn't make XML any less human readable really. Hell, if you read and write XHTML a lot, it pretty much speaks volumes of its readability.
For large docs, I'd prefer XML too. I mean, I'd rather hunt down a missing closing tag than a missing closing bracket when the thing is pages upon pages long.
Re: You're Using JSON, Why not MessagePack?
#50How does it stack up against Pickle? Because that's what I'd be using in Python in places where JSON doesn't make much sense.