Live data from Hacker News

Arguments against JSON-driven development

okigiveup.net

181–190 of 306 posts

Re: Arguments against JSON-driven development

#181

Earlier quoted context omitted.

Part of the problem is that JSON and Sexprs aren't that they AREN'T serialization formats. They've been pressed into service as such, but they are actually notation for datastructures: In python, it may not be idiomatic to crawl dicts like this, but in JS, those aren't dicts, they're objects. If they've been de-serialized to some degree, they may even have their own methods. By the same token, in Lisp, Sexprs aren't…

What are serialization formats, then? What makes them different from notations for data structures?

First off, make no mistake, JSON and sexprs ARE serialization formats. I didn't establish this well in my orignal comment. However, they're designed be a notation for specific structures: dicts and arrays in json's case, and cons cells, specifically Lisp code in the case of sexprs. This is unlike say, XML, which defines a tree hierarchy, but NOT what the underlying structures are. That's the parser's job.

Re: Arguments against JSON-driven development

#182
Arguments for dict/list driven development:

NOTE: This isn't JSON driven-development, JSON just mimics base types like dict, list, string, numeric etc of all languages and a big reason it is so common especially in Python/Javascript.

- Large unknown lists/dictionaries data structures can be deserialized into dicts/lists for any language without issue.

Sometimes keys/data are unknown such as large attribute data sets that may always have new keys. In that case strong typing to an OO object will always be broken. Example: a facebook attribute set, keys/data that aren't set will not appear, new ones are added all the time, which would create a cat/mouse serialization/deserialization game. Same problem a binary structure has (offsets) when you really need a flexible keyed structure.

One missing key doesn't break your whole serialization/deserialization system built with strongly typed OO. Validation can be done on accept and if necessary convert into an OO system.

- If needed, classes that are backed/extended/inherited by a dict/list or set (or composition) that can load in the JSON/dict/lists and only expose the needed values after validation are useful.

i.e. a class that inherits from or composes to Dictionary for instance in C# would only fill keys that are necessary for the view data, not a bunch of extra null fields because it might not have that key/property. It also has the ability to deserialize objects that may have new keys. Not everything is a perfect world where data structures are known before-hand.

- It reduces complexity many times, no need for an OO serialize/deserialize layer when you are passing back as basic dict/list or JSON.

Why add complexity to something simple?

- Unless you control the server and the client, real-world data structures aren't a perfect map of keys/values to OO properties.

Assuming that is a system ready to break in the real-world many times. Someone adds a field to the DB object then all clients that use it can't serialize/deserialize. Real world serialization/deserialization has to accept in basic types dict/list, validate and then use as needed (some go to OO objects behind scenes). I see too many systems where people just have an EF object and expose that over a web api and just expect it to work, that is a bad example of poor encapsulation. Some fields don't need to be serialized to public apis. In Microsoft land MVVM was created to help stop this practice but still creates two sets of OO objects and breaks on any new keys/data (though breaking here may be desired for strong typing).

- Dict/list data structures can be easily setup to have cleaner naming and keys without tons of attribute/helpers

i.e. first-name key instead of first_name, FirstName, or firstName. This is more friendly to web/url naming that is common.

- Noted in the article, less memory used in many cases and highly optimized time in basic lists/dicts.

There are many more reasons...

Dicts, lists and basic string, numeric types are the base of all languages and computer science types. The reason this is common is it is simple to work with these types without added cruft of OO when needed.

OO does add complexity and if it isn't necessary you are just upping complexity for no reason (and memory). It is similar to the complaints of C coders to C++, basic structs and sets are sometimes less complex than C++ OO objects. Same thing with dict/list of some monstrosity of an OO serialization/deserialization system that breaks on every new key or field and you have to update the server and client rather than just increment a version and validation. The longer you code the more you see this.

OO objects should not be used all the time just like dict/lists shouldn't be used all the time.

Re: Arguments against JSON-driven development

#183

Earlier quoted context omitted.

Not gp, but one difference is you can have a notation that can't capture some state: think of the date class in JavaScript. JSON can't serialize this without resorting to string encoding, whereas something like a protobuf or pickle could.

What's really being described there, I think, is the notion that you can serialize and unserialize some data and get, to some extent, the "same" data back. Whilst that's more possible with protocol buffers or pickling (or whatever your language calls it), I can't think of any languages offhand which can round-trip any data. It's generally not possible to serialize objects denoting external resources - such as open fi…

Some extended sexpr notations (particularly those used in Common Lisp, although many Schemes also support it, as do other lisps) support self-referential data structures.

fds and sockets CAN, IIRC, to some degree, be sent to other processes on the same machine, but it's fairly limited.

As for serializing closures, CHICKEN Scheme's s11n egg is the most prominent, although not the only, example. It's fairly limited, once again, to avoid sending the forest with the bannana, as Joe Armstrong would put it.

This has nothing to do with our discussion, I just thought it was cool.

Re: Arguments against JSON-driven development

#184

> Once you go dict, you won't go back. This style of development is too easy, since dictionaries are baked into Python, and there are many facilities for working effectively with them. How is this an argument against using dictionaries? After 10 years of Python development, I do find myself using dictionaries rather than objects, in just the way that the author proscribes, but I'm finding it to be a genuine pleasure.

And what happens when one of those keys changes?

Re: Arguments against JSON-driven development

#185
post #184

> Once you go dict, you won't go back. This style of development is too easy, since dictionaries are baked into Python, and there are many facilities for working effectively with them. How is this an argument against using dictionaries? After 10 years of Python development, I do find myself using dictionaries rather than objects, in just the way that the author proscribes, but I'm finding it to be a genuine pleasure.

And what happens when one of those keys changes?

You change your code. Loose coupling a nice goal to aim for, but at the end of the day, somewhere deep down inside the code, you have to tightly couple to actually get anything done. Where that transition occurs is entirely programmer's discretion.

Re: Arguments against JSON-driven development

#186
post #173

Earlier quoted context omitted.

A notation for a data structure is a serialization format. > In Lisp, that Sexpr will be crawled for data, or maybe even executed. A s-expression cannot be executed; it's just text. The object which it denotes can be walked or executed via eval. Before that happens, the s-expression must be converted to that object. In other words, deserialized by the reader.

In that s-expressions are a notation for computation, they can be executed by an interpreter. This is what we refer to as execution. Even assembly is like this, there's no other reasonable way for it to work right now. This feels like you're hairsplitting to no obvious benefit other than increasing confusion. I could as easily say "mathematical notation isn't math it's just text, you can't evaluate '1 + 2' without a…

No, kaz has a point. do not confuse the shadow for that which cast it: Sexprs are a serialization format/notation for sets of conses. Most of the time, saying so is splitting hairs, but it bears mentioning here, as we're discussing serialization formats.

Re: Arguments against JSON-driven development

#187

Earlier quoted context omitted.

> As things stand now, the main reason why Python can't do it is because it could potentially break programs. Since the documented behavior of Python is compatible with what is suggested as a change, any program that relies on the behavior not reflecting as described in the "change" is asking to be broken (and quite possibly already broken across different implementations -- including different versions of the same i…

So, which is the case? (0) Python provides a compound value abstraction, but this abstraction leaks, because we can distinguish between multiple representations of the same compound value. (1) Python doesn't provide a compound value abstraction. With a few exceptions (e.g., garbage collection, which pretends memory is infinite, even though it's not), I tend not to consider so-called “leaky abstractions” actual abstra…

Python provides a value abstraction that includes (but is not distinct for) compound values.

It also provides an independent mechanism for examining physical (rather than logical) identity -- which is object identity; for all compound values (and all but a very narrow subset of simple values) there is no guarantee that the logical identity of values is equivalent to physical/object identity.

I don't see that this makes the value abstraction "leaky", though; for values, value identity is tested by equality. For objects, equality tests object equality but not object identity. Because Python is an "everything is an object" language, you can also test object identity for values, but for values in general there is no defined relationship between object identity and value identity, so this has no defined general logical (as opposed to physical) meaning for value types. (There are some values which are defined to also be singleton objects so that value and object identity are equivalent for these values; but that's a feature not of the value abstraction in Python but of the value-object mapping in Python.)

So I don't see the value abstraction itself as leaky.

OTOH, I'd probably be happier if Python had (whether it was "is" or something else) a clean logical identity test operator.

Re: Arguments against JSON-driven development

#188

At this point everyone should be using an evolvable (thrift, protocol buffers, avro, etc) schema format when they are storing or transmitting their data if they want to run an always on service - there is no downtime for migrations in the real world. Trying to do this ad-hoc with JSON is a lost cause and will eventually lead you to failure at runtime or worse, data loss situations.

Anyone have a good blog post handy on this?

This is a related post that talks about the entity store that I built for my startup that was ultimately acquired by Twitter. They internally had their own similar store called ThriftStore that worked similarly. Google also builds their systems like this using protocol buffers. It is a pretty easy pattern and could theoretically be done with JSON if you provide some kind of schema and evolution strategy on read.

https://javarants.com/havrobase-a-searchable-evolvable-entit...

Re: Arguments against JSON-driven development

#189
post #120

Earlier quoted context omitted.

Part of the problem is that JSON and Sexprs aren't that they AREN'T serialization formats. They've been pressed into service as such, but they are actually notation for datastructures: In python, it may not be idiomatic to crawl dicts like this, but in JS, those aren't dicts, they're objects. If they've been de-serialized to some degree, they may even have their own methods. By the same token, in Lisp, Sexprs aren't…

JSON is a serialization format that was based on the data structure notation for Javascript. It is, however, a serialization format. Javascript objects are a superset of JSON, as they can contain arbitrary objects and functions, which JSON can not, and "true" Javascript object notation can elide quote marks or use apostrophes for keys, whereas JSON strictly specifies double-quotes around keys. The problem that arises…

I'm not sure I prefer the strongly typed approach: I come from Lisp, so the approach is: "read it, validate it, and then wrap it in functions to hide the implementation in case we change it."

This works well in Lisp, where the line where objects end, and structs, lists and functions begin is hazy at best. Besides with a bit of wrangling, you could probably just pass your validated serialized data to the object constructor as the arguments. Or you could just write a struct, which is simpler than an object, provides O(1) access, and you can still probably easily pass your datastructure, or something close, into the constructor as the arglist.

Re: Arguments against JSON-driven development

#190

Earlier quoted context omitted.

So, which is the case? (0) Python provides a compound value abstraction, but this abstraction leaks, because we can distinguish between multiple representations of the same compound value. (1) Python doesn't provide a compound value abstraction. With a few exceptions (e.g., garbage collection, which pretends memory is infinite, even though it's not), I tend not to consider so-called “leaky abstractions” actual abstra…

Python provides a value abstraction that includes (but is not distinct for) compound values. It also provides an independent mechanism for examining physical (rather than logical) identity -- which is object identity; for all compound values (and all but a very narrow subset of simple values) there is no guarantee that the logical identity of values is equivalent to physical/object identity. I don't see that this mak…

This simply doesn't make sense. It violates the indiscernibility of identicals, which is one of the cornerstones of Western logic. I can accept different opinions on several matters (e.g., the extent to which using objects is a good idea), but throwing logic out of the window is just too much.
Post reply on HN