Arguments against JSON-driven development • Hacker News Board

Re: Arguments against JSON-driven development

#41

Post by micimize » Thu, Aug 25, 2016, 2:20 PM UTC

While I see the point the Ulaş is getting at, I wouldn't call this JSON-driven development. I think JSON-driven development would use abstraction layers that are based on JSON, like JSON schema, and perhaps an OOP library that leverages it.

What I'd actually call this problem is a lack of abstraction. In functional programming, simple data structures are often preferred, and composable functions are used to manage complexity. A functional programmer might declare a function `to_structured_dict(enumerable, path)` and call it with `to_structured_dict(book_list, path=('shop_label', 'cell_label, 'book_id', count'))`

Re: Arguments against JSON-driven development

#42

Post by jack9 » Thu, Aug 25, 2016, 2:21 PM UTC

tscs37 wrote: post #28

JSON is best when it's solely used for serialization (or config files). Using it deep into the project makes no sense, the first step in handling JSON should always be to code it into native data structures.

First step is validation, then conversion, then logic. This keeps the json structure errors and changes from affecting the business logic.

Re: Arguments against JSON-driven development

#43

Post by K0nserv » Thu, Aug 25, 2016, 2:21 PM UTC

I agree with this and I've raised it several times in Objective-C codebases that can some times end up littered with NSDictionary:s everywhere taking no advantage of the Objective-C type mechanics. I don't think it's necessarily as bad in python or javascript. This is because these languages are dynamically typed and that diminishes the benefits of deserializing json to a native model. It's still valuable because you can guard against bad data by rejecting at the boundary of your program.

In statically typed languages however the added bonus is a lot more significant because the type system increases the benefits of deserializing JSON to native models. Take this Swift example

    import Foundation
    
    enum SerializationError: ErrorType {
        case InvalidData
    }
    
    struct Thing {
        let a: Int
        let b: String
        
        static func deserialize(fromDictionary data: [String:AnyObject]) throws -> Thing {
            guard let a = data["a"] as? Int, let b = data["b"] as? String else {
                throw SerializationError.InvalidData
            }
            
            return Thing(a: a, b: b)
        }
        
        static func deserialize(fromArray data: [[String: AnyObject]]) -> [Thing] {
            return data.flatMap {
                try? deserialize(fromDictionary: $0)
            }
        }
    }
    
    let data: [[String: AnyObject]] = [
        [
            "a": 10 as NSNumber,
            "b": "Hello" as NSString
        ],
        [
            "a": "10" as NSString,
            "b": 10 as NSNumber
        ]
    ]
    
    let models = Thing.deserialize(fromArray: data)

Not only do you end up with a native array of models you can also be certain that any type information is correct because invalid results have been thrown away during parsing.

Re: Arguments against JSON-driven development

#44

Post by PaulHoule » Thu, Aug 25, 2016, 2:22 PM UTC

adrianratnapala wrote: post #31

Cthulhu_ wrote: post #7

Earlier quoted context omitted.

When using parsers like e.g. Jackson or Gson for Java, this process is completely transparent and does not require any active thought from the developer - well, maybe if there's very specific formats that don't map 1:1 with the class that should be instantiated or generated from the json object. It's a bit more tricky in JS, both client-side and node. You can't work with the json string there, but after that you work…

I've never had good experiences with automated serialisation -- even though it sounds like other people do it with success. What's the secret? To give you a flavour of the kind of poblem: In C# (or rather .net) json.net reads JSON and calls setters from the target class. That means the setters have to be public, and you don't know what order they will be called in, and you have no real signal about when it is all don…

Automated serialization has gotten much better than it was in the bad old days of RPC and COM!

Re: Arguments against JSON-driven development

#45

Post by qwertyuiop924 » Thu, Aug 25, 2016, 2:24 PM UTC

wtbob wrote: post #5

> The fundamental advice on Unicode is decode and encode on system boundaries. That is, you should never be working on non-unicode strings within your business logic. The same should apply to JSON. Decode it into business logic objects on entry into system, rejecting invalid data. Instead of relying on key errors and membership lookups, leave the orthogonal business of type validity to object instantiation. This righ…

Part of the problem is that JSON and Sexprs aren't that they AREN'T serialization formats. They've been pressed into service as such, but they are actually notation for datastructures: In python, it may not be idiomatic to crawl dicts like this, but in JS, those aren't dicts, they're objects. If they've been de-serialized to some degree, they may even have their own methods.

By the same token, in Lisp, Sexprs aren't a serialization format. They're a notation for the linked cons cells that Lisp data is made of. In Lisp, that Sexpr will be crawled for data, or maybe even executed.

So while in Python, both may seem to be serialization formats, they aren't.

Either way, if the application programmer has any sense, they'll abstract away the format of their data. In a lisp app, you won't be cdring down a sexpr, you'll be calling a function to grab the necessary data for you, usually from a set of functions that abstract away the underlying sexpr implementation, and treat whatever it is as a separate datatype.

Of course, the sexpr might have been fed to an object constructor. Heck, it might be an object constructor, or a struct constructor. All of those types typically provide O(1) access, and autogenerated access functions, so it's the same story.

Re: Arguments against JSON-driven development

#46

Post by eropple » Thu, Aug 25, 2016, 2:27 PM UTC

adrianratnapala wrote: post #31

Cthulhu_ wrote: post #7

Earlier quoted context omitted.

When using parsers like e.g. Jackson or Gson for Java, this process is completely transparent and does not require any active thought from the developer - well, maybe if there's very specific formats that don't map 1:1 with the class that should be instantiated or generated from the json object. It's a bit more tricky in JS, both client-side and node. You can't work with the json string there, but after that you work…

I've never had good experiences with automated serialisation -- even though it sounds like other people do it with success. What's the secret? To give you a flavour of the kind of poblem: In C# (or rather .net) json.net reads JSON and calls setters from the target class. That means the setters have to be public, and you don't know what order they will be called in, and you have no real signal about when it is all don…

JSON.NET can use private setters. They just have to exist.

You do have to use thin constructors, but in JSON.NET there's a way to call a method post-deserialization.

Re: Arguments against JSON-driven development

#47

Post by thesmallestcat » Thu, Aug 25, 2016, 2:27 PM UTC

It's called a hash. Or a dict. Or a map. Not JavaScript Object Notation, FFS.

Re: Arguments against JSON-driven development

#48

Post by clifanatic » Thu, Aug 25, 2016, 2:29 PM UTC

Cthulhu_ wrote: post #8

I disagree with the anemic object argument. If an object is just there to store data and no behaviour, then that's fine - don't add behaviour if it doesn't need it. A large portion of back-end services are CRUD and data wrangling operations anyway - as in, convert data format A to data format B (which I guess could be a constructor or factory method if you're comfortable with having the conversion logic in a data cla…

> If an object is just there to store data and no behavior

Then why do you have it at all?

Re: Arguments against JSON-driven development

#49

Post by Bognar » Thu, Aug 25, 2016, 2:30 PM UTC

clifanatic wrote: post #48

Cthulhu_ wrote: post #8

I disagree with the anemic object argument. If an object is just there to store data and no behaviour, then that's fine - don't add behaviour if it doesn't need it. A large portion of back-end services are CRUD and data wrangling operations anyway - as in, convert data format A to data format B (which I guess could be a constructor or factory method if you're comfortable with having the conversion logic in a data cla…

> If an object is just there to store data and no behavior Then why do you have it at all?

To define a valid shape for related data.

Re: Arguments against JSON-driven development

#50

Post by lukasLansky » Thu, Aug 25, 2016, 2:31 PM UTC

adrianratnapala wrote: post #31

Cthulhu_ wrote: post #7

Earlier quoted context omitted.

When using parsers like e.g. Jackson or Gson for Java, this process is completely transparent and does not require any active thought from the developer - well, maybe if there's very specific formats that don't map 1:1 with the class that should be instantiated or generated from the json object. It's a bit more tricky in JS, both client-side and node. You can't work with the json string there, but after that you work…

I've never had good experiences with automated serialisation -- even though it sounds like other people do it with success. What's the secret? To give you a flavour of the kind of poblem: In C# (or rather .net) json.net reads JSON and calls setters from the target class. That means the setters have to be public, and you don't know what order they will be called in, and you have no real signal about when it is all don…

Yes, the automatic serialization is not a solution for the most pressing problems presented by the article -- it's just the first thing from all the things that has to be done at the boundary.

You have some DTO class that is your system typed idea about the structure of the JSON -- this class is quite useful as an implicit documentation, but it really has to stay internal to the boundary. You will use an autodeserializer to such class and then you will continue by constructing real object from deserialized data that can be presented to the rest of the application. During such construction you can validate state and return errros.

This step can be eased by some validating attributes on the boundary DTO properties, but there is always some custom logic that describes what is acceptable and what is not.