Aggregating Data Registries with Go

#1

Post by wwaites » Wed, Mar 30, 2011, 1:57 PM UTC

Aggregating Data Registries with Go
eris.okfn.org

Re: Aggregating Data Registries with Go

#2

Post by exch » Wed, Mar 30, 2011, 2:03 PM UTC

Interesting read for sure. And certainly nice to see Go being subjected to some real-world use.

About the need to represent the CKAN ORM in Go, in order to get some useful JSON deserialization going; I have found this be to useful myself in a few cases. mainly when wrapping RESTful services exposed as JSON.

You mentioned it was 'unexpected', but would you also qualify it as a downside to the way python handled this?

With a large object model, I can see how maintenance becomes a little tricky. Specially since Go has no notion of object relations.

Re: Aggregating Data Registries with Go

#3

Post by wwaites » Wed, Mar 30, 2011, 2:33 PM UTC

I think it depends. For the class of "quick hack to get some data from a JSON API and munge it" having to model the data can be a burden. Just using a Python dict and getting the values out is initially quite a lot simpler.

However for anything more complicated than that you start having to model the data anyways, in Python or Go or any other language otherwise it will quickly devolve into unmaintainable spaghetti. In this case, I was forced to do this early on, which made me think through the structure of the program more thoroughly than I might otherwise have done.

Re: Aggregating Data Registries with Go

#4

Post by wwaites » Wed, Mar 30, 2011, 2:37 PM UTC

Thinking about this some more, I think to replace model.Package with an interface. The existing struct of that name will become JsonPackage that implements it. This is because the CKAN representation of a dataset has some modelling problems (that are being addressed in parallel by the CKAN team) and it isn't clear that the existing models as represented in the JSON should overly inform the functioning of the ckand aggregator.