Show HN: Simperium, a realtime data layer

This is interesting. We built something very much like this internally at SeatMe. On the iOS side, it's almost identical. It's how we keep our iPads up to date. Might not of built something internally ourselves a year ago if there was a platform like this.

The biggest difference is that since we are not a general platform, we can make assumptions about the model and how version each release and we can built in some constrains and unique security models.

We took a lot of cues from the OData spec and Microsoft's reference design.

The rest of this comment is mostly targeted at the creators of Simperium.

You made very similar design decisions to us in a lot of ways. A lot problems though that you will face I see with your path here so I have a few tips for you.

* It sucks the iOS client isn't open source. I get scared of linking in third party libs into iOS projects because I have to account for anything you do when I go to Apple to submit my app.

* You really got to brush up on the objective-c naming conventions. Not to be harsh but `-(id)getCustomObjectForKey:(NSString * )key;` makes me cringe.

* Don't require me to have to know about your categories.

* Namespace your categories so you don't smash mine ("something like "SP_encodeBase64WithString" instead of "encodeBase64WithString")

* If you include third party libs, you MUST rename them and prefix with your prefix. I see you use ASIHTTPRequest, DDLogger, SocketIoClient, AsyncSocket, Reachability, and a few others. You will smash everyone else's implementations if they already had them included.

* Don't use ASIHTTPRequest internally (it's old and unmaintained and doesn't play nicely with ARC)

* PREFIX ALL YOUR CODE. We don't have any real name-spacing objective-c. As a framework developer, you have to be aware of that more than anyone else. I shouldn't be seeing DiffMatchPatch and SocketIoClient show up in my symbol list after linking your lib

* Your addDelegate/removeDelegate is funny. After you exhausted the need for one delegate, switch to NSNotificationCenter.

* DON'T USE XIB/NIBs. Interface builder for iOS was an after thought and it's only a 90% solution (unlike with Cocoa where interface builder was a first class product built side by side with AppKit). Especially don't make me have to include your XIBs in my bundle. At the very least give me a bundle with it in it.

* Separate your GIT repo. If I want to include your library as a submodule I have to take all the client libraries as well.

Now when it comes to the actual sync layer and how you generate JSON dictionaries and apply "patches" this is fine code.

Here are some feature requests:

* Instead of having to give you a single NSManagedObjectContext let us register them. We have a few (some use different concurrency types).

* Let us override what gets generated or if we want to ignore a field with userInfo keys in the core data model.

* Let us get an idea of your backend sync processes to be able to suspend and start them when we need and know if anything is pending. Give us a callback that you still have things to queued and when we are done so we can at our leisure set up UIBackgroundTasks on iOS 4+.

On an unrelated note, why not create this as a NSIncrementalStore and just put your code behind the persistent store coordinator instead of monitoring it? We are doing the same as you at the high level because we wrote our code pre iOS 5.0 but iOS 5 gives you an awesome new toy there.

The website mentions that you are using google-diff-match-patch in the JavaScript client to merge changes, it seems as the API does not require the developer to actually specify things like "the user added an A at this position" as opposed to just "commit the changes to this entire object". Is there a reason other than simplicity for this API (I guess maybe because CoreData doesn't have that abstraction, and I presume your timeline was to start with figuring out how to sync CoreData), and on iOS are you also using google-diff-match-patch (there was not the same explicit mention of it in the documentation there)? (edit: Actually, I guess the comment from zbowling about DiffMatchPatch answers the second half of that. ;P)

BTW, this is generally really awesome: I am (right now, as in I'm sitting there right now ;P) helping teach a class on cloud computing at UCSB that happens to currently be discussing database synchronization and replication; after spending a bunch of time today discussing "how PostgreSQL is implemented and the basis of different isolation levels in the SQL standard and in MVCC" I took the time to tell everyone about Simperium (which probably makes more sense if I mention that I've looked into building something similar before for my projects; I'm glad someone else finally seems to be coming at it from the correct mindset). Everyone here seems to agree: this is going in a great direction.

Thanks saurik! Yeah that's right, we've tried to make it as easy as we can for developers to use. One of our goals has been to let developers be able to work with data as they do normally if it were just local, while we handle figuring out whats changed.

Please demo this to Evernote. They need to leverage this asap!

Ok, with those semantics (and I think I understand the basis implementations that would cause them), do you perform the "resync" step (which has the property that you can at least get the catch-up changes in a single bulk and hopefully fast download for the entire bucket, even with the internal reordering that happened over time) as a single transaction (by locking the persistant store coordinator); and if not, why?

With keeping the objects on separate timelines, do you or can you run into any problems with CoreData inverse relationships? I'd be concerned that my objects would have issues where a change got synced for object A that had it linking to object B, but somehow my application crashed before it got the synchronization for object B sent. I can imagine various reasons why this concern is stupid, including "saurik doesn't understand how CoreData works" (as I only even learned of its existence a couple days ago).