Show HN: Simperium, a realtime data layer • Hacker News Board

Re: Show HN: Simperium, a realtime data layer

#41

Post by ecksor » Thu, May 10, 2012, 1:13 AM UTC

saurik wrote: post #38

Is every object on a separate timeline for purposes of operational transformation and synchronization? As an example where this would be noticeable, if I make a change to one object, and then a different one, am I guaranteed that everyone will see the first change before the second? Put from the different side, is it possible for me to not have seen changes on one object yet, but end up downloading a fresh copy of a…

Correct, each object is on a separate timeline for purposes of operational transform. If both clients are connected at the time the changes are made, then it shouldnt be possible to see the second change before the first. We keep an ordered history of all changes to all objects, though for efficiency, if a client that wasn't present when changes were initially made syncs, changes to multiple objects may appear in a different order. For example, a modification to object A, then object B, then object A again, would appear in the correct order for all clients when those changes are made. If a different client then syncs and asks for what's changed, we may group the two changes to object A such that there would appear to be two changes, a change to B then a change to A.

There is currently limited support for keeping a subset of data synchronized (we have plans to improve this) - if you're using the client libraries in an app, we've focused on supporting the common case of keeping all the data per bucket for a user synced. In the case of keeping all data mirrored to a backend, we provide an endpoint per bucket that you can listen to all changes for all users so you can keep the entire data store synchronized.

Re: Show HN: Simperium, a realtime data layer

#42

Post by RobotCaleb » Thu, May 10, 2012, 1:15 AM UTC

What is the brown text editor used in the video? Is it only on Mac?

Re: Show HN: Simperium, a realtime data layer

#43

Post by necubi » Thu, May 10, 2012, 1:17 AM UTC

cbsmith wrote: post #21

JaviSoto wrote: post #9

Absolutely priceless! This is an amazing product, and I can not start to imagine what developers are going to be able to do with this. Very exciting!

I'm missing something. This is just MVC style observer pattern. I get that it is well packaged/productized, but this is the kind of thing that developers have been doing for decades, with standard libraries for most of that time. Doing it with a browser is a bit newer, but ever since WebSockets it has been common place. A nice product yes, but I wouldn't expect this changes what developers are going to be able to do.

No, it is much more than that. You should watch the video, it demonstrates the platform solving some very hard problems with very little code.

The key here is (I believe) operational transformation (http://en.wikipedia.org/wiki/Operational_transformation), the algorithm behind products like Etherpad and Google Wave.

The observer pattern only handles the case where you have one client accessing the datastore. OT expands those ideas to handling multiple clients. Sync is a really hard problem, which is why almost nobody can get it right. Simperium seems to come closer than most and is packaged in a nice service.

It's really exciting to imagine the possibilities.

Re: Show HN: Simperium, a realtime data layer

#44

Post by greendestiny » Thu, May 10, 2012, 1:23 AM UTC

brianr wrote: post #27

cloudmike wrote: post #25

Earlier quoted context omitted.

Hey Brian, we know this is really important. Our beta label is mostly for the lack of pricing since the technology itself is production-ready. We're looking for more feedback and we aim to announce pricing soon. What do you think about Urban Airship's model based on active monthly users? What we like is that the costs are obvious and map clearly to your business.

MAU doesn't seem like a good model to me; it's not very well connected to your costs or customers' usage so it creates weird incentives on both sides. And it makes it expensive for an existing product with a large userbase to try out a small simperium-based feature. You guys are a hosting/infrastructure service, and it's probably for good reason that such services have historically charged based on usage. For you tha…

I'll second this. It makes you more comfortable as a user to know that if the incentives aren't misaligned your pricing will be more stable and easier for you to build a business on. I think its no surprise that AWS can keep making its services cheaper and App Engine had to change its pricing model entirely.

Re: Show HN: Simperium, a realtime data layer

#45

Post by jazzychad » Thu, May 10, 2012, 1:29 AM UTC

This looks incredible. My one burning question: For integrations on mobile devices, how does this affect the battery life? I'm concerned that there is some open-ended connection from the client to server listening for data changes which will drain the battery while the app is open.

Re: Show HN: Simperium, a realtime data layer

#46

Post by erichocean » Thu, May 10, 2012, 1:30 AM UTC

dtran wrote: post #18

"With Simperium, you own your data - that's important!" Not to take away from some of the great projects featured on HN recently, but statements by Simperium like this make me much happier as a developer and business owner: "We believe the best apps have both a great user experience and unique backend services." . From https://simperium.com/overview/ : It is not a backend-as-a-service. We believe the best apps have b…

I totally agree with the idea of owning your own data. I think that's the big thing lost with "web apps", and the cloud in general.

That said, the data needs to be on the cloud to be useful. So...what to do?

My sense is that a dedicated, trustworthy company needs to store the data and only the data. In particular, they need to not be in any other business, e.g. advertising, or selling stuff to developers, etc.

They need to be owned by the people entrusting them with their data, full stop. That means they have to charge for it.

BTW, I wrote code to do this very thing back in 2009, called HubSync. It's great to see all of these other companies making a go at it.

Re: Show HN: Simperium, a realtime data layer

#47

Post by onetwothreefour » Thu, May 10, 2012, 1:31 AM UTC

dtran wrote: post #18

"With Simperium, you own your data - that's important!" Not to take away from some of the great projects featured on HN recently, but statements by Simperium like this make me much happier as a developer and business owner: "We believe the best apps have both a great user experience and unique backend services." . From https://simperium.com/overview/ : It is not a backend-as-a-service. We believe the best apps have b…

This is definitely a cool and interesting service, sure, but how exactly do you "own your data"?

It's sitting in Simperium's database and you're locked into their platform once you decide to use it. Maybe there's an export, but that's irrelevant because all your apps in the wild and all your users are relying on this service to move data around.

If Simperium gets hacked, you get hacked. If Simperium goes down, you go down. If AWS goes down, Simperium goes down, and then you go down.

Say your app gets huge, and Simperium can't scale (and really, "half a million users" is nothing for a free service; that's not any meaningful scale and not a number that can really be cited as a signifier of ability to scale) -- what happens then?

Say you outgrow Simperium, and it's now costing you too much. What happens then?

The above isn't really about Simperium, but really about all backend-as-a-service services. They're great for starting out, but if you actually have a successful product you're going to need to be running your own backend at some point.

If you get too big/successful/etc and things start dying and you end up like Friendster before Facebook decides to buy your company, you're going to look back and wish you built your own backend and knew what was going on behind the scenes.

Re: Show HN: Simperium, a realtime data layer

#48

Post by erichocean » Thu, May 10, 2012, 1:35 AM UTC

Void_ wrote: post #10

This one is the best so far: 1. Works with any JavaScript library 2. You can write server-side logic 3. Operational Transformation baked in Seriously I cannot find anything bad about this project. (I really wanted to.) My questions: 1. Is the server side written in Python? 2. Is there offline support for JavaScript apps too? (I tried this with my own library and I think it was a wrong direction -- it only made it mor…

I can see a few missing things (correct me if I'm wrong):

1. peer-to-peer sync (no intermediate server)

2. cloud server isn't passive and stateless

No. 2 is very important for scalability. 1 is mostly a nice to have, especially on the WAN, where you can sync directly from, e.g. an iPad to your iPhone.

Both are doable (I did both in HubSync back in 2009).

Re: Show HN: Simperium, a realtime data layer

#49

Post by cloudmike » Thu, May 10, 2012, 1:38 AM UTC

jazzychad wrote: post #45

This looks incredible. My one burning question: For integrations on mobile devices, how does this affect the battery life? I'm concerned that there is some open-ended connection from the client to server listening for data changes which will drain the battery while the app is open.

Good question. We haven't heard any complaints about battery life so far. If you'd like you can check out Simplenote to judge for yourself: http://simplenoteapp.com

You do have control over the granularity of your updates. For example, we spoke to a game developer who would want to disable Simperium while a game is being played, and then enable it again at the end of levels. These coarse changes are supported since they'll resolve automatically when the client comes back online.

Re: Show HN: Simperium, a realtime data layer

#50

Post by erichocean » Thu, May 10, 2012, 1:42 AM UTC

onetwothreefour wrote: post #47

dtran wrote: post #18

"With Simperium, you own your data - that's important!" Not to take away from some of the great projects featured on HN recently, but statements by Simperium like this make me much happier as a developer and business owner: "We believe the best apps have both a great user experience and unique backend services." . From https://simperium.com/overview/ : It is not a backend-as-a-service. We believe the best apps have b…

This is definitely a cool and interesting service, sure, but how exactly do you "own your data"? It's sitting in Simperium's database and you're locked into their platform once you decide to use it. Maybe there's an export, but that's irrelevant because all your apps in the wild and all your users are relying on this service to move data around. If Simperium gets hacked, you get hacked. If Simperium goes down, you go…

The code gets open sourced, and you allow "syncing" to a local database, where everything is accessible.

I went through all of these business decisions for HubSync in late 2009, and came to the conclusion that you had to do the following to make "owning your own data" a reality:

1. Cannot charge developers or earn money on the actual data itself in any way. (Google will not be competing in this space.)

2. MUST, MUST, MUST allow removing any data from the cloud AND reloading it later, without losing any features/abilities.

3. USERS need to pay for cloud sync storage/service, NOT app developers.

4. Code must be open source, visible, and peer reviewed, just like a crypto algorithm.

If Simperium will do all of these, IMO they can raise money, and achieve a Dropbox-style exit. There's no doubt the potential is there.

2 is the most important. If you cannot remove your data from Simperium's backend and then reload it again later, either to their own backend, or a competitor, you have vendor lock in of the data, and you don't own your data.

3 means you have to have to do something like Dropbox, where you give away a certain amount of storage/sync for free. Ultimately, this is why I didn't pursue HubSync further, even though the products are nearly identical (even down to syncing CoreData on iOS with a web app running in the browser).

(In a nutshell, I didn't want to get funding and devote years of my life to data sync, when I'm in the middle of working on building a live action Pixar. But it's a great idea, and I really hope Simperium or someone else makes it happen.)