Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ditto: Real-time sync for apps even without the internet

ditto.live

Post reply on HN
Print view View original

71–80 of 89 posts

Re: Ditto: Real-time sync for apps even without the internet

#71

Post by bigfish24 »

techdragon wrote: post #70

Might seem like odd questions, but is there a plan to handle the "too much data" case? and how protocol agnostic can this be? This seems really cool, and I can think of some strait forward uses for it ... but working on a satellite project where communications are 75% of the hassle, this sort of thing seems like it could be very useful... If it can deal with all the "weird" (its very normal for space, just not used a…

Adam (cofounder) - great question! Yes, this situation is very common for our use-cases. Ditto supports the concept of `eviction` [1]. For example, imagine a restaurant that wants to store all of today's orders in their devices, but then evict the data when they close out at end of day (and data is synced into Cloud backend). Eviction is a core concept because we consider devices using Ditto's SDK a "Small peer" that should only have the data it wants - queries define which data is synced and you can always evict anything unneeded. Conversely, the server version of Ditto is a "Big Peer" that is a cluster designed for scalability and fault tolerance, so it wants to capture everything. Another way to put it is selfish vs. greedy replication.

To help users with eviction, we are adding support for dynamic queries, for example, using "now" for a date in the query to aid in creating TTL logic. In addition, to built-in TTL as well.

Satellite comms are definitely a great fit for another reason, since Ditto's replication is "delta-based" it will only sync differences. Our protocol is designed to overlay on top of custom arbitrary links, including unreliable ones. Not familiar with CCSDS, but we have explored Link16, and even built-in we replicate over BLE GATT.

Hope that helps and perhaps we can get Ditto into space!

[1] https://docs.ditto.live/ios/common/concepts/syncing-data#evi...

Re: Ditto: Real-time sync for apps even without the internet

#72

Post by jeff_tyrrill »

Animats wrote: post #69
bigfish24 wrote: post #67

Earlier quoted context omitted.

Adam (Ditto cofounder) - thanks for the feedback! I can see how this can create the wrong impression and will follow up with our counsel to improve the language. To be clear, we are not in the business of selling personal information. We are in the "picks and shovels" business - building great tools for other developers to use. Both myself and cofounder Max are app developers by background and we got into infrastruct…

You wrote your terms, which officially say you are in the business of selling personal information. Informal comments to the contrary have little contractual value.

This interpretation of the terms is straining it almost to the breaking point. The section you quoted says explicitly:

"We do not allow our third-party service providers to use your Personal Information for their own purposes and only permit them to process your Personal Information for specified purposes and in accordance with our instructions, unless the data is rendered fully anonymous."

You didn't quote this part.

To me, the most straightforward reading is that Ditto does their own advertising, and targets it to specific users by using third party advertisers (Ditto paying the advertisers to place an ad, not the advertisers paying Ditto for the information), who are contractually bound to only use that information to deliver the ad that Ditto placed and not to use the information for their own future benefit.

The only possible reading I can see where this can be interpreted as selling personal information is the fact that the "Advertising and marketing partners" bullet doesn't use the phrase "service providers", and that thus, the contractual restriction on "service providers" could thereby exempt that bullet. Is this what you are hanging your interpretation on?

(Reason for my interest: I regularly read privacy policies, looking for loopholes; there's a huge range not just in privacy practices but in how clearly they are described. Ditto's could be clearer - perhaps by including an explicit statement that they don't sell personal information, as some do - but Ditto's privacy policy is not what that of a company who sells personal information as part of their business model looks like.)

Re: Ditto: Real-time sync for apps even without the internet

#74

Post by mbalex99 »

brodouevencode wrote: post #29
mbalex99 wrote: post #25

Earlier quoted context omitted.

Max, CoFounder of Ditto here. Super cool to see us on HN! The use cases are for projects, products, enterprises to build collaborative apps that can sync with the internet but fall back to local P2P connections when internet connectivity fails. Furthermore we added offline first as an additional guard for some resiliency. We help solve a class of apps like: 1. Desk-less worker apps for the enterprise: think people wh…

We are in the process of buying it for retail applications (just signed the paperwork) and so far so good. I would ask that you make the documentation a little easier to find from the home page. Engineers want things to be easy ya know :). EDIT: is there ever a plan to open this as a spec?

Thanks!! We will make improvements.

Our replication system is very complex. I am not sure that even publishing our spec would get much usage because it’s so complex.

But maybe we can open source an API?

Re: Ditto: Real-time sync for apps even without the internet

#75

Post by mbalex99 »

dont__panic wrote: post #23
Rygian wrote: post #8

Earlier quoted context omitted.

My vision is that all the current use of IT should work in this mode. Depending on a central server for synchronization should be the exception, not the norm.

Interestingly, most of the Ditto folks originally worked on Realm, an object database that eventually monetized into the "sync via a central server" space. Of course, Realm was later acquired by MongoDB, and currently exists as some kind of Frankenstein's monster product in an attempt to drum up Atlas sales. Interesting how the acquisition pushed some of the devs to create better tech, but they had to give up the ori…

It’s just my cofounder Adam and I that came from Realm!

We’ve got Alumni from Pusher, Basho/Riak, Walmart Labs, Microsoft, to name a few

Re: Ditto: Real-time sync for apps even without the internet

#76

Post by mbalex99 »

jamil7 wrote: post #16
Rygian wrote: post #8

Earlier quoted context omitted.

My vision is that all the current use of IT should work in this mode. Depending on a central server for synchronization should be the exception, not the norm.

How would you handle granular permissions though?

We will be working on granular permissions as a top priority in 2023. We have a couple of ideas running around right now.

Re: Ditto: Real-time sync for apps even without the internet

#77

Post by mbalex99 »

zyklonix wrote: post #46

Great concept! @mbalex99 any plans to add a GraphQL interface to it? This would enable many existing platforms to "easily" migrate to a platform like this.

No unfortunately not in our core offering. We’d like to stick to what we do best and then leave it up to the community next year to build a graphql implementation.

Re: Ditto: Real-time sync for apps even without the internet

#78

Post by mbalex99 »

solarkraft wrote: post #57

This looks cool. Are there comparable open source projects? How do Y.js ( https://yjs.dev ) and Gun ( https://gun.js.org ) compare?

We have a mesh network that syncs devices using P2P wifi, BLE and over a LAN as well!

I think this is our biggest differentiator compared to other sync systems.

Re: Ditto: Real-time sync for apps even without the internet

#79

Post by techdragon »

bigfish24 wrote: post #71
techdragon wrote: post #70

Might seem like odd questions, but is there a plan to handle the "too much data" case? and how protocol agnostic can this be? This seems really cool, and I can think of some strait forward uses for it ... but working on a satellite project where communications are 75% of the hassle, this sort of thing seems like it could be very useful... If it can deal with all the "weird" (its very normal for space, just not used a…

Adam (cofounder) - great question! Yes, this situation is very common for our use-cases. Ditto supports the concept of `eviction` [1]. For example, imagine a restaurant that wants to store all of today's orders in their devices, but then evict the data when they close out at end of day (and data is synced into Cloud backend). Eviction is a core concept because we consider devices using Ditto's SDK a "Small peer" that…

Thanks for the reply! I had a very quick look at the documentation and didn’t see the eviction information. That’s excellent to know it’s already a built in capability. As for space, don’t worry if you’ve never heard of CCSDS, it’s just yet another standards group for various things, the key thing is that it’s delta based and already designed to handle non-TCP-IP based data transfer mechanisms. One of the big challenges I’ve found with bringing Linux and other modern software into an environment where “normal” (you can get a 24/7 IP link, but it’s unusual for satellite to maintain an open link like that) space communication is that a lot of useful modern open source tools is just not designed for a non IP based networking environment, so much so that you dig into things like delay tolerant networks and you’ll find mechanisms for trying to fake TCP-IP over the top of them just to help accomodate this software. Having a mechanism to replicate important data on the basis of just a transmission mechanism agnostic “packet” of delta update data, would be a huge boon for low cost satellite systems.

Im definitely going to have to dig in deeper and check out the rust stack! I’ll be sure to get in touch about the various CCSDS data transmission related stuff once I get there, since if you’ve looked at Link16 I’m sure you’re probably interested in adding more aerospace communications link standards/mechanisms.

Re: Ditto: Real-time sync for apps even without the internet

#80

Post by Flimm »

jeff_tyrrill wrote: post #72
Animats wrote: post #69

Earlier quoted context omitted.

You wrote your terms, which officially say you are in the business of selling personal information. Informal comments to the contrary have little contractual value.

This interpretation of the terms is straining it almost to the breaking point. The section you quoted says explicitly: "We do not allow our third-party service providers to use your Personal Information for their own purposes and only permit them to process your Personal Information for specified purposes and in accordance with our instructions, unless the data is rendered fully anonymous." You didn't quote this part…

Thank you for spending time reading privacy policies. I'm glad someone is looking at them. What keeps you motivated to do this (boring) work?
Post reply on HN