CDC File Transfer
github.com
CDC File Transfer
1–10 of 106 posts
Re: CDC File Transfer
#2Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.
Re: CDC File Transfer
#3This CDC is "Content Defined Chunking" - fast incremental file transfer.
Use case is to copy file over slow net, but the previous version is already there, so one can save time by only sending changed parts of the file.
Not to be confused with USB CDC ("communications device class"), an USB device protocol used to present serial ports and network cards. It can also be used to transfer files, the old PC-to-PC cables used it by implementing two network cards connected to each other.
Re: CDC File Transfer
#4Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.
> it’s just piracy in today’s drm world
...which is more important / needed than ever. I encourage every who asks to get my music from bit torrent instead of spotify.
Re: CDC File Transfer
#5Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.
What do you mean piracy in the a DRM world. Like being able to share your own PC games through the cloud?
Re: CDC File Transfer
#6Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.
for self-hosted game streaming you can use moonlight + sunshine, they work really well in my experience.
Re: CDC File Transfer
#7This CDC is "Content Defined Chunking" - fast incremental file transfer. Use case is to copy file over slow net, but the previous version is already there, so one can save time by only sending changed parts of the file. Not to be confused with USB CDC ("communications device class"), an USB device protocol used to present serial ports and network cards. It can also be used to transfer files, the old PC-to-PC cables u…
The clever trick is how it recognizes insertions. The standard trick of computing hashes on fixed sized blocks works efficiently for substitutions but is totally defeated by an insertion or deletion.
Instead with CDC the block boundaries are define by the content, so an insertion doesn’t change the block boundary, so it can tell the subsequent blocks are unchanged. I haven’t read the CDC paper but I’m guessing they just use some probabilistic hash function to define certain strings as block boundaries.
Re: CDC File Transfer
#8Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.
> it’s just piracy in today’s drm world ...which is more important / needed than ever. I encourage every who asks to get my music from bit torrent instead of spotify.
Why not something like Bandcamp, or other DRM-free purchase options?
I'm not above piracy if there's no DRM free option (or if the music is very old or the artist is long dead), but I still believe in supporting artists who actively support freedom.
Re: CDC File Transfer
#9This CDC is "Content Defined Chunking" - fast incremental file transfer. Use case is to copy file over slow net, but the previous version is already there, so one can save time by only sending changed parts of the file. Not to be confused with USB CDC ("communications device class"), an USB device protocol used to present serial ports and network cards. It can also be used to transfer files, the old PC-to-PC cables u…
The clever trick is how it recognizes insertions. The standard trick of computing hashes on fixed sized blocks works efficiently for substitutions but is totally defeated by an insertion or deletion. Instead with CDC the block boundaries are define by the content, so an insertion doesn’t change the block boundary, so it can tell the subsequent blocks are unchanged. I haven’t read the CDC paper but I’m guessing they j…
Probably worth noting that ordinary rsync can also handle insertions/deletions because it uses a rolling hash. Rsync's method is bandwidth-efficient, but not especially CPU-efficient.
Re: CDC File Transfer
#10Earlier quoted context omitted.
> it’s just piracy in today’s drm world ...which is more important / needed than ever. I encourage every who asks to get my music from bit torrent instead of spotify.
Why not something like Bandcamp, or other DRM-free purchase options? I'm not above piracy if there's no DRM free option (or if the music is very old or the artist is long dead), but I still believe in supporting artists who actively support freedom.
Yep, I put everything on bandcamp. https://justinholmes.bandcamp.com/
Even better though, is a P2P service that is censorship resistant.
But yeah I like Bandcamp plenty.
> artists who actively support freedom.
The bluegrass world is quickly becoming this.