181–190 of 229 posts
Earlier quoted context omitted.
And linux to linux?
We also ran the experiment with the native Linux rsync, i.e syncing Linux to Linux, to rule out issues with Cygwin. Linux rsync performed on average 35% worse than Cygwin rsync, which can be attributed to CPU differences. Source: https://github.com/google/cdc-file-transfer
Earlier quoted context omitted.
That's an interesting extension of the illegal numbers or coloured bits theories, but we don't really see it used that way in practise. When governments or media industry groups crack down on this stuff, they don't go after everybody that ever had those bits in memory. Maybe that's just for practical reasons, but we've never seen every router in between a buyer and seller get confiscated too as they've been somehow t…
I’m not suggesting the hashes themselves are illegal to possess, but that transferring the bytes corresponding to those hashes is problematic: if both sides are lowly trusted, that puts you at risk as a hoster of that content. This is indeed an issue with IPFS, for instance, where I believe the solutions are “pinning” content that is already vetted by another party, or denylists of “bad bits”. I assume it’s similar t…
This might be a dumb question.. but when I read this "scp always copies full files, there is no "delta mode" to copy only the things that changed, it is slow for many small files, and there is no fast compression." my thinking was: If you want to send diffs.. why not just use git? It does compression and chunks and all that. Maybe the defaults perform poorly for binary files? But couldn't you fix that with a custom d…
They are separate problems and not really comparable, git is a higher level tool. The diffing in git is for humans to compare code, and as such it is actually computed on the fly and not stored anywhere (unless you use git patch). Similarly, git does not provide any mechanism for sending/receiving files, it re-uses existing solutions like HTTP and SSH. This tool exists to send/receive files, and the diffing is an imp…
I don't know if git sends or receives deltas though, I think it does?
Earlier quoted context omitted.
But why not make cross platform? If already creating a file transfer program, make it cross platform. What're the complications?
Short answer, we didn't need it. While the code is largely cross-platform, there is some work involved when it gets down to the details. We are currently working on supporting Windows to Windows. Linux to Linux has lower priority as rsync already provides all functionality, it's just a bit slower on fast connections. On slow connections, rsync and cdc_rsync perform very similarly as the sync speed is dominated by the…
Slightly OT, but I like the schematic gifs used in the Readme.md (pretty amazing doc overall!) like this one [0]. Does anyone have suggestions what tools they might have used (or might be used in general) to create those? [0] https://github.com/google/cdc-file-transfer/blob/main/docs/l...
ImageMagick does the trick, for example `convert -delay 20 -loop 0 *.png out.gif`
Earlier quoted context omitted.
They are separate problems and not really comparable, git is a higher level tool. The diffing in git is for humans to compare code, and as such it is actually computed on the fly and not stored anywhere (unless you use git patch). Similarly, git does not provide any mechanism for sending/receiving files, it re-uses existing solutions like HTTP and SSH. This tool exists to send/receive files, and the diffing is an imp…
Oh right, git doesn't store diffs. So each time you push/pull, it fetches/uploads the whole file.. for every commit? Damn.. Then I guess even using this under the hood would be essentially a rewrite or how git fundamentally works internally
Earlier quoted context omitted.
That's an interesting extension of the illegal numbers or coloured bits theories, but we don't really see it used that way in practise. When governments or media industry groups crack down on this stuff, they don't go after everybody that ever had those bits in memory. Maybe that's just for practical reasons, but we've never seen every router in between a buyer and seller get confiscated too as they've been somehow t…
I’m not suggesting the hashes themselves are illegal to possess, but that transferring the bytes corresponding to those hashes is problematic: if both sides are lowly trusted, that puts you at risk as a hoster of that content. This is indeed an issue with IPFS, for instance, where I believe the solutions are “pinning” content that is already vetted by another party, or denylists of “bad bits”. I assume it’s similar t…
Files become piracy when there is evidence of intentional copyright infringement, for example when the chunk is part of a valid MPEG4 file and the MPEG4 file is titled "Wednesday_S2E4_FullHD_NetflixRip.MP4"
Earlier quoted context omitted.
They are separate problems and not really comparable, git is a higher level tool. The diffing in git is for humans to compare code, and as such it is actually computed on the fly and not stored anywhere (unless you use git patch). Similarly, git does not provide any mechanism for sending/receiving files, it re-uses existing solutions like HTTP and SSH. This tool exists to send/receive files, and the diffing is an imp…
Oh right, git doesn't store diffs. So each time you push/pull, it fetches/uploads the whole file.. for every commit? Damn.. Then I guess even using this under the hood would be essentially a rewrite or how git fundamentally works internally
It's kind of weird actually - at the architectural level we interact with, we talk in terms of diffs - both in terms of display and what we put in.
At the next level down (content addressable store) git is storing whole files, and the git tooling translates the diffs we communicate about down into whole files for each commit.
Then at the next level down, git puts files together in packfiles (when the repo is packed) which is a compression system to make use of the fact that most files are just tweaks of other files. So, once again it's diffs.