Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CDC File Transfer

github.com

Post reply on HN
Print view View original

211–220 of 229 posts

Re: CDC File Transfer

#211

Post by _ea1k »

capableweb wrote: post #164
_ea1k wrote: post #82

Looks interesting, but when did we start saying "synching" instead of "syncing"? I mean, its literally based on the idea behind rsync. Why the sudden need to add an "h"?

If by "we" you mean the internet, it seems most people still use "syncing" rather than "synching": https://trends.google.com/trends/explore?date=all&q=syncing,... Still, "syncing" is supposed to be shorthand for "synchronizing", which has the "h" you feel comes out of nowhere. So I guess both makes sense, but I don't use that form myself nor have seen anyone else use it in the wild.

Repo description: "Tools for synching and streaming files from Windows to Linux"

It then goes on to talk about how much like rsync their new "cdc_rsync" tool is.

I get that the long version uses an h, but why include the silent h when making a shorthand? It sort of defeats the purpose of making a shorthand.

Re: CDC File Transfer

#213

Post by hot_gril »

ssth wrote: post #180
hot_gril wrote: post #70

> Born from the ashes of Stadia Love it. Those were some very expensive ashes. I hope more comes from them too.

We can feel developer's bitterness from that sentence alone.

Stadia did impact my work a little, but not enough to matter to me. I'd be upset if I worked on it directly.

Re: CDC File Transfer

#214

Post by alixanderwang »

skies457 wrote: post #158
c7b wrote: post #130

Slightly OT, but I like the schematic gifs used in the Readme.md (pretty amazing doc overall!) like this one [0]. Does anyone have suggestions what tools they might have used (or might be used in general) to create those? [0] https://github.com/google/cdc-file-transfer/blob/main/docs/l...

D2 ( https://d2lang.com/tour/intro/ ) might be the tool you are looking for, but it doesn't handle animation as far as I know

Coming very soon!

Re: CDC File Transfer

#215

Post by paravz »

linsomniac wrote: post #53
paravz wrote: post #35

sounds like cdc_rsync should be a good replacement for rsync in a generic backup

Imagine an rsync that was A LOT smarter for backup use cases. Detecting files that have been renamed, files that have been compressed, similar files, files that have only been appended to, files that are the same across multiple systems (Oh, yeah, I probably already have a copy of this kernel file from this other host). Basically, deduplication as part of rsync.

Exactly what I'm looking for recently.. I dismissed ZFS (not mature on Linux) and btrfs (seems to be too complex and buggy as seen in a few horror stories)

So far I started using --link-dest for rsync, as explained in https://lincolnloop.com/insights/detecting-file-moves-rename... and used in https://github.com/dparoli/hrsync/blob/master/hrsync#L52

Re: CDC File Transfer

#216

Post by skanga »

ljusten wrote: post #175
skanga wrote: post #52

Earlier quoted context omitted.

Thanks! Does it work Windows to Windows also?

Windows to Windows is being worked on, see https://github.com/google/cdc-file-transfer/compare/main...s... . Linux to Linux is also an option if there is demand, but currently it's Windows to Linux only.

That's great. Thanks!

Re: CDC File Transfer

#217

Post by tga_d »

tmchow wrote: post #146

1. I thought this was going to be something about the Center for Disease Control 2. This is a really neat project, and super expensive ashes to rise from Stadia. 3. This is targeted at Windows to Linux, but given the speed advantages that it has over rsync couldn’t this be Linux to Linux? That said, I’d be afraid to use this over rsync just from the body of experience and knowledge that exists about rsync.

I was thinking something similar to your third point, though more for cdc_stream, since as of last year sshfs is no longer maintained anyway.

Re: CDC File Transfer

#218

Post by ketralnis »

klabb3 wrote: post #91
ketralnis wrote: post #72

Earlier quoted context omitted.

That's an interesting extension of the illegal numbers or coloured bits theories, but we don't really see it used that way in practise. When governments or media industry groups crack down on this stuff, they don't go after everybody that ever had those bits in memory. Maybe that's just for practical reasons, but we've never seen every router in between a buyer and seller get confiscated too as they've been somehow t…

I’m not suggesting the hashes themselves are illegal to possess, but that transferring the bytes corresponding to those hashes is problematic: if both sides are lowly trusted, that puts you at risk as a hoster of that content. This is indeed an issue with IPFS, for instance, where I believe the solutions are “pinning” content that is already vetted by another party, or denylists of “bad bits”. I assume it’s similar t…

It's not the illegalness I'm challenging, it's the problematicness. Maybe it is illegal to even think about those bit patterns. But I'm not aware of cases where people get _actually_ thrown in jail or fined for possessing or transmitting them. In all of the cases I know about there is intent involved.

Re: CDC File Transfer

#220

Post by manigandham »

raverbashing wrote: post #135
manigandham wrote: post #103

Earlier quoted context omitted.

What does this have to do with TCP being reliable?

If TCP was reliable enough, then these chuncked transfers wouldn't need to be used

These chunked transfers are used to efficiently sync only the changed parts of the file. The actual network traffic is still over TCP. You're conflating several layers and abstractions.
Post reply on HN