Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CDC File Transfer

github.com

Post reply on HN
Print view View original

21–30 of 106 posts

Re: CDC File Transfer

#21

Post by jakebasile »

sheepscreek wrote: post #20
rekttrader wrote: post #2

Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.

Probably wouldn’t have been feasible - I heard developers had to compile their games with Stadia support. Maybe it was an entirely different platform, with its own alternative to DirectX, or maybe had some kind of lightweight emulation (such as Proton) but I remember vaguely the few games I played had custom stadia key bindings (with stadia symbols). They would display like that within the game. So definitely some cu…

As I understand it, GeForce Now actually does require changes to the game to run in the standard and until recently only option of "Ready To Play". This is the supposed reason that new updates to games sometimes take time to get released on the service, since either the developers themselves or Nvidia needs to modify it to work correctly on the service. I have no idea if this is true, but it makes sense to me.

They recently added "Install to Play" where you can install games from Steam that aren't modified for the service. They charge for storage for this though.

Sadly, there's still tons of games unavaiable because publishers need to opt in and many don't.

Re: CDC File Transfer

#22

Post by ur-whale »

Great initiative, especially the new sync algorithm, but giant hurdles to adoption:

- only works on a weird combo of (src platform / dst platform). Why???? How hard is it to write platform-independent code to read/write bytes and send them over the wire in 2025?

- uses bazel, an enormous, Java-based abomination, to build.

Fingers crossed that these can be fixed, or this project is dead in the water.

Re: CDC File Transfer

#23

Post by EdSchouten »

I’ve also been doing lots of experimenting with Content Defined Chunking since last year (for https://bonanza.build/). One of the things I discovered is that the most commonly used algorithm FastCDC (also used by this project) can be improved significantly by looking ahead. An implementation of that can be found here:

https://github.com/buildbarn/go-cdc

Re: CDC File Transfer

#24

Post by MindSpunk »

sheepscreek wrote: post #20
rekttrader wrote: post #2

Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.

Probably wouldn’t have been feasible - I heard developers had to compile their games with Stadia support. Maybe it was an entirely different platform, with its own alternative to DirectX, or maybe had some kind of lightweight emulation (such as Proton) but I remember vaguely the few games I played had custom stadia key bindings (with stadia symbols). They would display like that within the game. So definitely some cu…

Stadia games were just run on Linux with Vulkan + some extra Stadia APIs for their custom swapchain and other bits and pieces. Stadia games were basically just Linux builds.

Re: CDC File Transfer

#26

Post by MaxikCZ »

jMyles wrote: post #4
rekttrader wrote: post #2

Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.

> it’s just piracy in today’s drm world ...which is more important / needed than ever. I encourage every who asks to get my music from bit torrent instead of spotify.

So you create and seed your torrents with your music, and present them prominently on your site?

Re: CDC File Transfer

#27

Post by Scaevolus »

EdSchouten wrote: post #23

I’ve also been doing lots of experimenting with Content Defined Chunking since last year (for https://bonanza.build/ ). One of the things I discovered is that the most commonly used algorithm FastCDC (also used by this project) can be improved significantly by looking ahead. An implementation of that can be found here: https://github.com/buildbarn/go-cdc

This lookahead is very similar to the "lazy matching" used in Lempel-Ziv compressors! https://fastcompression.blogspot.com/2010/12/parsing-level-1...

Did you compare it to Buzhash? I assume gearhash is faster given the simpler per iteration structure. (also, rand/v2's seeded generators might be better for gear init than mt19937)

Re: CDC File Transfer

#28

Post by laidoffamazon »

As I've gotten further in my career I've started to wonder - how many engineering quarters did it take to build this for their customers? How did they manage to get this on their own roadmap? This seems like a lot of code surface area for a fairly minimal optimization that would be redundant with a different development substrate (like running Windows on Stadia like how Amazon Luna worked...)

Re: CDC File Transfer

#29

Post by laidoffamazon »

rekttrader wrote: post #2

Nice to see Stadia had some long term benefit. It’s a shame they don’t make a self hosted version but if you did that it’s just piracy in today’s drm world.

Stadia was sadly engineered in such a way that this is impossible.

Speaking of which, who thought up the idea to use custom hardware for this that would _already be obsolete_ a year later? Who considered using Linux native instead of a compat layer? Why did the original Stadia website not even have a search bar??

Re: CDC File Transfer

#30

Post by AnonC »

Does anyone know if there’s work being done to integrate this into the standard rsync tool (even as an optional feature)? It seems like a very useful improvement that ought to be available widely. From this website it seems a bit disappointing that it’s not even available for Linux to Linux transfers.
Post reply on HN