Live data from Hacker News

YouTubeDrive: Store files as YouTube videos

github.com

171–180 of 316 posts

Re: YouTubeDrive: Store files as YouTube videos

#172
post #59

Years ago when Amazon had unlimited photo storage, you could “hide” gigabytes of data behind a 1px gif (literally concatenation together) so that it wouldn’t count against your quota.

They still do if you pay for Prime. I was surprised to see that even RAW files (which are uncompressed and quite large) were uploaded and stored with no issues. Not the same as "hiding" data but might still be possible.

In the interest of technical correctness, RAW files are frequently compressed and even lossily compressed. For example, Sony's RAW compression was only lossy until very recent cameras.

Given that there are the options for uncompressed, lossy compressed and lossless compressed, I'd say RAW files differ in the stage of the data processing where capture is being done and doesn't imply anything about the type of compression.

What is relevant is that the formats vary widely between manufacturers, camera lines and individual cameras, so unlike JPEG, it's really hard to create a storage service that compresses RAW files further after uploading in a meaningful way. So anything they do needs to losslessly compress the file.

Re: YouTubeDrive: Store files as YouTube videos

#174
post #166

I love that this is like tape in that it's a sequential access medium. It's storing a tape-like data stream in a digital version of what used to be tape itself (VHS).

I believe YouTube supports random access, or otherwise you wouldn’t be able to jump around in a video. Youtube-dl also supports resuming downloads in the middle, I believe.

True... But guessing YouTubeDrive 'decoder' needs whole video to get you back anything close to what you put in.

Otherwise each frame would have to have a ridiculous amount of encoded overhead.

Ahh, NM cant even see that working.

edit: Maybe a file table at built from from specified first N frames, that delivers frameset/file map ...

Still nothing like skipping spots in a video. That relies on key frames and time signatures.

Cool stuff nonetheless...

Re: YouTubeDrive: Store files as YouTube videos

#176
post #117
post #18

How much data can you store if you embedded a picture-in-picture file over a 10 minute video? I could totally see content creators who do tutorials embedding project files in this way.

Back of the envelope estimate: 4096 x 2160 x 24 x 60 is your theoretical max in bits/second, 127 billion. Assume that to counter YouTube's compression we need 16x16 blocks of no more than 256 colors and 15 keyframes/second; that reduces it to 256 * 135 * 8 * 15 = 4.1 million bits/sec. That's not too awful. Ten minutes of this would get you about 300MB of data, which itself might be compressed.

4k video is almost always 3840x2160

Re: YouTubeDrive: Store files as YouTube videos

#177
post #124
post #123

Earlier quoted context omitted.

YT might still recompress your video, possibly using proprietary algorithms that are not necessarily DCT based

As said, falls apart with re-encoding. But is a bit more interesting than what is more or less QR codes.

I find it a bit more interesting to have something that actually works on youtube, even if only as a proof of concept.

Re: YouTubeDrive: Store files as YouTube videos

#179
post #118

I only looked at the example video, but is the concept just "big enough pixels"? Would be neater (and much more efficient) to encode the data such that it's exactly untouched by the compression algorithm, e.g. by encoding the data in wavelets and possibly motion vectors that the algorithm is known to keep[1]. Of course that would also be a lot of work, and likely fall apart once the video is re-encoded. [1] If that's…

YouTube let’s you download your uploaded videos. I’ve never tested it, but supposedly it’s the exact same file you uploaded.[a] It probably wouldn’t work with this “tool” as it uses the video ID (so I assume it’s downloading what clients see, not the source), but it’s an idea for some other variation on this concept.

[a] That way, in the future, if there’s any improvements to the transcode process that makes smaller files (different codec or whatever), they still have the HQ source

Re: YouTubeDrive: Store files as YouTube videos

#180
post #118

I only looked at the example video, but is the concept just "big enough pixels"? Would be neater (and much more efficient) to encode the data such that it's exactly untouched by the compression algorithm, e.g. by encoding the data in wavelets and possibly motion vectors that the algorithm is known to keep[1]. Of course that would also be a lot of work, and likely fall apart once the video is re-encoded. [1] If that's…

Agree it would be cool to be "untouched" by the compression algorithm, but that's nearly impossible with YouTube. YouTube encodes down to several different versions of a video and on top of that, several different codecs to support different devices with different built-in video hardware decoders. For example, when I upload a 4K vid and then watch the 4K stream on my Mac vs my PC, I get different video files solely b…

What if you have an ML model that produces a vector from a given image. You have a set of vectors that correspond to bytes - for a simple example you have 256 "anchor vectors" that correspond to any possible byte.

To compress data an arbitrary sequence of bytes, for each byte, you produce an image that your ML model would convert to the corresponding anchor vector for that byte and add the image as a frame in a video. Once all the bytes have been converted to frames you then upload the video to YouTube.

To decompress the video you simply go frame by frame over the video and send it to your model. Your model produces a vector and you find which of your anchor vectors is the nearest match. Even though YouTube will have compressed the video in who knows what way, and even if YouTube's compression changes, the resultant images in the video should look similar, and if your anchors are well chosen and your model works well, you should be able to tell which anchor a given image is intended to correspond to.

Post reply on HN