Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YouTubeDrive: Store files as YouTube videos

github.com

Post reply on HN
Print view View original

191–200 of 316 posts

Re: YouTubeDrive: Store files as YouTube videos

#191

Post by mjochim »

qorrect wrote: post #186
marginalia_nu wrote: post #112

Earlier quoted context omitted.

This was a custom email server though, there never were any emails, it just presented files as though they were so that a client would download them. Actually caused some problems for email clients, as they usually assumed emails were small. I got a few of them to crash with 200 Mb "attachments" (although this was in the early 00s, 200Mb was bigger than it is today).

I'm still confused on how this worked, did you email some address and get a reply with the attachment ?

Since GP says it was a POP3 server, I suppose you would set up an email account in your client with its inbox server pointing to that POP3 server. When the client requests the content of the inbox, the server responds with a list of "emails" that are really just files with some email header slapped on; so your email client's inbox window essentially becomes a file browser.

Re: YouTubeDrive: Store files as YouTube videos

#192

Post by RcouF1uZ4gsC »

adzm wrote: post #77
itake wrote: post #63

Earlier quoted context omitted.

Back in the day when @gmail was famous for their massive free storage for email, ppl wrote scripts to chunk large files and store them as email attachments.

People did this on AOL in the 90s as well!

With AOL, in the early 90’s you didn’t even need to do that. You could just reformat and reuse the floppy disks they were always sending you for free storage.

Re: YouTubeDrive: Store files as YouTube videos

#193

Post by Gigachad »

behnamoh wrote: post #109
marginalia_nu wrote: post #75

Earlier quoted context omitted.

This is a tangent. I must have been maybe 15-16 at the time, so somewhere around 20 years ago: One of the first pieces of software I remember building was a POP3 server that served files, that you could download using an email client where they would show up as attachments. Incredibly bizarre idea. I'm not sure who I thought would benefit from this. I guess I got swept up in RFC1939 and needed to build... something.

apparently e-mail is not much reliable for storing/keeping files. there have been cases where an old email with an attachment would not load correctly because the servers just erased the attachment file.

Interestingly, if you take a look at your emails from a few years ago, most of the non attached images will fail to load now.

Re: YouTubeDrive: Store files as YouTube videos

#194

Post by pbhjpbhj »

javajosh wrote: post #72
throwaway92394 wrote: post #43

Earlier quoted context omitted.

Just gotta add some good 'ol steganography

This brings up an interesting question: what is the upper-bound of hidden data density using video steganography? E.g. how much extra data can you add before noticeable degradation? It's interesting because it requires both a detailed understanding of video encoding and also understanding of human perception of video.

I'd expect you could store more data steganographically than the raw video data.

You can probably do things like add frames that can't be decoded and so are skipped by a decoder; that effectively allows arbitrary added hidden data. That's maybe cheating.

If you stipulate that you can't already have a copy of the unaltered file, and the data has to be extractable from a pixel copy of the rendered frames ... that becomes more interesting, I think.

Re: YouTubeDrive: Store files as YouTube videos

#195

Post by rasguanabana »

ALittleLight wrote: post #180
softfalcon wrote: post #141

Earlier quoted context omitted.

Agree it would be cool to be "untouched" by the compression algorithm, but that's nearly impossible with YouTube. YouTube encodes down to several different versions of a video and on top of that, several different codecs to support different devices with different built-in video hardware decoders. For example, when I upload a 4K vid and then watch the 4K stream on my Mac vs my PC, I get different video files solely b…

What if you have an ML model that produces a vector from a given image. You have a set of vectors that correspond to bytes - for a simple example you have 256 "anchor vectors" that correspond to any possible byte. To compress data an arbitrary sequence of bytes, for each byte, you produce an image that your ML model would convert to the corresponding anchor vector for that byte and add the image as a frame in a video…

Why go that way. I’m no digital signal processing expert, but images (and series thereof, i.e videos) are 2D signals. What we see is spatial domain and analyzing pixel by pixel is naive and won’t get you very far.

What you need is going to frequency domain. From my own experiment in university times most significant image info lays in lowest frequencies. Cutting off frequencies higher than 10% of lowest leaves very comprehensible image with only wavey artifacts around objects. You have plenty of bandwidth to use even if you want to embed info in existing media.

Now here you have full bandwidth to use. Start with frequency domain, set expectations of lowest bandwidth you’ll allow and set the coefficients of harmonic components. Convert to spatial domain, upscale and you got your video to upload. This should leave you with data encoded in a way that should survive compression and resizing. You’ll just need to allow some room for that.

You could slap error correction codes on top.

If you think about it, you should consider video as - say - copper wire or radio. We’ve come quite far transmitting over these media without ML.

Re: YouTubeDrive: Store files as YouTube videos

#196

Post by hiq »

babanin wrote: post #148
marginalia_nu wrote: post #75

Earlier quoted context omitted.

This is a tangent. I must have been maybe 15-16 at the time, so somewhere around 20 years ago: One of the first pieces of software I remember building was a POP3 server that served files, that you could download using an email client where they would show up as attachments. Incredibly bizarre idea. I'm not sure who I thought would benefit from this. I guess I got swept up in RFC1939 and needed to build... something.

On my first job (in the beginning of the millennium) there was a limit on files you could download, something around 5Mb. If you wanted to download something bigger, you had to ask sysadmins to do that and wait... That was really annoying. So I and my colleague end up writing a service, that could download a file to local storage and chop it into multiple 5Mb attachments and send multiple emails to requestor. After s…

I couldn't download .exe files at some $CORPORATION. They had to be whitelisted or something, and the download just wouldn't work otherwise. But once you had the .exe you could run it just fine. You just had to ping some IT person to be able to retrieve your .exe.

Of course it was still possible to browse the internet and visualize arbitrary text, so splitting the .exe into base64-encoded chunks and uploading them on GitHub from another computer was working perfectly fine... I briefly argued against these measures, given how unlikely they are to prevent any kind of threat, but they're probably still in place.

Re: YouTubeDrive: Store files as YouTube videos

#197

Post by pbhjpbhj »

jklinger410 wrote: post #39
throwaway0a5e wrote: post #6

Earlier quoted context omitted.

>Is it really abuse if the videos are viewable / playable? Presumably the ToS either already forbids covert channel encoding or soon will. If creators start encoding their source and material into their content Google would probably be fine with that because it gives them data but also gives them context for that data. Edit: I meant like "director's commentary" and "notes about production" type stuff like you used to…

> If creators start encoding their source material into their files Google would probably be fine with that Not true at all, lol. Google has a paid file storage solution. YouTube is for streaming video and that's the activity they expect on that platform. I couldn't imagine any service designed for one format would "probably be fine" with users encoding other files inside of that format.

I think the parent comment is limiting themselves to the embedding of metadata specific to the containing file. It would be like adding a single frame, but would potentially give useful information to Google. In those limited circumstances I think the parent is correct.

Re: YouTubeDrive: Store files as YouTube videos

#199

Post by anyfoo »

ALittleLight wrote: post #180
softfalcon wrote: post #141

Earlier quoted context omitted.

Agree it would be cool to be "untouched" by the compression algorithm, but that's nearly impossible with YouTube. YouTube encodes down to several different versions of a video and on top of that, several different codecs to support different devices with different built-in video hardware decoders. For example, when I upload a 4K vid and then watch the 4K stream on my Mac vs my PC, I get different video files solely b…

What if you have an ML model that produces a vector from a given image. You have a set of vectors that correspond to bytes - for a simple example you have 256 "anchor vectors" that correspond to any possible byte. To compress data an arbitrary sequence of bytes, for each byte, you produce an image that your ML model would convert to the corresponding anchor vector for that byte and add the image as a frame in a video…

That's essentially a variant of "bigger pixels". Just like them, your algorithm cannot guarantee that an unknown codec will still make the whole thing perform adequately.

Even if you train your model to work best for all existing codecs (I assume that's the "ML" part of the ML model), the no free lunch theorem pretty much tells us that it can't always perform well for codecs it does not know about.

(And so does entropy. Reducing to absurd levels, if your codec results in only one pixel and the only color that pixel can have is blue, then you'll only be able to encode any information in the length of the video itself.)

Re: YouTubeDrive: Store files as YouTube videos

#200

Post by anyfoo »

rasguanabana wrote: post #195
ALittleLight wrote: post #180

Earlier quoted context omitted.

What if you have an ML model that produces a vector from a given image. You have a set of vectors that correspond to bytes - for a simple example you have 256 "anchor vectors" that correspond to any possible byte. To compress data an arbitrary sequence of bytes, for each byte, you produce an image that your ML model would convert to the corresponding anchor vector for that byte and add the image as a frame in a video…

Why go that way. I’m no digital signal processing expert, but images (and series thereof, i.e videos) are 2D signals. What we see is spatial domain and analyzing pixel by pixel is naive and won’t get you very far. What you need is going to frequency domain. From my own experiment in university times most significant image info lays in lowest frequencies. Cutting off frequencies higher than 10% of lowest leaves very c…

We started with that approach, by assuming that the compression is wavelet based, and then purposefully generating wavelets that we know survive the compression process.

For the sake of this discussion, wavelets are pretty much exactly that: A bunch of frequencies where the "least important" (according to the algorithm) are cut out.

But that's pretty cool, seems like you've re-invented JPEG without knowing it, so your understanding is solid!

Post reply on HN