71–80 of 107 posts
By hand colorization: http://www.marinamaral.com/portfolio-2/
> And yes, I'm definitely interested in doing video As someone familiar with the libraries space, I'd actually be very interested in seeing a machine learning model that could deal with "cleaning up" old film (I've actually brought this up w/ several of my ML friends occasionally). One of the biggest challenges in the world of media preservation is migrating analogue content to digital media before physical deteriora…
I believe that Peter Jackson's recent endeavour in cleaning up WW1 footage employs significant ML for de-noising, frame interpolation, and colorising. I haven't seen the final film, but some of the clips are staggeringly good: https://www.bbc.com/news/av/entertainment-arts-45884501/pete... Edit: Here's maybe a better link -- https://www.bbc.com/news/av/entertainment-arts-45803977/pete...
Maybe another cool avenue to explore would be combining models like this with some NLP approach that parses a historian's rough description of how the scene should be colored and biases the generator with prior information that way. (Maybe related to visual question answering or something.)
> The model loves blue clothing. Not quite sure what the answer is yet, but I'll be on the lookout for a solution!
Just throwing a thought out here that you might have considered, but, maybe it's because traditional black-and-white film is over-sensitive to blue? It's why when one uses traditional black and white films one usually uses at least a yellow filter and if you have blue sky in a shot you use a red filter. This may or may not be useful; either way, keep up the awesome work!
Earlier quoted context omitted.
You probably need an enormous GPU (24GB RAM) as well to make as large model as possible for as good generalization as you can (there are so many different types of objects/surfaces/fabric and their compositions).
Something amusing about needing a ridiculously large model to claim good generalization. Analytical models typically go the other way, right?
From the article:
"BEEFY Graphics card. I'd really like to have more memory than the 11 GB in my GeForce 1080TI (11GB). You'll have a tough time with less. The Unet and Critic are ridiculously large but honestly I just kept getting better results the bigger I made them."
When I saw "restoring" in the title I was expecting higher resolution. For example seeing in modern photo level detail eyelashes, wrinkles, etc. I get that, like the colors, this would require the adding lots of made up information about scene and feature details but IMO it would blur the lines between restoration and reconstruction/storytelling in a really awesome way. Old photos are cool in their own way but their…
There reminds me of a recent 99% Invisible episode [1] in which they discuss the same topic in the world of dinosaurs. It details how dinosaurs used to be depicted with the goal of only showing the things that we are confident in being true (although what we are confident in obviously changes over time). This results in mostly just greenish-brown skin draped over a muscle structure attached to the fossilized skeleton…
> And yes, I'm definitely interested in doing video As someone familiar with the libraries space, I'd actually be very interested in seeing a machine learning model that could deal with "cleaning up" old film (I've actually brought this up w/ several of my ML friends occasionally). One of the biggest challenges in the world of media preservation is migrating analogue content to digital media before physical deteriora…
I believe that Peter Jackson's recent endeavour in cleaning up WW1 footage employs significant ML for de-noising, frame interpolation, and colorising. I haven't seen the final film, but some of the clips are staggeringly good: https://www.bbc.com/news/av/entertainment-arts-45884501/pete... Edit: Here's maybe a better link -- https://www.bbc.com/news/av/entertainment-arts-45803977/pete...
I'd love to hear otherwise but I'm not aware of any commercial "machine learning" for post-production aside from the Nvidia Optix denoiser and one early beta of an image segmentation plugin.
Earlier quoted context omitted.
Something amusing about needing a ridiculously large model to claim good generalization. Analytical models typically go the other way, right?
It's Deep Learning, not much to do with any analytical model, it's not thinking like a human :-(. Recently even good NLP processing needs 24GB+ for training (won't fit into 16GB), a good quality colorizing (no spills, natural colors) could be expected to be as demanding. From the article: "BEEFY Graphics card. I'd really like to have more memory than the 11 GB in my GeForce 1080TI (11GB). You'll have a tough time wit…
Earlier quoted context omitted.
Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…
(I'm a moderator on HN.) Your account was being rate limited by our software. I'm sorry! We've marked it legit so this won't happen again. Please participate as much as you like.
Oh wait, not me. Got it.