> Granted, the model isn't always perfect. This one's red hand drives me nuts because it's otherwise fantastic
Plot twist: It was actually red in reality.
Colorizing and restoring old images with deep learning
41–50 of 107 posts
Re: Colorizing and restoring old images with deep learning
#42Seems like an easy problem for DL, as you have an enormous amount of data available (just take any color image, convert it to grayscale and you have a pair of training images). (This is also the case for e.g. the superresolution problem.)
You probably need an enormous GPU (24GB RAM) as well to make as large model as possible for as good generalization as you can (there are so many different types of objects/surfaces/fabric and their compositions).
Re: Colorizing and restoring old images with deep learning
#43When I saw "restoring" in the title I was expecting higher resolution. For example seeing in modern photo level detail eyelashes, wrinkles, etc. I get that, like the colors, this would require the adding lots of made up information about scene and feature details but IMO it would blur the lines between restoration and reconstruction/storytelling in a really awesome way. Old photos are cool in their own way but their…
Author here: It's early, and currently resolution is limited primarily by model size. Which drives me nuts. It's one of my top priorities to address because that would be a great improvement. Adding super-resolution to the pipeline should also be pretty easy but I want to at least output a reasonable base resolution on the photos first before I go that route.
Oh yeah to answer your question- super resolution does indeed make up details as you describe there and arguably does blur the line with restoration/story telling. But so does colorization- not all the colors added by the model are going to be what was actually going on there, of course.
Re: Colorizing and restoring old images with deep learning
#44Can someone run this against some manga?
Google "colorize manga neural" and you'll find many projects for AI-assisted manga colorization.
Re: Colorizing and restoring old images with deep learning
#45There is a HN discussion about a similar model from 2016: https://news.ycombinator.com/item?id=11403653
Re: Colorizing and restoring old images with deep learning
#46This is one of the few colorizing algorithms that I've seen which creates desirable output. The images really do look like old colorized images. I wonder how the authors dealt with the differences in spectral sensitivity of their source material. There's clearly some orthochromatic plates or film being used. The image of the Seneca native 1908 is a good example. Notice how dark the field is on the patch on her skirt.…
Author here. Easy to answer that one- altering the training photos with random lighting/contrast changes (yet keeping the color targets the same) really helped to deal with varying qualities of photos. But also neural networks are just particular good at picking up on context, so that has a lot to do with why the results are so robust.
Re: Colorizing and restoring old images with deep learning
#47Seems like an easy problem for DL, as you have an enormous amount of data available (just take any color image, convert it to grayscale and you have a pair of training images). (This is also the case for e.g. the superresolution problem.)
You probably need an enormous GPU (24GB RAM) as well to make as large model as possible for as good generalization as you can (there are so many different types of objects/surfaces/fabric and their compositions).
Something amusing about needing a ridiculously large model to claim good generalization. Analytical models typically go the other way, right?
Re: Colorizing and restoring old images with deep learning
#48The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.
Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…
Might make a good addition to the "Known Issues" section or somewhere else. Just somewhere to indicate all the little points of human intervention.
Re: Colorizing and restoring old images with deep learning
#49What happens with modern black and white photos?
Re: Colorizing and restoring old images with deep learning
#50The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.
Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…
(I'm a moderator on HN.) Your account was being rate limited by our software. I'm sorry! We've marked it legit so this won't happen again. Please participate as much as you like.